Introducing Augmented JavaScript

Today I’d like to introduce my latest project named “Augmented JavaScript” to the world.

What is Augmented JavaScript?

Augmented javascript is a javascript-preprocessor written in javascript. It’s function is similar to the more known javascript preprocessor CoffeScript, however, while CoffeScript has more or less invented a new language alltogether, Augmented JavaScript is a lot more similar to plain old JavaScript.

Currently Augmented JavaScript adds 3 new features to the JavaScript language. All of these features are new, and not very well tested, so bugs should be expected. These new features are:

  • Await
  • Yield
  • For..Of-Loops
  • Shorthand Function-Syntax

Await

The await-keyword is taken from the newer versions of C# and VB.NET. It’s job is to pause the execution of a piece of code untill new data is availible, and then continue off where it left. This results in the ability to write async functions very easily without the need for callback-nesting. For instance, let’s asume we have a “magic” function called load whos task it is to perform a simple xhr-request and callback when the data is availible. Using regular javascript one would in general write code such as this:

function login(username, password, cb) {
    getUser(username, password, function(user) {
        getGroups(user.id, function(groups) {
            cb({
                user: user,
                groups: groups
            });
        });
    });
}

function getUser(username, password, cb) {
    load('/getUser', {
        username: username,
        password: password
    }, function(data) {
        if(data.success) {
            cb(data.user);
        } else {
            cb(null);
        }
    });
}

function getGroups(userId, cb) {
    load('/getGroups/' + userId, function(data) {
        if(data.success) {
            cb(data.groups);
        } else {
            cb(null);
        }
    });
}

As you can see, there’s a lot of nesting of functions, and it’s not always easy to understand what is going to happen and in what order. People who are unused with javascript typically expect that when you’ve run the login-function, the user will be logged in immidiately, which isn’t the case because our load-function is async. Now, take a look at the same code rewritten to use Augmented JavaScript (here ofcause I assume that the load-function has been written in such a way that it is usable with the await-keyword and not just callbacks as in the previous example, more on that later).

function login(username, password) {
    var user = await getUser(username, password);
    var groups = await getGroups(user.id);
    return {
        user: user,
        groups: groups
    };
}

function getUser(username, password) {
    var data = await load('/getUser', {
        username: username,
        password: password
    });
    if(data.success) {
        return data.user;
    } else {
        return null;
    }
}

function getGroups(userId) {
    var data = await load('/getGroups' + userId);
    if(data.success) {
        return data.groups;
    } else {
        return null;
    }
}

Quite a bit easier to read, no? The result of these two snippets of code should theoretically be the same. However, as said, the probability of experiencing bugs in Augmented JavaScript is still fairly high, and I have not explicitly tested the snippets found here yet.

What is awaitable

Internally, Augmented JavaScript uses the exelent Q-library to handle waiting. That means that anything that can be understood by the Q-library, can be waited for.

Yield

The second new feature is the yield-keyword. The yield-keyword works in certain ways almost like the await keyword, with the exception that instead of waiting for whatever you pass after the yield-keyword it is returned to the calling function. The yield-keyword is simply put used to create generators (or iterators). A simple example illustrates best how it works:

function testGenerator() {
    yield 1;
    yield 2;
    yield 3;
}

function runTest() {
    var iter = testGenerator();
    console.log(iter.next()); // outputs 1
    console.log(iter.next()); // outputs 2
    console.log(iter.next()); // outputs 3
    console.log(iter.next()); // throws StopIteration
}

Using yield you can easily create functions that have no end, because only the data you request is generated. For instance, one can create a simple fibionacci-generator like this:

function fib() {
    var a = 1, b = 0;
    while(true) {
        yield a;
        b = a + b;
        yield b;
        a = a + b;
    }
}

For..Of-Loops

The for..of-loop only works on iterators or objects that have the same signature. Use for..of to loop though all the values generated from a generator. For instance, if you take the function runTest created in the previous section, you could rewrite the code to this using for..of:

function runTest() {
    for(var n of testGenerator())
        console.log(n);
}

break and continue works as one would expect, so to print all the fib-numbers less than 100 you could do this:

function printFib() {
    for(var n of fib()) {
        if(n < 100) break;
        console.log(n);
    }
}

Shorthand Function-Syntax

The shorthand function-syntax is fairly straight forward. The valid forms are as follow (all of these are expressions):

1. <identifier> => <expression>
2. <identifier> => <statement>
3. (<arguments>) => <expression>
4. (<arguments>) => <statement>

<arguments> is a comma-separated array of 0 or more identifiers. If any of the variants that ends in an expression is used, said expression is automatically transformed into a returnstatement. This allow for simple map-functions to be written a lot shorter than in regular JavaScript. Here are some samples:

var add = (a, b) => a + b;
function makeFunction(itm) {
    return () => itm;
}
load(data => {
    // do stuff with data
});

Yielding results – the process involved with generating generators

One of the nice things about the language C# is it’s ability to create generators with the yield keyword. Now, for those of you who do not know what I’m talking about, take a look at the following code:

public IEnumerable<int> MakeGenerator(int num0)
{
    yield return num0;    

    int i = 0;
    Console.WriteLine("yield 0");
    yield return 0;
    try
    {
        i += 1;
        Console.WriteLine("yield 1");
        yield return i;
    }
    catch {}
    Console.WriteLine("end");
}

The code above will return a IEnumerable, and will not do anything else, until the IEnumerable is used. In other words, if I invoke MakeGenerator, none of the code I wrote will actually be run. Now, this might sound completely ridiculous, but I assure you, there is a good reason for this. To explain it as simple as possible, consider the following code, what would you guess the output to be like?

foreach(int num in MakeGenerator(5))
{
    Console.WriteLine("Got " + num);
}

The answer might surprise you. If you run the code above, you will get 6 lines of output, in the following order:

Got 5
yield 0
Got 0
yield 1
Got 1
end

The reason for this is that first when GetNext on the enumerator which the foreach-loop get’s from the IEnumerable is called, the code in the MakeGenerator-method is actually run. However, as you can see, not all of it is run. When we hit any of the yield-statements, the method is returned from. Now, this is a construct that’s actually quite useful in a lot of cases, cause for once, it enables lazy evaluation of your code over enumerables. LINQ for once, relies heavily on this (note; I do not know if LINQ uses yields or customarily created enumerators, but they both facilitate lazy evaluation over enumerables). For instance, if you have a list of a million elements (called myList), and you do myList.Select(CpuHeavyMethod), this will not fry your CPU into oblivion, rather, it will not run the CpuHeavyMethod at all! First when you actually have need for an element passed through the CpuHeavyMethod it will be called.

For those of you who still do not know what a generator is, you can learn more here: http://en.wikipedia.org/wiki/Generator_(computer_programming)

Creating generators

Now that (hopefully) all of you have a basic understanding of what generators are, and how they work, let’s look at how to make them. In other words, let’s look at what the compiler does with the yield keyword. Now, before I start, I’d just like to say that this is most likely different from language to language, and this may or may not be the way C# handles compiling generators (I’ve not looked at how the C#/mono compiler solves the problem, what I’ve read is how the IronPython compiler/interpreter does it. However, I’ve also changed that a bit to simplify).

As it turns out, generating generators is actually rather simple, there are just a few things you have to look out for:

  • Try statements (with or without finally) must be handled separately (and with a little bit of caution).
  • There cannot be yield-statements in catch or finally-blocks.

Note: these are rules I’ve set for the generator I create. It may be possible to have yields in both catch and finally in C#, though it would require further rewriting of the method, and will not be discussed in this article.

The first thing we need to do (in C# anyways) is to create 2 classes. One for the IEnumerable, and one for the IEnumerator. Also, these two classes will be nested, to enable the inner one to access the outer one’s private variables.

Let’s start with these two completely empty classes (save for the interface-implementations, and _state and _current values which I’ll get back to later).

class MakeGenerator_Generator : IEnumerable<int>
{
	IEnumerator<int> IEnumerable<int>.GetEnumerator()
	{
		return new MakeGenerator_Enumerator(this);
	}

	IEnumerator IEnumerable.GetEnumerator()
	{
		return new MakeGenerator_Enumerator(this);
	}

	class MakeGenerator_Enumerator : IEnumerator<int>
	{
		private readonly MakeGenerator_Generator _gen;
		private int _state = -1;
		private int _current;

		public MakeGenerator_Enumerator(MakeGenerator_Generator gen)
		{
			_gen = gen;
		}

		int IEnumerator<int>.Current
		{
			get { return _current; }
		}

		object IEnumerator.Current
		{
			get { return _current; }
		}

		bool IEnumerator.MoveNext()
		{
			throw new NotImplementedException();
		}

		void IEnumerator.Reset()
		{
			throw new NotImplementedException();
		}

		void IDisposable.Dispose()
		{
			throw new NotImplementedException();
		}
	}
}

The first thing we need to do is to take all the method-parameters given to the MakeGenerator method and make them class-level readonly members of the Generator-class. Then we need to add a constructor that takes the same number of arguments, and saves them to the class members.

class MakeGenerator_Generator : IEnumerable<int>
{
	readonly int param_num0;

	public MakeGenerator_Generator(int num0)
	{
		param_num0 = num0;
	}

Then we need to copy those to private members of the Enumerator-class during it’s Reset() call, and lastly we need to call Reset() from the constructor like this:

	class MakeGenerator_Enumerator : IEnumerator<int>
	{
		private readonly MakeGenerator_Generator _gen;
		private int _state;
		private int _current;
		private int val_num0;

		public MakeGenerator_Enumerator(MakeGenerator_Generator gen)
		{
			_gen = gen;
			((IEnumerator)this).Reset();
		}

		void IEnumerator.Reset()
		{
			val_num0 = _gen.param_num0;
		}

The next thing we need to do is to copy the entire content of the original method into the MoveNext method of the Enumerator (note, this will make for invalid code, but we’ll fix that in a bit). The result should look like this:

		bool IEnumerator.MoveNext()
		{
			yield return num0;

			int i = 0;
			Console.WriteLine("yield 0");
			yield return 0;
			try
			{
				i += 1;
				Console.WriteLine("yield 1");
				yield return i;
			}
			catch {}
			Console.WriteLine("end");
		}

Now that we have the base-foundation of the generator, we can start editing the original code to make it work as a generator (in other words, rewrite it to a proper MoveNext method).
What needs to be done in the MoveNext methods is two things for startes. First; all variables needs to be hoisted out to class-level variables, and second, we need to enumerate the yield-statements (starting from 1). Than you change the nth yield-statement into this:

_state = <n>;
_current = <yield_value>;
return;
yield_label_<n>:

The result of doing those two actions looks like this:

		bool IEnumerator.MoveNext()
		{
			_state = 1;
			_current = val_num0;
			return;
			yield_label_1:;

			val_i = 0;
			Console.WriteLine("yield 0");

			_state = 2;
			_current = 0;
			return;
			yield_label_2:;

			try
			{
				val_i += 1;
				Console.WriteLine("yield 1");

				_state = 3;
				_current = i;
				return;
				yield_label_3:;
			}
			catch {}
			Console.WriteLine("end");
		}

For those of you who have never seen a label in C# before, this code might look a bit confusing. A label in C# is created by typing a name followed by a colon, like “yield_label_0:” The use of a label is with goto, where you jump to a label. The reason I’ve added a semicolon after the label, is simply to add an empty statement you jump to. This would not strictly be necessary, however, if I do not the yield_label_3 would be invalid (as you cannot have a label that goes “off a cliff”, ie. out of a method, if, while, block, etc.). The next part is simply to add a switch over the state in the beginning of the MoveNext method, that jumps to the right continuation. Also, we need to add the final state (_state == 0, finnished). The result looks like this:

		bool IEnumerator.MoveNext()
		{
			switch(_state)
			{
				case 1: goto yield_label_1;
				case 2: goto yield_label_2;
				case 3: goto yield_label_3;
				case 0: goto yield_label_0;
			}

			_state = 1;
			_current = val_num0;
			return;
			yield_label_1:;

			val_i = 0;
			Console.WriteLine("yield 0");

			_state = 2;
			_current = 0;
			return;
			yield_label_2:;

			try
			{
				val_i += 1;
				Console.WriteLine("yield 1");

				_state = 3;
				_current = i;
				return;
				yield_label_3:;
			}
			catch {}
			Console.WriteLine("end");

			_state = 0;
			yield_label_0:;
			return _state != 0;
		}

This is basically a working generator. The two first values would work just fine if we simply left it like this (and removed the try), however, since we do have a try (cause I wanted to show how to solve this), this is what we do.

  1. Start with the outermost try (only do this if the try contain yield-statements).
  2. Create a new label in front of the try.
  3. Rewrite all goto-statements that point to a yield inside the try so they point to the new label instead.
  4. Rewrite the try so it jumps based on state.

In other words, it goes like this:

switch(_state)
{
	case 1: goto yield_label_1;
	case 2: goto yield_label_2;
	case 3: goto try_label_1; // 3. Rewrite goto-statement to point to start of try
	case 0: goto yield_label_0;
}

_state = 1;
_current = val_num0;
return;
yield_label_1:;

val_i = 0;
Console.WriteLine("yield 0");

_state = 2;
_current = 0;
return;
yield_label_2:;

try_label_1:; // 2. create label in front of the try
try
{
	switch(_state) // 4. Rewrite the try so it jumps based on state
	{
		case 3: goto yield_label_3;
	}

	val_i += 1;
	Console.WriteLine("yield 1");

	_state = 3;
	_current = i;
	return;
	yield_label_3:;
}
catch {}
Console.WriteLine("end");

_state = 0;
yield_label_0:;
return _state != 0;

And there you have it. We’ve made ourselves a generator. The only thing that needs to be fixed now is to remove the NotImplementedException inside the Dispose-method. What you should probably do inside the Dispose-method is set the _gen-variable to null, and check for it being null in both MoveNext and Current (and if it is, throw a InvalidOperationException because it’s already disposed), however, I’ll leave that up to you guys. Also, I was planing on showing how to create Generators with the DLR, though it seems this post became longer than I planned it, and if I were to start explaining the DLR stuff now, I’d probably more than double the length, so that will come later. I hope this made it easier to understand how generators work, and what the compiler does with them.

Until next time.
- Alxandr

Await anything – A tale about asynchronity

As most people who do any .NET related programming probably know, a short while ago, the next major version of the .NET framework and the tools that goes with it was released. This includes (but may not be limited to) Visual Studio 2012, .NET 4.5 and C# 4.5. One of the major new features of the .NET framework is its new support for async programming. Now, async programming has always (or rather, for as long as I can remember having wanted to do anything async) existed. For instance, if you create a Socket-instance, you have “BeginReceive” and “EndReceive” which provides asynchronous capability, however using them can be cumbersome. Normally you need a class to keep track of state, or you end up with a bunch of closures (given that you have .NET which is new enough to support closures). What this new capability does is to make it easy to create asynchronous code, and it generally does so with the help of Tasks.

Now, Tasks (System.Threading.Tasks.Task) have existed in .NET for a while, and I’ve used them for a while, cause I find they provide a simpler way of chaining calls one after another. For me, which have done most of my work in javascript, the ContinueWith<TIn, TResult>(Func<Task<TIn>, TResult>) feels natural, and easy to use. Handling completion-events is much more cumbersome for me. However, with .NET 4.5 the use of Tasks has been simplified even more, and of-cause I’m talking about the async and await keywords. However, I’m not going to explain how to create async functions, how these functions return Tasks, and what are great about them, cause there are probably already a thousand other post explaining this, what I want to look at more in-depth is the await keyword itself.

Awaiters

My quest for answers with regards to the await-keywords starts with the new release of Reactive Extensions. Reactive Extensions mainly does it’s work with a single interface; “IObservable<T>”. Now, because this is an interface, there is no way (that I know of) it could implement Task<T>. Neither, does (as far as I know) Task<T> implement IObservable<T>, meaning there is no connection between them what so ever. Here you might probably wonder why I’m stressing this out, and the reason for that is rather simple. While writing code working with IObservable<T>, Visual Studio kept telling me how IObservable<T> was awaitable! In other words, I could write await in front of any instance of IObservable<T>!
How could this be? Certainly, as I just explained, there is no connection between IObservable<T> and Task<T>. The answer, has to do with awaiters.

Awaiters, as I found after some googling, turns out to be one of the most curious compiler-patterns I’ve ever seen. From what I understand, for a type (lets call the instance myWait for now) to be awaitable, the following must be true:

  1. myWait.GetAwaiter() must be a valid expression
  2. myWait.GetAwaiter().IsCompleted must be a bool-property that is gettable, and returns a value indicating weather or not myWait is completed.
  3. myWait.GetAwaiter().GetResult() must be a valid expression, and must be a method that returns the result of the async operation, in other words, the result of “(await myWait)”.
  4. myWait.GetAwaiter().OnCompleted(Action continuation) must be a valid method that takes a single Action and quest that method for execution when the async operation is finnished.

Notice that I’ve stated that myWait.GetAwaiter must be a valid expression? What I find curious about this, is that it must be a valid expression for the compiler, not a valid method of myWait. In other words, GetAwaiter can be an extension method! This makes it possible to do some rather absurd things with await like await a TimeSpan (to delay the current execution), given that you provide a one-line extension-method for TimeSpan. More about this can be found here: http://blogs.msdn.com/b/pfxteam/archive/2011/01/13/10115642.aspx

SpotiFire

Last in this article I’d like to say a few words about a small project of mine, named SpotiFire. SpotiFire is a wrapper around the libspotify API’s, that makes it fairly easy for a .NET application to interact with spotify. The last update to SpotiFire (uploaded today) enables the use of async for a lot of operations. Here is a simple sample showing how easy some of the operations that used to be hard to do is:

Push to PC – A cry for attention

Currently I’m registering about 2 paradigmatic changes going on in this PC world, and they have been going on for a while. One of them is that everything should be moved to the web. Or the cloud if you like. This includes the notion of storing all data in the cloud, but also includes the process of having everything run in your browser. In 10-15 years, it might be that everything except for the browsers are coded in html and JavaScript, but I’m not speculating about that right now.

The second paradigmatic change is that everything should be considered a standalone app. Even though it’s simply a browser page (dearly note that these aren’t necessarily simple anymore). This point also includes the fusion (or partial fusion) of mobile (or at least tablet) and pc, a notion all major operating-systems are currently moving towards. From Microsoft you have Windows 8 (and WindowsRT, where the later one is actually a tablet-only OS, and the former one is WinRT + Win7 more or less), and on the OSX side of the world you have a bunch of iOS features being integrated into OSX.

There are a lot of complaints to Microsoft regarding this shift in windows, however I’m not here to complain about this either. Personally I find windows 8 to be the best windows so far, and the windows RT experience stays out of your way if you don’t want to use it.

Web-apps as first class applications

One of the things I do frequently on my computer (currently running windows 8 RP) is to check mail. The obvious solution to this problem is to use the “Mail”-app provided with Windows 8. It looks like this:

Now, if you ask me, this is a fairly nice looking, and functional application, however, the main problem about this application is that it’s made for hotmail/exchange. That does not mean it will not work with my gmail-account, far from it, but it means that it does things slightly differently (also, as a side note, it crashes on start on this computer, but it is a preview after all, so I’m guessing this will have been resolved in the release of windows 8). These slight differences are annoying, because my handling of email (how I put them in folders, when I archive them, etc.) is built arround how Gmail handles email. This means that in order to get things “my way” I either need to use a different client, or I simply need to use the browser version, and for me this is a simple choice.

Browser-based applications have pros and cons with regards to full-fledged apps, and a lot of the cons are attempted being removed with the use of new html-standards (like offline caching), and javascript-standards (like local database, filesystem etc.). Other cons are being removed by the browser-vendors, like chrome’s possibility to make “program-shortcuts” to web-pages (or web-apps if you like). Now, these are good steps along the way to make the browser-based Gmail-client function just as good as the built-in email-client in windows 8, however there is one thing that the built-in client does infinitely better than the browser-based client.

You have mail

As you can probably see, this isn’t actually a mail-notification (as I can’t get my Mail-application working, I can’t screen-shot that, nor could I find one on google images), but it’s a notification non the less. In fact, the newest version of all major operating systems (counting windows 8 as the newest version of the windows family) have support for some sort of native notifications. And any email-application that comes with the system is bound to take advantage of that. This means that if I simply booted up my pc (without starting a single application), and somebody sent me an email, I would get a notification in my face.

For those of you who use email on a “smart-phone” (windows phone 7+, android or iOS) this should be very familiar, because these kinds of notifications have been around on our cell-phones for several years already. You shouldn’t have to actually open your email-application to check if there are new emails, the email-application should tell you when there are. This is definitely the case with the built-in windows 8 email app, however, it is not the case with GMail running in my browser.

Platform support

Currently, a lot of platforms however (or rather, the platform of a lot of users, meaning windows < 8) do not support native notifications. Still, this is fairly simple to achieve, using a popup-window that contains what would otherwise be served by a native notification, when there is no way of doing it natively. On OSX prior to mountain-lion this can be achieved using Growl for instance. Also, the newest version of Growl (Growl 2, still in beta currently) supports the notification-center on OSX, meaning that on OSX everything could potentially be handled by the Growl-framework.

I’m not familiar enough with the different linux-distros out there to begin guessing how to handle something like this on Linux, so I will not say too much about it, however, to my knowledge, Ubuntu Linux has had a built-in growl-like notification-utility the last few years which could probably be utilized without much further ado.

API and protocol

What I am proposing is that there should be made a standard API for enabling cross platform, native (if provided by the platform) push notifications that can be subscribed to using simple JavaScript. also, there need to be made a standard protocol for pushing notifications to the clients.

However, who am I to propose such a huge demanding thing to be made and standardized? The sad truth is that when it comes to things like this, I am nobody. I am but a simple, single voice in the huge internet, and that is why I’m writing this post, in hope that someone bigger than me will see it and think that my reasoning and proposal is somewhat fair, and that it in time will get picked up.

Still, even though I am far from qualified to undertake such a huge task on my own (cause it is a huge task), I have been thinking and drafting on this “specification” for a while, and thought I’d air my thoughts here, to show more or less what I imagine the functionality should/could be like.

The API

The first part of the specification needed is the client API (JavaScript API) to register for push-notifications. This should be simple, yet secure, and should conform to the same-origin policy for safety. I’m suggesting 2 simple properties and a function should be added to the client to facilitate push-notifications.

window.pushNotifications = {
  // state property, read only.
  // should return one of the following values:
  //   0: pushNotifications not registered
  //   1: attempting registering for push notifications
  //   2: pushNotifications registered

  state: readonly property,
  // onstatechange property, defaults to undefined
  // if function, called whenever state is changed
  onstatechange: readWrite property

  // register, function
  // takes a single parameter, which is a callback-function
  // callback-function is called when registering is done
  // independent on whether or not the registering succeeded
  // the callback-function takes a single argument, which
  // on failure is { success: false, error: Error-object },
  // and on success is { success: true, endpoint: push-endpoint }
  // push-endpoint is a DOMString. Described later.
  register: function(callback) {}
}

When register is called, if push-notifications are already registered, it should simply run the callback with the previously registered endpoint. If push-notifications are not already registered the user should be presented with a choice (similar to the choices to enable location and other such API’s) before anything else happens. If the user disapproves that the web-page can send push-notifications, the callback should be called with an error message stating that the user disapproved, else it should call up to the notification-server to register for an end-point. When the response is returned the notification-endpoint should be given to the callback function. Once the client has hold of a notification-endpoint, it should store this on its server.

The protocol

The protocol is unfortunately a bit more complex than the client API, but should still be fairly simple. All calls to the notification-server should be done over https. When register for a notification-endpoint, 3 peaces of data should be included. A client-id to identify the client, the domain of the web-page running the register-call and whether or not the domain uses https. The notification-server should then store these, and generate a unique notification-endpoint https url. This url should then be returned to the client. Note that the client-id does not necessarily have to be unique for the client, but might instead be unique for the user, but should definitely not be unique for the machine. Two users on the same machine should not receive each-others push-notifications. However, since they can be unique per user, this also means that the same unique identifier might be used across several machines where the same user is authenticated, meaning that all the user has to do is log in to some sort of synchronization-environment in his browser, and he would receive push-notifications that he registered for elsewhere.

After a user is registered for push-notifications, the server can begin sending them. To make sure that the push-notifications are as similar as they can be between all platforms, the information being sent should be limited to 2 strings (a title and a details string), a url to an image, and a url that should be opened when the user clicks the notification. This should be sent to the server using json serialization of the data. These are some examples of valid push-notifications:

{ "title": "New email" }
{ "title": "New email from test@example.com",
"description": "Subject of said email" }
{ "title": "New email from test@example.com",
"description": "Subject of said email",
"url": "http://my-email.example.com/mail/id_of_email" }
{ "title": "New email from test@example.com",
"description": "Subject of said email",
"url": "http://my-email.example.com/mail/id_of_email",
"icon": "http://my-email.example.com/contact_example/profile_picture.jpg" }

The only information that is required is the title. If no url is provided than the domain-name registered should be used as an url (in this case “http://my-email.example.com/&#8221; if the domain does not use https). If no icon is provided, the favicon of the website should be used. Title and description should both have a max-length, title of 255, description of twice that. The server needs to make sure that any push-notifications sent is sent from the same domain that registered for the end-point. This is to prevent spamming of push-notifications if a third-party at some point got a hold of the push-endpoint. However, it should be possible to enable other servers to send push-notificaitons (in cases of server-farms for instance). I’m proposing using DNS to enable this, with similar behaviour as the SPF records used for email. However, here is one of the points where I most certainly am not qualified to speculate on the best way to resolve this problem, thus I will not write any more about it.

The plea

This concludes my description of a protocol and an API I wish existed, and my plea is simple; if you too wish that something like this existed, spread the word, so that in time, someone who knows where to start with something like this might pick it up. And if you have any comments or questions please leave a comment below. And I hope, that some day, in a not too distant future, this can be made into a reality.

Thanks for reading, and please spread the word.

CKEditor and all it’s merrits (and the likes)

I’ve been working a lot with CKEditor the last couple of days, trying to upgrade our website from an old and probably fairly outdated FCKEditor. The funny part about FCKEditor is that at first everything feels like a breeze. I think the time it took me to replace the old FCKEditor with something that looked like it was working was about 2-5 minutes, and to get it actually working was only a couple of minutes more. However, the joyride down to wonderland (while wearing a grin saying “OMG; this is so easy, and the result looks so nice!”) kinda ends there. If any of you have worked (extensively) with CKEditor you will probably know that it’s made to be highly customizable, with lots of settings and stuff for you to play with. However, the problem arises once you try to do something you weren’t supposed to. For instance; on one of the things we didn’t want on our page was to set the default cellspacing and padding on tables to "" (empty string; ie. remove the attributes). This sounds like something that should be fairly easy to do (and probably would be too if I didn’t use the minified code). At first I looked for settings that would enable me to set this, however I was unable to find any, so what I ended up doing was to change '1' in plugins/table/dialog/table.js associated with the cellspacing and the cellpadding fields to ''.  That however, was the easy part. The next thing that was requested was to by default mark tables with table-layout: fixed. Now to do so turned out to be harder than I thought it would be. And just to make sure I’ve said it, this doesn’t necessarily mean it’s the only way to achieve this result in CKEditor, nor is it necessarily the right way to do it, but it was the first and quickest way that I managed to find (without having to write my own table-plugin for CKEditor). What, however, worked for me was locating if(!this._.selectedElement){var p=l.append(h('tbody')) in the existing table.js plugin, and then adding l.setStyle('table-layout','fixed');. Also, if you would (by default) like to have a custom class on all tables created with the table-plugin, here is the place to add it. Simpy do l.addClass('class-name-here'); in the same if-statement.

Filemanager

One of the not so fun things about the “new” CKEditor (with regards to the “old” FCKEditor) is that it was split in two. Now you have CKEditor and CKFinder. However, the story doesn’t quite end there, cause CKFinder is a commersial tool that costs money, and if you’re just working on some pet project you don’t want to kough up a couple of hundreds of dollars just to be able to add files to your user-content. While, however, this is one of the things I consider bad about the new CKEditor, it must be said that the opening of the CKFinder is an extension-point in CKEditor, meaning that you just give it an URL of something that works like CKFinder, and CKEditor will be happy as a newborn child. The answer, ofcause (like all other things in life) is to make it yourself. Or rather, get someone else to make it for you, that doesn’t want to (or at least doesn’t require) to get paid, but that’s the beauty of open-source, right? Everybody contributes their part, and in the end there is awesomeness. However, getting back from my quick digression, I found something called simply “filemanager” (meaning it’s impossible to google for). It’s apparently made by a company named c5 (and pull-requests from whoever wanted), and it can be found somewhere around here, it’s github-repository is found here.

The original filemanager looks like this:

If you ask me, I think it’s rather pretty made, and like the design, but I was asked to simplify it a bit (remove “unneeded” elements). A couple of the things that wasn’t wanted was the “Current folder” up in the left corner, the home button, the new folder button, and also, the upload button. This is not to say that I should disable file-upload. I should simply make it so that whenever a file is selected by clicking the “browse”-button, instead of having to click upload afterwords, the form should automatically be submitted. In jQuery (which the plugin uses) this is as simple as $('#newfile').change(function() { $('#upload').click(); }). Also, I had to provide my own translation, cause a norwegian translation was not present, but as I only needed some of the features, only some of the features was translated. At last, the “dimensions”-column should be removed.

The resulting filemanager looks like this:

One of the things to notice is that the browse-button is moved to the upper left corner and is localized. This localization is not the browser that renames the browse-button, it’s a customarily made button (that doesn’t require any flash-upload or the likes, however that’s a feature I would like to implement at a later point given the chance). The html to achieve this looks like this:

Once again, I’ve uglified the file a bit, however it was simply the fastest way of achieving the desired result. The file-upload “hack” I’ve provided should work in all modern browsers (safe for Opera which I have never bothered tested with as our whole site generally don’t work in Opera at all it’s not a requirement). A general improvement over the initial version I started with (if you ask me) is the fact that once I upload a file (click the browse-button and select a file), the file is also automatically selected and thrown back into CKEditor. Some users might not want this, however, for our usage we concluded that whenever a user clicks the browse-button, they’ve been sent here from CKEditor to select a file, and they’ve concluded that the file needs to be uploaded first. There should be no point in uploading a file simply to upload it.

One of the other things I fixed was the date-column (shown as “Endret” in norwegian). With the original code it would show dates sent from the server as text (I would get the string “25.06.20012″ from the server). Now this doesn’t seem to bad until you start sorting by date. Just think about it for a second; since year isn’t first (like in YYYY-MM-DD) the sorting will fail. This might not be the case for american dates, cause I have a suspicion that the fancy table-sorter included in Filemanager could handle those, however, for anything else it would fail horribly. Also, Filemanager contained no way of changing how the dates are shown (cause they were just strings sent from the server).

What I did was simply to send the dates as UNIX timestamps from the server, than read those in as date-objects in javascript. And since it’s just numbers the sorting works magnificently too. The code for handling timestamps sent from the server looks like this:

This of cause outputs the date in Norwegian format, however, adding formatting that could be modified in config was not something that I had to do, as our instance of Filemanager will simply be used in Norwegian. If one wanted to globalize, the simplest thing would probably be to include something like datejs and setting the format in lang-files.

There were a few other minor modifications I did (like remove a column, make clicks on files automatically select the file instead of go to an info-screen etc.), and all of them can be found in my filemanager.js-file. Also, most of the changes I did can be toggled on and of in the settings-file by setting new settings I made. I’m not posting my entire config-file (cause it had to be heavily modified to enable our scenario with regards to what folders should be treated at root depending on a lot of things), but most of my changes should be found here. Feel free to use it however you like.

Counting file-lines in bash

I’ve been wanting to know how many lines of code a given project has, so I wrote a short bash-script that counted the lines of all files ending with .cs in bash. Later, I figured that I could extend this bash-script so that it could count lines in all files specified, and this is what I came up with:

This can probably be made both faster and better, but as it’s my first bash-script I think I’m ok with the way it is now. To use the script simply call it with the suffixes of the files you want to find. Here’s an example run:

Oh, and by the way, this also works on windows in msysgit.

Binders in the DLR

I’ve been (somewhat) continuing my work on Totem on the DLR lately (and not much, given that I’m in the middle of my finals currently). However, since it’s been so long since my last post I thought I’d do a quick one right now :) . The DLR has given me a lot of troubles trying to figure out how stuff fit together. Examples are either way too complex, way to simple or way to narrow to figure out how to solve problems that arises, and how to structure the project is also a hassle. However, having achieved some progress, I just thought I’d leave this here: