Dynamic Dispatch – How dynamic work in C#

This post was inspired by this Stack Overflow question.

Dynamic Dispatch

Let’s start at the beginning. What is a dynamic dispatch?

In computer sciencedynamic dispatch is the process of selecting which implementation of a polymorphic operation (method or function) to call at runtime. Dynamic dispatch contrasts with static dispatch in which the implementation of a polymorphic operation is selected at compile-time. The purpose of dynamic dispatch is to support cases where the appropriate implementation of a polymorphic operation can’t be determined at compile time because it depends on the runtime type of one or more actual parameters to the operation. – Wikipedia

In C# there are several ways of making calls that would qualify as being a dynamic dispatch according to the Wikipedia documentation (or my understanding of it), however today we’re going to look at a specific one. We’re going to look at operations involving C#’s “dynamic” keyword/type. However, before we get to that, let’s start of simple:

string obj = "test";
int length = obj.Length;

Now, before people start screaming about there being nothing dynamic about this code, I already know that. This is the basic, statically-compiled code that we’re going to work with to explorer how dynamic actually works. But to understand how dynamic works, it’s imperative to understand how non-dynamic (ie. static) works. When the following code get’s compiled (inside a method of cause, otherwise it wouldn’t be valid), it creates a string-variable named obj and puts “test” into said variable. Then it creates a int-variable named length, and it puts the result of calling the getter of the Length-property on string on obj. This can be done efficiently because the compiler knows (at compile time) that obj is indeed a string, and it knows how to resolve Length on a string. In other words, what happens (pseudo-code) is more or less this:

string obj;
int length;
obj = (string)"test";
length = (int)string::get_Length((string)obj);

Note; the explicit casts are just to illustrate the types of things, not to be interpreted as actual casts. There’s nothing really interesting about this code, so let’s see what happens when we throw a “dynamic” in there.

Dynamic Call Sites

If we “spice up” the code a little by changing both variables to be of type “dynamic”, like this:

dynamic obj = "test";
dynamic length = obj.Length;

and then decompile the result, the code looks quite different from what I stated above. What you end up with is something like this:

object obj = "test";
CallSite<Func<CallSite, Object, Object>> callSite =
    CallSite<Func<CallSite, Object, Object>>
        .Create(Binder.GetMember(CSharpBinderFlags.None, "Length", typeof(Program),
            new CSharpArgumentInfo[] { CSharpArgumentInfo.Create(CSharpArgumentInfoFlags.None, null) }));
object length = callSite.Target(callSite, obj);

Note that this code has been slightly simplified.
Now, there is a lot of information about our dynamic call generated from the simple line of obj.Length. The first thing to notice, is that a “Binder” is created (by the “Binder.GetMember” method). GetMember indicates that we want a GetMemberBinder, which is used to access members of types, and as we all knowns, a property is a kind of member. The binder needs information about what property we want to access (“Length” in this case), and what arguments we want to supply to the operation (only one argument in our case). It also takes some binding-flags (which is just set to None), and the encapsulating type (which in my case is Program). Now, you might be wondering, what’s that one argument? The property takes no arguments, so having one argument looks kind of strange. Well, if you look at the pseudo-code in the previous section, you’ll see that what happens when you call a property (or any member-method for that sake), the object on which you call the property/method is passed in as the first (zeroth) parameter behind the scenes. This than get’s access through the this-keyword. So the parameter we need for our dynamic property-method is the object on which we want to find the property (in our case, the obj object).

After a binder has been constructed, it’s fed into a CallSite<Func>. Now this callsite looks scary, but it’s actually quite simple. The callsite provides some logic to cache a bound operation (bound by the binder, we’ll get back to that later), so that if you give it the same type of object multiple times in succession, it’ll just use the same function it used last time, and it’ll be almost as fast as if you ran without using dynamic at all! With exception of the first run ofcause, where it has to do the actual binding.

Dynamic Binders

If the callsite itself has not cached a method (target) on how to perform the dynamic action (or if the cached action does not fit with the given type), it asks the binder to do so for it. The binder also maintains (a larger) cache of operations it has already bound, and if any these are a match it uses it, if else it performs a new binding. This is where things start to get interesting. The callsite calls “Bind” on the binder (which is overridden and sealed in DynamicMetaObjectBinder). The Bind-method creates DynamicMetaObjects for each of the given arguments (which in our case is only 1), it then executes another overload of Bind which takes DynamicMetaObjects, which in GetMemberBinder get’s rerouted to a call to the first DynamicMetaObject’s BindGetMember (we’ll look at this in a bit). The default DynamicMetaObject simply calls into the binder’s FallbackGetMember-method. At a glance, this is more or less what happens:

1. we call callsite.Target(callsite, obj);
2. callsite realizes it has no cache (or an invalid one), and calls update
3. DynamicMetaObjectBinder::Bind(object[] args, Expression[] expressions) is invoked
4. DynamicMetaObject is created for each arg, and GetMemberBinder::Bind(DynamicMetaObject target, DynamicMetaObject[] args) is invoked
5. GetMemberBinder invokes target.BindGetMember(this /*binder*/);
6. DynamicMetaObject (target) invokes binder.FallbackGetMember(this /*target*/);
7. GetMemberBinder invokes this.FallbackGetMember(target, null /*errorSuggestion*/)
8. CSharpGetMemberBinder uses some black magic (reflection) to figure out that "Length" corresponds to string.Length property.
9. A function (looking like (string str) => str.Length;) is returned to the callsite (through the chain, and some caching).

The important thing to notice here is the fact that on line 5, BindGetMember is invoked on the actual DynamicMetaObject representing the target (in our case the string named obj). This means that objects that implement IDynamicMetaDataProvider has the ability to override the behavior that happens when they are involved in a dynamic dispatch. However, since string does not implement IDynamicMetaDataProvider the default DynamicMetaObject is used, which simply calls back into the binder to let it decide what happens. This means that if you switch out the binder, you’d get different behavior for all objects that do not implement IDynamicMetaObjectProvider (and for those that chose not to change the default behavior). This is the reason why C#’s compiler generates a CSharpGetMemberBinder, while VB would probably generate a VBGetMemberBinder (or something like that), to make sure the language’s semantics (like the fact that VB is case independent) are maintained in dynamic operations.

DynamicObject

The .NET framework provides us with a simple utility-class if one wants to create a dynamic object (instead of going through the whole IDynamicMetaObject and custom DynamicMetaObject approach). This is called DynamicObject. So let’s go ahead and implement our own (very simple, mind you) dynamic type.

class CustomString : DynamicObject
{
    private readonly string _value;

    public CustomString(string <span class="hiddenGrammarError" pre="">value)
    {
        _value</span> = value;
    }

    public override bool TryGetMember(GetMemberBinder binder, out object result)
    {
        if (binder.Name.Equals("Length", binder.IgnoreCase ? StringComparison.OrdinalIgnoreCase : StringComparison.Ordinal))
        {
            result = _value.Length;
            return true;
        }

        return base.TryGetMember(binder, out result);
    }
}

DynamicObject implements IDynamicMetaObjectProvider, and it returns a custom DynamicMetaObject which overrides all the Bind*-methods. What it does however, is quite simple, yet also quite brilliant. The BindMember method of the hidden MetaDynamic inner class on DynamicMetaObject simply invokes TryGetMember on the DynamicMetaObject. Now, I say simply, but there’s a good bit of logic to handle cases where you haven’t overridden TryGetMember, and if you return false, or an exception is thrown, etc. However, the general gist is that TryGetMember is invoked. This means that if we do this:

dynamic obj = new CustomString("test");
dynamic length = obj.Length;

our override of TryGetMember will be invoked. The binder will also have it’s Name set to “Length”, so as you might expect we would get the int 4 back, just as with the normal string. This actually makes it possible to make CustomString work like string in almost all cases if we were to override all the required methods in DynamicObject, however, I’ll leave that as an exercise to the reader.

I hope by now you have gotten a better understanding of how dynamic dispatch works in C#. I will not go into implementing a custom DynamicMetaObject, because that’s worthy a whole blog-post for itself. Maybe another time. If there are any questions, please leave them in the comments.

Until next time.

Implementing continuations in JavaScript

In computer science and computer programming, a continuation is an abstract representation of the control state of a computer program. A continuation reifies the program control state, i.e. the continuation is a data structure that represents the computational process at a given point in the process’ execution; the created data structure can be accessed by the programming language, instead of being hidden in the runtime environment. Continuations are useful for encoding other control mechanisms in programming languages such as exceptions, generators, coroutines, and so on.

- Wikipedia

Continuations is a fairly complex construct used in some programming-languages to change the flow of the program. For instance, imagine the following program:

function getA() {
    return 5;
}

function getB() {
    return 10;
}

var a = getA();
var b = getB();
var c = a + b;
alert(c);

If this was run on a regular JavaScript runtime, it’s fairly easy to imagine how the code would execute, but just to make sure let’s walk the execution step by step (ignoring method-detection/registering).

1. Execute getA();
2. assign 5 to return-field.
3. assign return-field to a.
4. Execute getB();
5. assign 10 to return-field.
6. assign return-field to b.
7. calculate a + b and assign to return-field.
8. assign return-field to c.
9. alert c.

Now, this explanation is dumbed down and in some cases do not explain well how some things are done. For instance, the “return-field” is normally only the top of some stack, and there’s all sort of framing and other stuff going on with regards to the function-calls, but this is enough to illustrate my point. And the point is that the code basically executes like this:

1. Run getA();
2. Run getB();
3. Addition
4. Alert

However, in other languages, this might not hold true. In some languages, things are (explicitly or implicitly) not computed until they are needed. In such cases, the execution might look like this:

1. Set a to a lazy computed getA result.
2. Set b to a lazy computed getB result.
3. Request a (at this point, the value of a get's computed and set to 5).
4. Request b (at this point, the value of b get's computed and set to 10).
5. Addition
6. Alert

Now, if we run this on a system using a simple construct like Lazy with polling, this example holds little value, however, if you run this on a system using scheduling (and lazy is implemented using said scheduling), things become a lot more interesting. These values (a and b) might for instance be computed on a completely different thread altogether. To illustrate how this might be executed I’ll show a javascript-example that does roughly the same (as how this might have been implemented on some systems).

var a = function() {
    a = 10;
};
var b = function() {
    b = 5;
};
var rest = function() {
    var c = a + b;
    alert(c);
};
setTimeout(a, 0);
setTimeout(b, 0);
setTimeout(rest, 0);

As you can probably see, the execution of the script is quite different. But the point is that in the original code, when you get to the point where you actually need “a”, you say to the system: ok, go ahead and compute “a”, and run the rest of this method when you are finished with that. You can extend this even further. For instance, the system might simply assign the “computation of a + b” to c, and neither a, b, nor c would get computed until you called alert. And the cool part is that if you didn’t call alert, it wouldn’t even get computed at all! That’s one of the ways continuations might work. And as you can see, rewriting regular code to work like continuations requires quite a bit of change to the code. There is also a loss in code-readability and performance.

Another, more used form of continuations is the yield, or generators in general. Take for instance this code (written in Augmented JavaScript):

function test() {
    yield 0;
    alert(10);
    yield 1;
}

var gen = test();
alert(gen.next());
alert(gen.next());

The code above alert first 0 then 10 and lastly 1. If I remove the second call to “gen.next”, only 0 is alerted. As you can see, the test-method is “suspended” in the middle of the method, and is “continued” at the exact point where it left, variables and state being retained.

Turning yield into goto

I’ve already written a post on generating generators, so I’m not going to get too much into that. The post is about generating yield-like behavior in C# (which already supports yield natively), but the steps taken are mostly the same in Augmented JavaScript. The main difference is that javascript has functions instead of classes, so instead of creating a class with class-members for the method-variables, you simply create an enclosing function and hoist the variables into that function instead. Thus state can be retained in a fairly simple manner.

There is however one small problem with the solution presented as to how to make generators in C#. JavaScript do not support goto-statements. The solution provided is fairly dependent upon gotos. They are what enables the method to continue execution where it left off, without having to start at the beginning each and every time, thus a solution cannot be implemented without the use of goto. At least not using the algorithm described. Now, I’ve been told there are other, more elegant solutions to solving this problem in JavaScript that splits the function-body into multiple functions, but I’m unable to mentally picture how such an algorithm would deal with if’s and loops, etc., and as such I have not used it. Instead, I used the same algorithm provided in the C# example, and came up with an algorithm on how to implement goto in JavaScript.

Now, before people start screaming about how evil goto is, let me state a few things. The algorithm I use is not a generic goto-resolver. It can not deal with any and all gotos, only those that follow a few rules. For instance, all jumps must be made forward. Also, Augmented JavaScript does not, in and of itself, support goto. You cannot write goto in your code and expect it to process it. The gotos are only used at an intermediate stage when rewriting a function into a generator, and is subject to the fact that since we generate them, we know how which formats they appear in. And it’s in all actuality only a single type of gotos that are used, it’s a forward-jumping goto placed in a switch-statement at the top of a function. A prime example of this displayed below.

// Augmented JavaScript
function myGenerator() {
    yield 0;
    yield 1;
}

// Rewritten using goto
function myGenerator() {
    var state;
    function moveNext() {
        switch(state) {
            case 1: goto state1;
            case 2: goto state2;
        }

        state = 1;
        return [false, 0];

        state1:;
        state = 2;
        return [false, 1];

        state2:;
        state = -1;
        return[true];
    }
    return new Generator(moveNext);
}

Just to be clear, the steps taken to rewrite a yield-function into a generator-function using goto has been simplified here. In reality, there’s a bunch of exception-handling and making sure the state is ok, but this still shows the general algorithm used.

Resolving the gotos

To solve a simple goto-function like this is rather easy. You simply turn the code into a switch-statement with fallthrough, and you get something like this (enclosing function omitted):

function moveNext() {
    var _goto = false;
    switch(state) {
        case 1: _goto = "state1";
        case 2: _goto = "state2";
    }

    switch(_goto) {
        case "state1":
            state = 2;
            return [false, 1];

        case "state2":
            state = -1;
            return [true];

        default:
            state = 1;
            return [false, 0];
    }
}

Notice however, that the first case from the previous sample (yield 0) got thrown at the bottom. This is because we have fallthrough on the switch-statement (notice the distinct lack of break-statements). Now, you might argue that because all the cases ends in a return-statement, this doesn’t matter (and you would be right), but there are cases where it does matter. I’ve found 2 ways to resolve this issue. One is to set _goto to something like “state0″ on default (in the switch on state), and the other is to put the switch in a loop. While the first one might be easier to read, the second one seems to me to handle nesting of scopes (I’ll get back to that in a bit) a lot better. But, since we’re inside a switch, we can’t simply exit the loop by breaking, so we need to put a label on the loop to handle this. Also, after the default case is done, what we want to happen is the “state1″ case (because, that’s the normal execution-flow if you look on the original code), and the way to make this happen is to introduce a new “special” _goto-state that will always enter the first switch-statement (as having been flowed into from the last one).

The resulting code after taking the actions described above looks like this:

function moveNext() {
    var _goto = false;
    switch(state) {
        case 1: _goto = "state1";
        case 2: _goto = "state2";
    }

    scope0: while(true) {
        switch(_goto) {
            case "enter":
            case "state1":
                _goto = false;
                state = 2;
                return [false, 1];

            case "state2":
                _goto = false;
                state = -1;
                return [true];
                break scope0;

            default:
                state = 1;
                return [false, 0];
                _goto = "enter";
        }
    }
}

There are a few things to notice here. First of all, there is unreachable code at several locations. For instance, in this example, _goto will never be set to “enter”, simply because default has a return-statement. The “break scope0;” will also never be reached, as it is also after a return-statement. However, there are cases where this is not true, and as I’m not that good at three-traversal to figure out what is definitely dead code, I’m adding them allways, just to make sure. Also, the JavaScript-compiler/interpreter in your browser/environment should hopefully optimize the dead code away.

The algorithm thus far is more robust than before, but there are still things it cannot handle well. Let’s take a look at another example, shall we?

'
// Augmented JavaScript
function test() {
    var testVar = true;
    if(testVar) {
        testVar = false;
        yield 0;
        yield 1;
    }
}

// Rewritten - testVar exists outside the scope of moveNext
function moveNext()
{
    var _goto = false;
    switch(state) {
        case 1: _goto = "state1";
        case 2: _goto = "state2";
    }

    // What to do here?
}

This example is mostly the same as the previous one, however there is one big difference, everything that happens happens inside an if-statement. The way to resolve this is first by looping through all scopes (starting by the innermost ones), and check if they contain any jump-labels (like “state1:;”), and if they do, run gotorewriting inside that scope. So, let’s do that one step at a time, starting with rewriting the inner if-scope.

function moveNext()
{
    var _goto = false;
    switch(state) {
        case 1: _goto = "state1";
        case 2: _goto = "state2";
    }

    if(testVar) {
        scope0: while(true) {
            switch(_goto) {
                case "enter":
                case "state1":
                    _goto = false;
                    state = 2;
                    return [false, 1];

                case "state2":
                    _goto = false;
                    state = -1;
                    return [true];
                    break scope0;

                default:
                    testVar = false;
                    state = 1;
                    return [false, 0];
                    _goto = "enter";
            }
        }
    }
}

The algorithm is the same as last time, only we run it inside the inner scope this time, and left the if-test as-is. One of the problems you might notice is that after we enter the if-test for the first time, there is no simple way to reenter it. To solve this, all tests have to be rewritten to allow through in states where they should begin in the middle of a guarded block (like an if, if-else, while, for, etc). The rewriting-rules for the test is simple. Simply find all states that are inside, and allow those to get through (without actually running the original test, as this might have side-effects). So instead of “if(testVar)” we get “if((_goto == ‘state1′ || _goto == ‘state2′) || testVar)”.

The current code is almost done. All we have to do is rerun the same algorithm on the outer scope, although we need to pretend that all the state-labels inside the if lies just ahead of it.

The resulting code looks like this:

function moveNext()
{
    var _goto = false;
    switch(state) {
        case 1: _goto = "state1";
        case 2: _goto = "state2";
    }


    scope1: while(true) {
        switch(_goto) {
            case "enter":
            case "state1":
            case "state2":
                if((_goto == "state1" || _goto == "state2") || testVar) {
                    scope0: while(true) {
                        switch(_goto) {
                            case "enter":
                            case "state1":
                                _goto = false;
                                state = 2;
                                return [false, 1];

                            case "state2":
                                _goto = false;
                                state = -1;
                                return [true];
                                break scope0;

                            default:
                                testVar = false;
                                state = 1;
                                return [false, 0];
                                _goto = "enter";
                        }
                    } // end scope 0
                }
                break scope1;

            default:
                // nothing here. The if was the first-statement.
                _goto = "enter";
        }
    } // end scope 1
}

And there we have it. Continuation in plain old JavaScript. If I haven’t overlooked anything this should be equal in functionality to the original snippet. There are some cases I haven’t covered though, like loops, but they are mostly the same. Another thing I omitted to (try to) keep things short, is the fact that you have to make sure that _goto = “enter” don’t “leak” into inner scopes. As you can see, the end-result isn’t pretty, and this isn’t even that complex an example, so the sourcecode generated from Augmented JavaScript isn’t supposed to be read too intently (unless you are searching for bugs in the preprocessor itself).

I hope this has been educating, or that at least a few people might have gotten a thing or two from this ^^.

Until next time.

Introducing Augmented JavaScript

Today I’d like to introduce my latest project named “Augmented JavaScript” to the world.

What is Augmented JavaScript?

Augmented javascript is a javascript-preprocessor written in javascript. It’s function is similar to the more known javascript preprocessor CoffeScript, however, while CoffeScript has more or less invented a new language alltogether, Augmented JavaScript is a lot more similar to plain old JavaScript.

Currently Augmented JavaScript adds 3 new features to the JavaScript language. All of these features are new, and not very well tested, so bugs should be expected. These new features are:

  • Await
  • Yield
  • For..Of-Loops
  • Shorthand Function-Syntax

Await

The await-keyword is taken from the newer versions of C# and VB.NET. It’s job is to pause the execution of a piece of code untill new data is availible, and then continue off where it left. This results in the ability to write async functions very easily without the need for callback-nesting. For instance, let’s asume we have a “magic” function called load whos task it is to perform a simple xhr-request and callback when the data is availible. Using regular javascript one would in general write code such as this:

function login(username, password, cb) {
    getUser(username, password, function(user) {
        getGroups(user.id, function(groups) {
            cb({
                user: user,
                groups: groups
            });
        });
    });
}

function getUser(username, password, cb) {
    load('/getUser', {
        username: username,
        password: password
    }, function(data) {
        if(data.success) {
            cb(data.user);
        } else {
            cb(null);
        }
    });
}

function getGroups(userId, cb) {
    load('/getGroups/' + userId, function(data) {
        if(data.success) {
            cb(data.groups);
        } else {
            cb(null);
        }
    });
}

As you can see, there’s a lot of nesting of functions, and it’s not always easy to understand what is going to happen and in what order. People who are unused with javascript typically expect that when you’ve run the login-function, the user will be logged in immidiately, which isn’t the case because our load-function is async. Now, take a look at the same code rewritten to use Augmented JavaScript (here ofcause I assume that the load-function has been written in such a way that it is usable with the await-keyword and not just callbacks as in the previous example, more on that later).

function login(username, password) {
    var user = await getUser(username, password);
    var groups = await getGroups(user.id);
    return {
        user: user,
        groups: groups
    };
}

function getUser(username, password) {
    var data = await load('/getUser', {
        username: username,
        password: password
    });
    if(data.success) {
        return data.user;
    } else {
        return null;
    }
}

function getGroups(userId) {
    var data = await load('/getGroups' + userId);
    if(data.success) {
        return data.groups;
    } else {
        return null;
    }
}

Quite a bit easier to read, no? The result of these two snippets of code should theoretically be the same. However, as said, the probability of experiencing bugs in Augmented JavaScript is still fairly high, and I have not explicitly tested the snippets found here yet.

What is awaitable

Internally, Augmented JavaScript uses the exelent Q-library to handle waiting. That means that anything that can be understood by the Q-library, can be waited for.

Yield

The second new feature is the yield-keyword. The yield-keyword works in certain ways almost like the await keyword, with the exception that instead of waiting for whatever you pass after the yield-keyword it is returned to the calling function. The yield-keyword is simply put used to create generators (or iterators). A simple example illustrates best how it works:

function testGenerator() {
    yield 1;
    yield 2;
    yield 3;
}

function runTest() {
    var iter = testGenerator();
    console.log(iter.next()); // outputs 1
    console.log(iter.next()); // outputs 2
    console.log(iter.next()); // outputs 3
    console.log(iter.next()); // throws StopIteration
}

Using yield you can easily create functions that have no end, because only the data you request is generated. For instance, one can create a simple fibionacci-generator like this:

function fib() {
    var a = 1, b = 0;
    while(true) {
        yield a;
        b = a + b;
        yield b;
        a = a + b;
    }
}

For..Of-Loops

The for..of-loop only works on iterators or objects that have the same signature. Use for..of to loop though all the values generated from a generator. For instance, if you take the function runTest created in the previous section, you could rewrite the code to this using for..of:

function runTest() {
    for(var n of testGenerator())
        console.log(n);
}

break and continue works as one would expect, so to print all the fib-numbers less than 100 you could do this:

function printFib() {
    for(var n of fib()) {
        if(n < 100) break;
        console.log(n);
    }
}

Shorthand Function-Syntax

The shorthand function-syntax is fairly straight forward. The valid forms are as follow (all of these are expressions):

1. <identifier> => <expression>
2. <identifier> => <statement>
3. (<arguments>) => <expression>
4. (<arguments>) => <statement>

<arguments> is a comma-separated array of 0 or more identifiers. If any of the variants that ends in an expression is used, said expression is automatically transformed into a returnstatement. This allow for simple map-functions to be written a lot shorter than in regular JavaScript. Here are some samples:

var add = (a, b) => a + b;
function makeFunction(itm) {
    return () => itm;
}
load(data => {
    // do stuff with data
});

Yielding results – the process involved with generating generators

One of the nice things about the language C# is it’s ability to create generators with the yield keyword. Now, for those of you who do not know what I’m talking about, take a look at the following code:

public IEnumerable<int> MakeGenerator(int num0)
{
    yield return num0;    

    int i = 0;
    Console.WriteLine("yield 0");
    yield return 0;
    try
    {
        i += 1;
        Console.WriteLine("yield 1");
        yield return i;
    }
    catch {}
    Console.WriteLine("end");
}

The code above will return a IEnumerable, and will not do anything else, until the IEnumerable is used. In other words, if I invoke MakeGenerator, none of the code I wrote will actually be run. Now, this might sound completely ridiculous, but I assure you, there is a good reason for this. To explain it as simple as possible, consider the following code, what would you guess the output to be like?

foreach(int num in MakeGenerator(5))
{
    Console.WriteLine("Got " + num);
}

The answer might surprise you. If you run the code above, you will get 6 lines of output, in the following order:

Got 5
yield 0
Got 0
yield 1
Got 1
end

The reason for this is that first when GetNext on the enumerator which the foreach-loop get’s from the IEnumerable is called, the code in the MakeGenerator-method is actually run. However, as you can see, not all of it is run. When we hit any of the yield-statements, the method is returned from. Now, this is a construct that’s actually quite useful in a lot of cases, cause for once, it enables lazy evaluation of your code over enumerables. LINQ for once, relies heavily on this (note; I do not know if LINQ uses yields or customarily created enumerators, but they both facilitate lazy evaluation over enumerables). For instance, if you have a list of a million elements (called myList), and you do myList.Select(CpuHeavyMethod), this will not fry your CPU into oblivion, rather, it will not run the CpuHeavyMethod at all! First when you actually have need for an element passed through the CpuHeavyMethod it will be called.

For those of you who still do not know what a generator is, you can learn more here: http://en.wikipedia.org/wiki/Generator_(computer_programming)

Creating generators

Now that (hopefully) all of you have a basic understanding of what generators are, and how they work, let’s look at how to make them. In other words, let’s look at what the compiler does with the yield keyword. Now, before I start, I’d just like to say that this is most likely different from language to language, and this may or may not be the way C# handles compiling generators (I’ve not looked at how the C#/mono compiler solves the problem, what I’ve read is how the IronPython compiler/interpreter does it. However, I’ve also changed that a bit to simplify).

As it turns out, generating generators is actually rather simple, there are just a few things you have to look out for:

  • Try statements (with or without finally) must be handled separately (and with a little bit of caution).
  • There cannot be yield-statements in catch or finally-blocks.

Note: these are rules I’ve set for the generator I create. It may be possible to have yields in both catch and finally in C#, though it would require further rewriting of the method, and will not be discussed in this article.

The first thing we need to do (in C# anyways) is to create 2 classes. One for the IEnumerable, and one for the IEnumerator. Also, these two classes will be nested, to enable the inner one to access the outer one’s private variables.

Let’s start with these two completely empty classes (save for the interface-implementations, and _state and _current values which I’ll get back to later).

class MakeGenerator_Generator : IEnumerable<int>
{
	IEnumerator<int> IEnumerable<int>.GetEnumerator()
	{
		return new MakeGenerator_Enumerator(this);
	}

	IEnumerator IEnumerable.GetEnumerator()
	{
		return new MakeGenerator_Enumerator(this);
	}

	class MakeGenerator_Enumerator : IEnumerator<int>
	{
		private readonly MakeGenerator_Generator _gen;
		private int _state = -1;
		private int _current;

		public MakeGenerator_Enumerator(MakeGenerator_Generator gen)
		{
			_gen = gen;
		}

		int IEnumerator<int>.Current
		{
			get { return _current; }
		}

		object IEnumerator.Current
		{
			get { return _current; }
		}

		bool IEnumerator.MoveNext()
		{
			throw new NotImplementedException();
		}

		void IEnumerator.Reset()
		{
			throw new NotImplementedException();
		}

		void IDisposable.Dispose()
		{
			throw new NotImplementedException();
		}
	}
}

The first thing we need to do is to take all the method-parameters given to the MakeGenerator method and make them class-level readonly members of the Generator-class. Then we need to add a constructor that takes the same number of arguments, and saves them to the class members.

class MakeGenerator_Generator : IEnumerable<int>
{
	readonly int param_num0;

	public MakeGenerator_Generator(int num0)
	{
		param_num0 = num0;
	}

Then we need to copy those to private members of the Enumerator-class during it’s Reset() call, and lastly we need to call Reset() from the constructor like this:

	class MakeGenerator_Enumerator : IEnumerator<int>
	{
		private readonly MakeGenerator_Generator _gen;
		private int _state;
		private int _current;
		private int val_num0;

		public MakeGenerator_Enumerator(MakeGenerator_Generator gen)
		{
			_gen = gen;
			((IEnumerator)this).Reset();
		}

		void IEnumerator.Reset()
		{
			val_num0 = _gen.param_num0;
		}

The next thing we need to do is to copy the entire content of the original method into the MoveNext method of the Enumerator (note, this will make for invalid code, but we’ll fix that in a bit). The result should look like this:

		bool IEnumerator.MoveNext()
		{
			yield return num0;

			int i = 0;
			Console.WriteLine("yield 0");
			yield return 0;
			try
			{
				i += 1;
				Console.WriteLine("yield 1");
				yield return i;
			}
			catch {}
			Console.WriteLine("end");
		}

Now that we have the base-foundation of the generator, we can start editing the original code to make it work as a generator (in other words, rewrite it to a proper MoveNext method).
What needs to be done in the MoveNext methods is two things for startes. First; all variables needs to be hoisted out to class-level variables, and second, we need to enumerate the yield-statements (starting from 1). Than you change the nth yield-statement into this:

_state = <n>;
_current = <yield_value>;
return;
yield_label_<n>:

The result of doing those two actions looks like this:

		bool IEnumerator.MoveNext()
		{
			_state = 1;
			_current = val_num0;
			return;
			yield_label_1:;

			val_i = 0;
			Console.WriteLine("yield 0");

			_state = 2;
			_current = 0;
			return;
			yield_label_2:;

			try
			{
				val_i += 1;
				Console.WriteLine("yield 1");

				_state = 3;
				_current = i;
				return;
				yield_label_3:;
			}
			catch {}
			Console.WriteLine("end");
		}

For those of you who have never seen a label in C# before, this code might look a bit confusing. A label in C# is created by typing a name followed by a colon, like “yield_label_0:” The use of a label is with goto, where you jump to a label. The reason I’ve added a semicolon after the label, is simply to add an empty statement you jump to. This would not strictly be necessary, however, if I do not the yield_label_3 would be invalid (as you cannot have a label that goes “off a cliff”, ie. out of a method, if, while, block, etc.). The next part is simply to add a switch over the state in the beginning of the MoveNext method, that jumps to the right continuation. Also, we need to add the final state (_state == 0, finnished). The result looks like this:

		bool IEnumerator.MoveNext()
		{
			switch(_state)
			{
				case 1: goto yield_label_1;
				case 2: goto yield_label_2;
				case 3: goto yield_label_3;
				case 0: goto yield_label_0;
			}

			_state = 1;
			_current = val_num0;
			return;
			yield_label_1:;

			val_i = 0;
			Console.WriteLine("yield 0");

			_state = 2;
			_current = 0;
			return;
			yield_label_2:;

			try
			{
				val_i += 1;
				Console.WriteLine("yield 1");

				_state = 3;
				_current = i;
				return;
				yield_label_3:;
			}
			catch {}
			Console.WriteLine("end");

			_state = 0;
			yield_label_0:;
			return _state != 0;
		}

This is basically a working generator. The two first values would work just fine if we simply left it like this (and removed the try), however, since we do have a try (cause I wanted to show how to solve this), this is what we do.

  1. Start with the outermost try (only do this if the try contain yield-statements).
  2. Create a new label in front of the try.
  3. Rewrite all goto-statements that point to a yield inside the try so they point to the new label instead.
  4. Rewrite the try so it jumps based on state.

In other words, it goes like this:

switch(_state)
{
	case 1: goto yield_label_1;
	case 2: goto yield_label_2;
	case 3: goto try_label_1; // 3. Rewrite goto-statement to point to start of try
	case 0: goto yield_label_0;
}

_state = 1;
_current = val_num0;
return;
yield_label_1:;

val_i = 0;
Console.WriteLine("yield 0");

_state = 2;
_current = 0;
return;
yield_label_2:;

try_label_1:; // 2. create label in front of the try
try
{
	switch(_state) // 4. Rewrite the try so it jumps based on state
	{
		case 3: goto yield_label_3;
	}

	val_i += 1;
	Console.WriteLine("yield 1");

	_state = 3;
	_current = i;
	return;
	yield_label_3:;
}
catch {}
Console.WriteLine("end");

_state = 0;
yield_label_0:;
return _state != 0;

And there you have it. We’ve made ourselves a generator. The only thing that needs to be fixed now is to remove the NotImplementedException inside the Dispose-method. What you should probably do inside the Dispose-method is set the _gen-variable to null, and check for it being null in both MoveNext and Current (and if it is, throw a InvalidOperationException because it’s already disposed), however, I’ll leave that up to you guys. Also, I was planing on showing how to create Generators with the DLR, though it seems this post became longer than I planned it, and if I were to start explaining the DLR stuff now, I’d probably more than double the length, so that will come later. I hope this made it easier to understand how generators work, and what the compiler does with them.

Until next time.
– Alxandr

Await anything – A tale about asynchronity

As most people who do any .NET related programming probably know, a short while ago, the next major version of the .NET framework and the tools that goes with it was released. This includes (but may not be limited to) Visual Studio 2012, .NET 4.5 and C# 4.5. One of the major new features of the .NET framework is its new support for async programming. Now, async programming has always (or rather, for as long as I can remember having wanted to do anything async) existed. For instance, if you create a Socket-instance, you have “BeginReceive” and “EndReceive” which provides asynchronous capability, however using them can be cumbersome. Normally you need a class to keep track of state, or you end up with a bunch of closures (given that you have .NET which is new enough to support closures). What this new capability does is to make it easy to create asynchronous code, and it generally does so with the help of Tasks.

Now, Tasks (System.Threading.Tasks.Task) have existed in .NET for a while, and I’ve used them for a while, cause I find they provide a simpler way of chaining calls one after another. For me, which have done most of my work in javascript, the ContinueWith<TIn, TResult>(Func<Task<TIn>, TResult>) feels natural, and easy to use. Handling completion-events is much more cumbersome for me. However, with .NET 4.5 the use of Tasks has been simplified even more, and of-cause I’m talking about the async and await keywords. However, I’m not going to explain how to create async functions, how these functions return Tasks, and what are great about them, cause there are probably already a thousand other post explaining this, what I want to look at more in-depth is the await keyword itself.

Awaiters

My quest for answers with regards to the await-keywords starts with the new release of Reactive Extensions. Reactive Extensions mainly does it’s work with a single interface; “IObservable<T>”. Now, because this is an interface, there is no way (that I know of) it could implement Task<T>. Neither, does (as far as I know) Task<T> implement IObservable<T>, meaning there is no connection between them what so ever. Here you might probably wonder why I’m stressing this out, and the reason for that is rather simple. While writing code working with IObservable<T>, Visual Studio kept telling me how IObservable<T> was awaitable! In other words, I could write await in front of any instance of IObservable<T>!
How could this be? Certainly, as I just explained, there is no connection between IObservable<T> and Task<T>. The answer, has to do with awaiters.

Awaiters, as I found after some googling, turns out to be one of the most curious compiler-patterns I’ve ever seen. From what I understand, for a type (lets call the instance myWait for now) to be awaitable, the following must be true:

  1. myWait.GetAwaiter() must be a valid expression
  2. myWait.GetAwaiter().IsCompleted must be a bool-property that is gettable, and returns a value indicating weather or not myWait is completed.
  3. myWait.GetAwaiter().GetResult() must be a valid expression, and must be a method that returns the result of the async operation, in other words, the result of “(await myWait)”.
  4. myWait.GetAwaiter().OnCompleted(Action continuation) must be a valid method that takes a single Action and quest that method for execution when the async operation is finnished.

Notice that I’ve stated that myWait.GetAwaiter must be a valid expression? What I find curious about this, is that it must be a valid expression for the compiler, not a valid method of myWait. In other words, GetAwaiter can be an extension method! This makes it possible to do some rather absurd things with await like await a TimeSpan (to delay the current execution), given that you provide a one-line extension-method for TimeSpan. More about this can be found here: http://blogs.msdn.com/b/pfxteam/archive/2011/01/13/10115642.aspx

SpotiFire

Last in this article I’d like to say a few words about a small project of mine, named SpotiFire. SpotiFire is a wrapper around the libspotify API’s, that makes it fairly easy for a .NET application to interact with spotify. The last update to SpotiFire (uploaded today) enables the use of async for a lot of operations. Here is a simple sample showing how easy some of the operations that used to be hard to do is:

Push to PC – A cry for attention

Currently I’m registering about 2 paradigmatic changes going on in this PC world, and they have been going on for a while. One of them is that everything should be moved to the web. Or the cloud if you like. This includes the notion of storing all data in the cloud, but also includes the process of having everything run in your browser. In 10-15 years, it might be that everything except for the browsers are coded in html and JavaScript, but I’m not speculating about that right now.

The second paradigmatic change is that everything should be considered a standalone app. Even though it’s simply a browser page (dearly note that these aren’t necessarily simple anymore). This point also includes the fusion (or partial fusion) of mobile (or at least tablet) and pc, a notion all major operating-systems are currently moving towards. From Microsoft you have Windows 8 (and WindowsRT, where the later one is actually a tablet-only OS, and the former one is WinRT + Win7 more or less), and on the OSX side of the world you have a bunch of iOS features being integrated into OSX.

There are a lot of complaints to Microsoft regarding this shift in windows, however I’m not here to complain about this either. Personally I find windows 8 to be the best windows so far, and the windows RT experience stays out of your way if you don’t want to use it.

Web-apps as first class applications

One of the things I do frequently on my computer (currently running windows 8 RP) is to check mail. The obvious solution to this problem is to use the “Mail”-app provided with Windows 8. It looks like this:

Now, if you ask me, this is a fairly nice looking, and functional application, however, the main problem about this application is that it’s made for hotmail/exchange. That does not mean it will not work with my gmail-account, far from it, but it means that it does things slightly differently (also, as a side note, it crashes on start on this computer, but it is a preview after all, so I’m guessing this will have been resolved in the release of windows 8). These slight differences are annoying, because my handling of email (how I put them in folders, when I archive them, etc.) is built arround how Gmail handles email. This means that in order to get things “my way” I either need to use a different client, or I simply need to use the browser version, and for me this is a simple choice.

Browser-based applications have pros and cons with regards to full-fledged apps, and a lot of the cons are attempted being removed with the use of new html-standards (like offline caching), and javascript-standards (like local database, filesystem etc.). Other cons are being removed by the browser-vendors, like chrome’s possibility to make “program-shortcuts” to web-pages (or web-apps if you like). Now, these are good steps along the way to make the browser-based Gmail-client function just as good as the built-in email-client in windows 8, however there is one thing that the built-in client does infinitely better than the browser-based client.

You have mail

As you can probably see, this isn’t actually a mail-notification (as I can’t get my Mail-application working, I can’t screen-shot that, nor could I find one on google images), but it’s a notification non the less. In fact, the newest version of all major operating systems (counting windows 8 as the newest version of the windows family) have support for some sort of native notifications. And any email-application that comes with the system is bound to take advantage of that. This means that if I simply booted up my pc (without starting a single application), and somebody sent me an email, I would get a notification in my face.

For those of you who use email on a “smart-phone” (windows phone 7+, android or iOS) this should be very familiar, because these kinds of notifications have been around on our cell-phones for several years already. You shouldn’t have to actually open your email-application to check if there are new emails, the email-application should tell you when there are. This is definitely the case with the built-in windows 8 email app, however, it is not the case with GMail running in my browser.

Platform support

Currently, a lot of platforms however (or rather, the platform of a lot of users, meaning windows < 8) do not support native notifications. Still, this is fairly simple to achieve, using a popup-window that contains what would otherwise be served by a native notification, when there is no way of doing it natively. On OSX prior to mountain-lion this can be achieved using Growl for instance. Also, the newest version of Growl (Growl 2, still in beta currently) supports the notification-center on OSX, meaning that on OSX everything could potentially be handled by the Growl-framework.

I’m not familiar enough with the different linux-distros out there to begin guessing how to handle something like this on Linux, so I will not say too much about it, however, to my knowledge, Ubuntu Linux has had a built-in growl-like notification-utility the last few years which could probably be utilized without much further ado.

API and protocol

What I am proposing is that there should be made a standard API for enabling cross platform, native (if provided by the platform) push notifications that can be subscribed to using simple JavaScript. also, there need to be made a standard protocol for pushing notifications to the clients.

However, who am I to propose such a huge demanding thing to be made and standardized? The sad truth is that when it comes to things like this, I am nobody. I am but a simple, single voice in the huge internet, and that is why I’m writing this post, in hope that someone bigger than me will see it and think that my reasoning and proposal is somewhat fair, and that it in time will get picked up.

Still, even though I am far from qualified to undertake such a huge task on my own (cause it is a huge task), I have been thinking and drafting on this “specification” for a while, and thought I’d air my thoughts here, to show more or less what I imagine the functionality should/could be like.

The API

The first part of the specification needed is the client API (JavaScript API) to register for push-notifications. This should be simple, yet secure, and should conform to the same-origin policy for safety. I’m suggesting 2 simple properties and a function should be added to the client to facilitate push-notifications.

window.pushNotifications = {
  // state property, read only.
  // should return one of the following values:
  //   0: pushNotifications not registered
  //   1: attempting registering for push notifications
  //   2: pushNotifications registered

  state: readonly property,
  // onstatechange property, defaults to undefined
  // if function, called whenever state is changed
  onstatechange: readWrite property

  // register, function
  // takes a single parameter, which is a callback-function
  // callback-function is called when registering is done
  // independent on whether or not the registering succeeded
  // the callback-function takes a single argument, which
  // on failure is { success: false, error: Error-object },
  // and on success is { success: true, endpoint: push-endpoint }
  // push-endpoint is a DOMString. Described later.
  register: function(callback) {}
}

When register is called, if push-notifications are already registered, it should simply run the callback with the previously registered endpoint. If push-notifications are not already registered the user should be presented with a choice (similar to the choices to enable location and other such API’s) before anything else happens. If the user disapproves that the web-page can send push-notifications, the callback should be called with an error message stating that the user disapproved, else it should call up to the notification-server to register for an end-point. When the response is returned the notification-endpoint should be given to the callback function. Once the client has hold of a notification-endpoint, it should store this on its server.

The protocol

The protocol is unfortunately a bit more complex than the client API, but should still be fairly simple. All calls to the notification-server should be done over https. When register for a notification-endpoint, 3 peaces of data should be included. A client-id to identify the client, the domain of the web-page running the register-call and whether or not the domain uses https. The notification-server should then store these, and generate a unique notification-endpoint https url. This url should then be returned to the client. Note that the client-id does not necessarily have to be unique for the client, but might instead be unique for the user, but should definitely not be unique for the machine. Two users on the same machine should not receive each-others push-notifications. However, since they can be unique per user, this also means that the same unique identifier might be used across several machines where the same user is authenticated, meaning that all the user has to do is log in to some sort of synchronization-environment in his browser, and he would receive push-notifications that he registered for elsewhere.

After a user is registered for push-notifications, the server can begin sending them. To make sure that the push-notifications are as similar as they can be between all platforms, the information being sent should be limited to 2 strings (a title and a details string), a url to an image, and a url that should be opened when the user clicks the notification. This should be sent to the server using json serialization of the data. These are some examples of valid push-notifications:

{ "title": "New email" }
{ "title": "New email from test@example.com",
"description": "Subject of said email" }
{ "title": "New email from test@example.com",
"description": "Subject of said email",
"url": "http://my-email.example.com/mail/id_of_email" }
{ "title": "New email from test@example.com",
"description": "Subject of said email",
"url": "http://my-email.example.com/mail/id_of_email",
"icon": "http://my-email.example.com/contact_example/profile_picture.jpg" }

The only information that is required is the title. If no url is provided than the domain-name registered should be used as an url (in this case “http://my-email.example.com/&#8221; if the domain does not use https). If no icon is provided, the favicon of the website should be used. Title and description should both have a max-length, title of 255, description of twice that. The server needs to make sure that any push-notifications sent is sent from the same domain that registered for the end-point. This is to prevent spamming of push-notifications if a third-party at some point got a hold of the push-endpoint. However, it should be possible to enable other servers to send push-notificaitons (in cases of server-farms for instance). I’m proposing using DNS to enable this, with similar behaviour as the SPF records used for email. However, here is one of the points where I most certainly am not qualified to speculate on the best way to resolve this problem, thus I will not write any more about it.

The plea

This concludes my description of a protocol and an API I wish existed, and my plea is simple; if you too wish that something like this existed, spread the word, so that in time, someone who knows where to start with something like this might pick it up. And if you have any comments or questions please leave a comment below. And I hope, that some day, in a not too distant future, this can be made into a reality.

Thanks for reading, and please spread the word.

CKEditor and all it’s merrits (and the likes)

I’ve been working a lot with CKEditor the last couple of days, trying to upgrade our website from an old and probably fairly outdated FCKEditor. The funny part about FCKEditor is that at first everything feels like a breeze. I think the time it took me to replace the old FCKEditor with something that looked like it was working was about 2-5 minutes, and to get it actually working was only a couple of minutes more. However, the joyride down to wonderland (while wearing a grin saying “OMG; this is so easy, and the result looks so nice!”) kinda ends there. If any of you have worked (extensively) with CKEditor you will probably know that it’s made to be highly customizable, with lots of settings and stuff for you to play with. However, the problem arises once you try to do something you weren’t supposed to. For instance; on one of the things we didn’t want on our page was to set the default cellspacing and padding on tables to "" (empty string; ie. remove the attributes). This sounds like something that should be fairly easy to do (and probably would be too if I didn’t use the minified code). At first I looked for settings that would enable me to set this, however I was unable to find any, so what I ended up doing was to change '1' in plugins/table/dialog/table.js associated with the cellspacing and the cellpadding fields to ''.  That however, was the easy part. The next thing that was requested was to by default mark tables with table-layout: fixed. Now to do so turned out to be harder than I thought it would be. And just to make sure I’ve said it, this doesn’t necessarily mean it’s the only way to achieve this result in CKEditor, nor is it necessarily the right way to do it, but it was the first and quickest way that I managed to find (without having to write my own table-plugin for CKEditor). What, however, worked for me was locating if(!this._.selectedElement){var p=l.append(h('tbody')) in the existing table.js plugin, and then adding l.setStyle('table-layout','fixed');. Also, if you would (by default) like to have a custom class on all tables created with the table-plugin, here is the place to add it. Simpy do l.addClass('class-name-here'); in the same if-statement.

Filemanager

One of the not so fun things about the “new” CKEditor (with regards to the “old” FCKEditor) is that it was split in two. Now you have CKEditor and CKFinder. However, the story doesn’t quite end there, cause CKFinder is a commersial tool that costs money, and if you’re just working on some pet project you don’t want to kough up a couple of hundreds of dollars just to be able to add files to your user-content. While, however, this is one of the things I consider bad about the new CKEditor, it must be said that the opening of the CKFinder is an extension-point in CKEditor, meaning that you just give it an URL of something that works like CKFinder, and CKEditor will be happy as a newborn child. The answer, ofcause (like all other things in life) is to make it yourself. Or rather, get someone else to make it for you, that doesn’t want to (or at least doesn’t require) to get paid, but that’s the beauty of open-source, right? Everybody contributes their part, and in the end there is awesomeness. However, getting back from my quick digression, I found something called simply “filemanager” (meaning it’s impossible to google for). It’s apparently made by a company named c5 (and pull-requests from whoever wanted), and it can be found somewhere around here, it’s github-repository is found here.

The original filemanager looks like this:

If you ask me, I think it’s rather pretty made, and like the design, but I was asked to simplify it a bit (remove “unneeded” elements). A couple of the things that wasn’t wanted was the “Current folder” up in the left corner, the home button, the new folder button, and also, the upload button. This is not to say that I should disable file-upload. I should simply make it so that whenever a file is selected by clicking the “browse”-button, instead of having to click upload afterwords, the form should automatically be submitted. In jQuery (which the plugin uses) this is as simple as $('#newfile').change(function() { $('#upload').click(); }). Also, I had to provide my own translation, cause a norwegian translation was not present, but as I only needed some of the features, only some of the features was translated. At last, the “dimensions”-column should be removed.

The resulting filemanager looks like this:

One of the things to notice is that the browse-button is moved to the upper left corner and is localized. This localization is not the browser that renames the browse-button, it’s a customarily made button (that doesn’t require any flash-upload or the likes, however that’s a feature I would like to implement at a later point given the chance). The html to achieve this looks like this:

Once again, I’ve uglified the file a bit, however it was simply the fastest way of achieving the desired result. The file-upload “hack” I’ve provided should work in all modern browsers (safe for Opera which I have never bothered tested with as our whole site generally don’t work in Opera at all it’s not a requirement). A general improvement over the initial version I started with (if you ask me) is the fact that once I upload a file (click the browse-button and select a file), the file is also automatically selected and thrown back into CKEditor. Some users might not want this, however, for our usage we concluded that whenever a user clicks the browse-button, they’ve been sent here from CKEditor to select a file, and they’ve concluded that the file needs to be uploaded first. There should be no point in uploading a file simply to upload it.

One of the other things I fixed was the date-column (shown as “Endret” in norwegian). With the original code it would show dates sent from the server as text (I would get the string “25.06.20012” from the server). Now this doesn’t seem to bad until you start sorting by date. Just think about it for a second; since year isn’t first (like in YYYY-MM-DD) the sorting will fail. This might not be the case for american dates, cause I have a suspicion that the fancy table-sorter included in Filemanager could handle those, however, for anything else it would fail horribly. Also, Filemanager contained no way of changing how the dates are shown (cause they were just strings sent from the server).

What I did was simply to send the dates as UNIX timestamps from the server, than read those in as date-objects in javascript. And since it’s just numbers the sorting works magnificently too. The code for handling timestamps sent from the server looks like this:

This of cause outputs the date in Norwegian format, however, adding formatting that could be modified in config was not something that I had to do, as our instance of Filemanager will simply be used in Norwegian. If one wanted to globalize, the simplest thing would probably be to include something like datejs and setting the format in lang-files.

There were a few other minor modifications I did (like remove a column, make clicks on files automatically select the file instead of go to an info-screen etc.), and all of them can be found in my filemanager.js-file. Also, most of the changes I did can be toggled on and of in the settings-file by setting new settings I made. I’m not posting my entire config-file (cause it had to be heavily modified to enable our scenario with regards to what folders should be treated at root depending on a lot of things), but most of my changes should be found here. Feel free to use it however you like.