Home > Uncategorized > Adventures in Roslyn: Adding crazily powerful operator overloading to C# 6

Adventures in Roslyn: Adding crazily powerful operator overloading to C# 6

I’m going to show you how to enable a new kind of operator overloading by adding exactly four (4) lines of code to a single file in the C# 6 compiler preview. Yes, I was surprised too!

After seeing the video of Anders Hejlsberg showing how easy it is to hack the new open source C# compiler, I had to give it a try.

My aim was (I assumed) a lot more ambitious and crazy than his demo. I thought it would take ages to figure out. But it was still tempting to aim high and actually implement a substantial new feature, because there are a few I’ve been wondering about over the years.

Ever since LINQ query syntax was added to the language, I’ve wished that operator overloading worked the same way. The where keyword gets turned into a call to a Where method. And it doesn’t matter where or how that method is defined. It can be an extension method, an ordinary method, a virtual method. In fact there are few other language features that map to well known member names: a collection initializer is turned into several calls to a method called Add, and the await keyword expands into calls to several methods.

So my use case is very simple. Suppose I have a couple of sequences of values:

var nums1 = new[] { 1, 2, 3 };
var nums2 = new[] { 4, 5 };

var nums3 = nums1 + nums2;

That last line, which I’d like to concatenate the two sequences, simply isn’t going to work because operator + is not defined on IEnumerable. Nor is there a way to make it work for all sequences in standard C#. That would require the ability to implement an operator overload using extension methods! Such a thing does not exist. But it would be pretty useful.

Suppose if the compiler couldn’t find a standard meaning for +, it tried looking for a method available the left-hand-side value called Addition. (NB. Add is already taken by collection initializers as I previously noted).

public static class EnumerableOperatorExtensions
    public static IEnumerable<T> Addition<T>(this IEnumerable<T> left, IEnumerable<T> right)
        return left.Concat(right);

Of course, Concat is already there to do the real work for us: the above incantation just makes it available under a standardised name.

So let’s get to work. To play along at home, download the Roslyn source, read the instructions for building/debugging, get all the prerequisites (the instructions seem to be flawless as far as I can tell), and make sure you’re able to build and hit F5 to bring up Visual Studio. You’ll find you can set breakpoints and they will be hit (from multiple threads) as VS2013 compiles code on the fly as you edit it, to provide intellisense, etc.

The first thing I had to do was find those well-known member names, such as Where. Obviously it wouldn’t be that easy, but I tried a simple search for the quoted string "Where"… Oh, turns out it really is that easy!

This is the first hit:

void ReduceWhere(WhereClauseSyntax where, QueryTranslationState state, DiagnosticBag diagnostics)
    // A query expression with a where clause
    //     from x in e
    //     where f
    //     ...
    // is translated into
    //     from x in ( e ) . Where ( x => f )
    var lambda = MakeQueryUnboundLambda(state.RangeVariableMap(), state.rangeVariable, where.Condition);
    var invocation = MakeQueryInvocation(where, state.fromExpression, "Where", lambda, diagnostics);
    state.fromExpression = MakeQueryClause(where, invocation, queryInvocation: invocation);

That MakeQueryInvocation looked intriguing. It calls onto another helper called MakeInvocationExpression, which takes a receiver for the method call, a method name and an immutable array of arguments, and is commented as:

// Helper method to create a synthesized method invocation expression.

On searching for calls to it, as you’d expect, I found it being used for collection initializers and await in exactly the same way. All I needed was to find a spot in the binding of operators where we’re just about to give up and emit an error, and then try MakeInvocationExpression.

The next part I did with a mixture of searching for likely words in the source and then setting breakpoints to see if they got hit. Eventually I found a method Binder.BindSimpleBinaryOperator in the file Binder_Operator.cs. Actually there are two overloads of it: the four-argument overload does the real work. (The two-argument overload is just a wrapper that avoids too much recursion when dealing with chained operators by implementing its own stack.)

Anyway, it works by calling another helper, BinaryOperatorOverloadResolution, which implements the standard C# rules, and then it checks if it worked:

if (!best.HasValue)
    resultOperatorKind = kind;
    resultType = CreateErrorType();
    hasErrors = true;

That’s where it gives up! So that’s where we need MOAR CODE:

if (!best.HasValue)
    string methodName = Enum.GetName(typeof(BinaryOperatorKind), kind);                
    var methodCall = MakeInvocationExpression(node, left, methodName, ImmutableArray.Create(right), diagnostics);
    if (methodCall != null && !methodCall.HasAnyErrors)
        return methodCall;

    resultOperatorKind = kind;
    resultType = CreateErrorType();
    hasErrors = true;

Look how damn lazy I was. The enum BinaryOperatorKind defines Addition, Subtraction, etc., so I just get the string name of the value to use as the method name. If MakeInvocationExpression seems to have worked, I return the result.

But I was also quite careful. By ensuring the standard is followed first, and the new behaviour only kicks in for code that would otherwise be malformed, I don’t change the meaning of existing programs.

And that’s it. Here’s a look at what happens in Visual Studio when I run it and enter my test case, but first without defining an extension method:

Rosyln shows error message

Note the error message! It’s telling us we need to write an Addition method that takes one argument. In the intellisense! I didn’t have to do anything in particular to make that happen.

Then when we add the declaration of the extension method:

Rosyln shows no error message

The red squiggle has gone, and num3 has the right type. And when I hit F5, I see the expected concatenated output.

I am astonished.

Here’s a fork with this small change.

There is still more to investigate. For example:

interface IThing
    IThing Addition(IThing other);
    IThing Subtraction(IThing other);

static void Test<T>(T a, T b) where T : IThing
    var r1 = a + b;
    var r2 = a - b;

That works! Oh yes, you can do operators on generic parameter types. Couldn’t do that before.

However, what about ==? That doesn’t work – it seems the compiler handles equality comparison separately, and doesn’t get to my new code. Maybe that’s a good thing… But on the other hand, maybe not. Will take another look tomorrow.

Categories: Uncategorized Tags:
  1. April 25, 2014 at 8:05 am

    Wouldn’t it be better to directly compile the + to the Cast instead of Addition? That would autoamtically map to Enumerable.Concat in case System.Linq is included.

    • earwicker
      April 25, 2014 at 4:56 pm

      Not sure what you mean. How would a cast map to Concat?

      Just to be clear, the idea is to allow general operator overloading without pre-specifying a meaning, e.g. addition, subtraction, multiply, divide, all these are allowed, and can be defined to do whatever makes sense for a type. I’m not trying to solve the one specific problem “How can I make + mean Concat?”. That’s just an example. The aim is to generally allow the programmer to define the meaning of +, -, *, /, etc, on types that they didn’t author via extension methods, or via virtual methods overridden in derived classes, etc.

      • Steven
        April 25, 2014 at 4:59 pm

        Oops! I meant Concat of course; not Cast. That would be silly.

  2. June 2, 2014 at 12:45 pm

    It is definitely cool and allows to evaluate long-awaited ideas. But how would you use it in production project? May be pack your compiler together with code.. or share as compiler extension somehow.. Any ideas?

    • earwicker
      June 24, 2014 at 9:17 pm

      I haven’t looked at using it in production. Personally I wouldn’t! The disadvantage of ending up with a code base that depends on a special compiler I have to maintain myself is likely to outweigh the advantage of any one language feature (also the features I covered in these blog posts are pretty half-baked).

  3. July 10, 2014 at 10:36 am

    Hello Daniel!
    I’ve tried to follow your post in code, but something wrong happens. Could I contact you by email or skype, and clarify how do you debug and modify Roslyn codebase?

    • earwicker
      July 30, 2014 at 9:59 pm

      Hi, I just followed the instructions on Codeplex and found that they worked perfectly. If you are still experiencing problems I’d suggest asking a question on Stack Overflow – I may try to answer it! Or you might get an answer from a *real* expert.

  1. No trackbacks yet.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: