December 11, 2019

C# Loves Code Patterns

This post is part of the third annual C# Advent. Check out the home page for up to 50 C# blog posts in December 2019! Thanks, Matthew D. Groves for organizing it.

You might recall that LINQ has two forms. The first way is to call LINQ methods and the second is the special LINQ syntax. Here’s an example of the LINQ syntax.

var myExampleData = new[] {1, 2, 3, 4, 5, 6, 7, 8, 9};
var oddNumbersAsString = from n in myExampleData
    where n % 2 == 0
    select n.ToString();

This is the same as calling the LINQ methods:

var myExampleData = new[] {1, 2, 3, 4, 5, 6, 7, 8, 9};
var oddNumbersAsString = myExampleData.Where(n => n % 2 == 0)
    .Select(n => n.ToString());

How are these two ways related? Maybe you need to implement IEnumberable<T> or IQueryble<T> to support the LINQ methods and syntax? When the C# compiler encounters the LINQ syntax it translates it into a series of method calls for each keyword. For example the LINQ where corresponds to a Where(Func<AnyType, bool> predicate) or a Where(Expression<Func<AnyType, bool>> predicate). As long as such a method exists, no matter where it is defined, the query works. You can define the methods on the class/struct, it can be generic or not, inherited from an interface or provided as an extension method.

For example, this compiles and runs:

public class MyFakeLinq
{
    public MyFakeLinq Where(Func<int, bool> predicate)
    {
        // TODO: Implement something reasonable ;)
        Console.Out.WriteLine("Where called");
        return this;
    }
}

// Later
var myFakeLinq = new MyFakeLinq();
var result = from m in myFakeLinq
    where m > 2
    select m;

Notice that in this minimal example the code compiles despite no having a .Select() method. That is because the select doesn’t do anything, so the C# compiler ommits a ,Select() call. As soon as the select does work there needs to be a Select(Func<AnyType, TResult> selector).

Fails To Compile:
var myFakeLinq = new MyFakeLinq();
var result = from m in myFakeLinq
    where m > 2
    select m.ToString();
^^^^^^
Program.cs(59, 36): [CS1936] Could not find an implementation of the query pattern for source type 'Program.MyFakeLinq'.  'Select' not found.

Adding a Select method fixes the issue. Note, in this minimal example select isn’t generic. It only supports returning strings. If you don’t need to support arbitrary return types you can skip a generic Select method.

public class MyFakeLinq
{
    public MyFakeLinq Where(Func<int, bool> predicate)
    {
        // TODO
        Console.Out.WriteLine("Basic Where");
        return this;
    }
    public string Select(Func<int, string> selection)
    {
        // TODO
        Console.Out.WriteLine("Basic Select");
        return "";
    }
}

var myFakeLinq = new MyFakeLinq();
var result = from m in myFakeLinq
    where m > 2
    select m.ToString();

In conclusion, when you are using the LINQ syntax the C# compiler creates a series of method calls. As long as these methods are found you are good. Or course most of the time you inherit IEnumberable or IQueryble interface and use the existing implementation.

Here is a overview of the translation:

Keyword Method

from

After 1st one: .SelectMany

where

.Where

select

.Select

group

.GroupBy

orderby

.OrderBy and more arguments to .ThenBy

join

.Join

let

No method, just names variables

The Wise C# Compiler LINQ’s up with his Students
Figure 1. The Wise C# Compiler LINQ’s up with his Students

Foreach is also a Pattern

That LINQ is a series of method calls is maybe not a surprise, but LINQ is not alone. The very basic foreach keyword behaves similarly. You might expect that you need to implement the IEnumerable<T> interface to support foreach, but you don’t. The target object needs a .GetEnumerator() method and the returned object then needs a .Current property and a .MoveNext() method. As soon as these methods are available the foreach keyword works.

class Moves // Note no IEnumerable
{
    public MovesEnumerator GetEnumerator() => new MovesEnumerator(5);

    public struct MovesEnumerator // Note no IEnumerator
    {
        private int movesLeft;

        public MovesEnumerator(int movesLeft)
        {
            this.movesLeft = movesLeft;
        }

        public bool MoveNext()
        {
            var moved = 0 < movesLeft;
            if (moved)
            {
                movesLeft--;
            }
            return moved;

        }

        public int Current
        {
            get { return movesLeft; }
        }
    }
}

var example = new Moves();
foreach (var e in example)
{
    Console.Out.WriteLine("Moves left: " + e);
}
// Result:
// Moves left: 4
// Moves left: 3
// Moves left: 2
// Moves left: 1
// Moves left: 0

foreach has another trick up its sleeves. When the Enumerator implements the IDisposable interface, the enumerator is disposed after the foreach block. So foreach also acts like using. This is useful when the application iterates over data from an external source like a file or a database.

public struct MovesEnumerator : Disposable { // Note no IEnumerator
    // ...
    public void Dispose()
    {
        Console.Out.WriteLine("Release some important resource when done iterating");
    }
}

Also you can have multiple .GetEnumerator() methods. For example to have a very specific .GetEnumerator() method returning a concrete class/struct for your type. And a .GetEnumerator() implementation for the IEnumerable interface. This allows avoiding interface calls for better performance when the concrete type is known. For example the List<T> and other collections are using this trick:

List.cs
public Enumerator GetEnumerator()
    => new Enumerator(this);

IEnumerator<T> IEnumerable<T>.GetEnumerator()
    => new Enumerator(this);

IEnumerator IEnumerable.GetEnumerator()
    => new Enumerator(this);
The Wise C# Compiler Helps his Students for each Step
Figure 2. The Wise C# Compiler Helps his Students for each Step

Yes, Async is also a Pattern

Again, you might think that await' only works on `Task`s. Yet again, the only thing required is a `GetAwaiter method which returns an object having an `IsComplete' and 'GetResult()' method and implementing the INotifyCompletion interface. Here’s an example:

public static MyAwaitable AsyncMethod() => new MyAwaitable();
internal class MyAwaitable
{
    public MyAwaiter GetAwaiter() => new MyAwaiter();

    internal struct MyAwaiter : INotifyCompletion
    {
        public bool IsCompleted { get; private set; }

        public int GetResult() => 42;

        public void OnCompleted(Action continuation)
        {
            Console.Out.WriteLine("OnComplete callback added");
            // TODO: Once the async operation completes we the continuation should be called.
            IsCompleted = true;
            continuation.Invoke();
        }
    }
}

var result = await AsyncMethod();
Console.Out.WriteLine("Result: " + result);
// Result:
// OnComplete callback added
// Result: 42
The Wise C# Compiler Helps Waiting
Figure 3. The Wise C# Compiler Helps Waiting

Dynamic Type

Yes, you guessed rights, using dynamic objects in C# also boils down to having a specific method available. In this case, it is a bit more complicated and I’m not going into the details. The C# compiler invokes a whole reflection machinery which by default can invoke methods, properties, and fields. As soon as the object implements IDynamicMetaObjectProvider the object gets runtime control on what is invoked.Most of the time you want to inherit from DyanmicObject which implements IDynamicMetaObjectProvider and provides more convenient methods to do a fully dynamic object.

class MyDynamic : DynamicObject // could be IDynamicMetaObjectProvider, but that is more work
{
    public override bool TryInvokeMember(InvokeMemberBinder binder, object[] args, out object result)
    {
        // Print out the method name and the arguments
        Console.Out.WriteLine(binder.Name + "(" + string.Join(',',args) + ")");
        result = this;
        return true;
    }
}

dynamic dyn = new MyDynamic();
dyn.Hello();
dyn.Hi().There();
// Result:
// Hello()
// Hi()
// There()
The Wise C# Compiler Goes CRAAAZY!!!
Figure 4. The Wise C# Compiler Goes CRAAAZY!!!

Conclusion

Some syntactic sugar and advanced features of C# are translated into a pattern of method calls. This allows you to take full advantage of these C# features for your own specific classes. In most cases already implemented interfaces like IEnumberable<T>, IQueryable<T>, Task etc. are the best choice. However it is good to know that you can go one level deeper and provide your own implementation when required.

Update 12. December 2019

On Reddit from MiffTheFox pointed out that C#7’s destruction to tuples uses the same strategy. I suspected that but ran out of time to double check. I bet there is other cases I’m not even aware of.

Another good comment on Reddit from chucker23n is that the MS compiler development team refers to this as duck typing. I was considering saying that this feels like duck typing, so nice to know that the C# team also uses that term.

Tags: C#