Compatibility with System.ValueTuple<...> for the Tuple Layer and Key Space API

[KrzysFR]

Story Time

The current Tuple Layer implementation dates from ~2012-2013 were the only thing that would approximate tuples in the BCL was the dreaded System.Tuple<..>, which are reference types (allocated on the heap!) and also of fixed size.

Most of the layers and recipes for FoundationDB were implemented in Python which has rich support of tuples, and also deals with variable size tuples (more like vectors), with appending and truncating tuples one of the core aspect of writing layers.

The various STuple<..> types and ITuple interface were created to bridge the gap in .NET:

• ITuple supports the "vector" aspect of things, and deal with size, appending, truncating, and so on.
• All the STuple<T1>, STuple<T1, T2., ...> structs are for all the cases were the tuples are created inline and we can prevent heap allocations with some JIT trickery (using generic methods other structs so that the JIT produces dedicated assembly code) and remove most of the allocations.
• ListTuple<..>, SlicedTuple<...>, LinkedTuple<..> each support a specific scenario were vectors are combined, or parsed.

Finally, the Tuple Layer would handle encoding and decoding ITuples into bytes and back.

The Present

The BCL as now given us "real" tuple support via the System.ValueTuple<...> structs and first-class support at the language level to create, passe around and deconstruct tuples. This make all the existing STuple<...> structs redundant. BUT there is two key differences:

• ValueTuple<..> are mutable while STuple<..> are immutable. This is not a big deal, but having readonly struct tuples will be nicer with the readonly struct JIT optimizations.
• ValueTuple<...> are fixed-size in nature, and there is not practical way to treat them as vectors: there IS an internal interface ITuple which has the notion of size, but it is not currently exposed, and there is not API to append or truncate vectors.

This means that we still need to have our own ITuple to work well with FoundationDB Layers, but can at least take advantage of the literal tuple syntax in C# to remove all the STuple.Create("foo", 123) and replace them with nicer ("foo", 123).

One way to do this nicely is to add implicit conversions between all the STuple<...> and ValueTuple<..>, as well as add some overloads that directly take ValueTuples as arguments.

We can also add deconstruction support to existing STuple do better integrate them with moden C# code.

Code like this:

var key = TuPack.Pack(("hello", 123));

var location = new TypedKeySpace<string, int>(....);
var key = location.Keys.Encode(("hello", 123));
(string x, int y) = location.Keys.Decode(key);

Would be transformed into these monstrosity by the compiler

var key = TuPack.Pack<string, int>(
    STuple<string, int>.implicit_cast<ValueTuple<string, int>>(
        new ValueTuple<string, int>("hello", 123)
    )
);

var location = new TypedKeySpace<string, int>(....);
var key = location.Keys.Encode<string, int>(new ValueTuple<string, int>("hello", 123));
string x;
int y;
location.Keys.Decode(key).Deconstruct(out string x, out int y);

Hopefully, all of this would be optimized away by the JIT into very optimized code and the cost of all the conversions and method calls would evaporate.

Issues

We need to take a dependency on the package System.ValueTuple to be able to support ValueTuples in .NET Framework (not sure if required in .NET Core 2.x?). Which means deciding on the lower package version needed.

This could have repercussions on applications that references other third party libraries which also take a dependence on System.ValueTuple but at lower or higher versions.

The latest version is 4.5.0-preview2-26406-04 (such a lovely name), which the stable version of choice could be either 4.3 or 4.4:

• 4.3 has 3.5 M downloads on Nuget while 4.4 has 2.5 M
• 4.5 is still in preview with a few K downloads.

The file history on ValueTuple.cs in corefx does not seem to have any code changes for a while, so the versionning seems to be purely for packaging purpose only.

This means we could target System.ValueTuple v4.3.0 and be compatible with most of the libraries out there.

[KrzysFR]

Tried referencing System.ValueTuple v4.3.0 and 4.3.1 and they both add a tons of unwanted dependencies:

Version 4.4.0 and above are a lot cleaner:

So I guess 4.4 it is!

[KrzysFR]

PR #73

[KrzysFR]

PR has been merged, closing this.