Optimizer hints - Unsafe.Assume

[jkotas]

C/C++ compilers allows passing hints to the optimizer using __assume or similar constructs. It would be useful to have same capability in .NET.

One use case was discussed in https://github.com/dotnet/corefx/issues/5474#issuecomment-172653606:

    Unsafe.Assume((p & 0x1F) == 0);
    Unsafe.Write<Vector<double>>(p, v);

category:proposal
theme:optimization
skill-level:expert
cost:large

[jkotas]

cc @CarolEidt

[CarolEidt]

It would be great to design such a feature to capture both the assertion about the expected behavior as well as allowing that behavior to be validated. That is, perhaps in Debug one might check the condition, while when generating optimized code (e.g. in a Release build), one might assume it to be true. We've already got Debug.Assert and Trace.Assert. Maybe this should be Unsafe.Assert, and it would check the condition in Debug builds, but assume the condition for optimization purposes in the Release build.

[nietras]

Sounds interesting. How would this be expressed in IL?

[mikedn]

Just like any other method call. It's just that the JIT assumes that the expression passed as argument always evaluates to true and can attempt to infer useful information from that.

That said, the "infer" part is a bit of a can of worms. Any bool expression can be used but only the simplest ones will be useful.

[GSPP]

This can be tremendously useful because it's a way turn off any JIT safety. x != null turns off a null check. index >= 0 && index < array.Length turns off a bounds check.

This design is very extensible because with one API addition more and more assumptions can be made to work in the future.

There should be a small catalog of patterns that are recognized.

(Update: Removed the builtin_expect thought and posted them elsewhere.)

[nietras]

@GSPP elsewhere would then be https://github.com/dotnet/coreclr/issues/6024 just so we have the link :)

[Daniel-Svensson]

Sounds like you want the exising Contract.Assume but without the tooling.

https://msdn.microsoft.com/en-us/library/system.diagnostics.contracts.contract.assume(v=vs.110).aspx

[redknightlois]

@Daniel-Svensson Using Contract.Assume is not a bad idea at all (it is there already and there are many annotations that the JIT could understand if properly stated).

Another interesting use of Assume is hinting that a particular store on an array should be a non-temporal write. Therefore, you can use non-temporal instructions and avoiding polutting the cache if you are going to write and never read that information in that method. For example:

public int[] Sum ( int[] op1, int[] op2 )
{
    var result = new int[op1.Lenght];
    Contract.Assume( Performance.NonTemporal(result) );

    /// here goes result[i] = op1[i] + op2[i]
}

And that can be done to achieve very interesting results

public int[] Sum ( int[] op1, int[] op2 )
{
    if ( op1.Length > 4096 ) // or whatever it makes sense for the kernel
    {
       Contract.Assume( Performance.NonTemporal(op1) );
       Contract.Assume( Performance.NonTemporal(op2) );
    }
    var result = new int[op1.Lenght];
    Contract.Assume( Performance.NonTemporal(result) );

    /// here goes result[i] = op1[i] + op2[i]
}

This is pretty handy optimization because by the time the kernel ends the cache is poluted with only the last batch of data.

[GrabYourPitchforks]

As an extension to this, it might be useful to specify contracts on return values from methods. A strawman:

[return: AssumeNotNull]
public static object MyMethod();

[return: AssumeNonNegative]
public static int MyOtherMethod();

For example, the expression new Span<T>(Array.Empty<T>()) still performs the null check at the beginning of the ctor because the JIT doesn't have enough information to prove that the return value of Array.Empty<T>() is non-null.

[GSPP]

@GrabYourPitchforks The caller can already specify anything like this:

var result = MyMethod();
Assume(Anything(result));

The advantage of this would be that a single unified system (the assume method call) suffices for anything. This would be per callsite customizable as well.

On the other hand, those attributes would help all callsites and can be library provided.

[GSPP]

A related idea would be to automatically analyze methods and discover invariants such as non-null return value. This seems very expensive to do in the JIT. It could be implemented as a tool that can optionally be run on any assembly or application. The tool would produce a file that records all discovered invariants and data flow information. The JIT would consume that data. This could discover all kinds of useful information such as non-nullness, more precise types, integer ranges and constants.

[redknightlois]

The tool would produce a file that records all discovered invariants and data flow information. The JIT would consume that data. This could discover all kinds of useful information such as non-nullness, more precise types, integer ranges and constants.

You don't need an external file, you can bake that into the assembly itself when building.

[tannergooding]

I had provided a similar proposal (these used to be linked, but I guess that was dropped in the migration): #24593

I believe the main difference is that this one is a general Assume(bool expression) which would need to rely on the thing producing IL to not optimize expression to a constant case and for the JIT to recognize specific IL patterns.

While in my proposal, I suggested explicit methods (such as AssumeAlignment(address, alignment)) which could be used to make the intent more obvious. It would also apply to cases like which branch is more likely to be taken or if they are split and a conditional move would be beneficial.

[deeprobin]

The common suggestion is:

Unsafe.Assume((p & 0x1F) == 0);

However, wouldn't it make more sense and be more maintainable in our current architecture that we use methods similar to test libraries:

namespace System.Runtime.CompilerServices {
    public static unsafe partial class Unsafe
    {
        [Intrinsic, NonVersionable]
        public static void AssumeEqual<T>(T first, T second);

        [Intrinsic, NonVersionable]
        public static void AssumeNotEqual<T>(T first, T second);

        [Intrinsic, NonVersionable]
        public static void AssumeNull<T>(T? val);

        [Intrinsic, NonVersionable]
        public static void AssumeNotNull<T>(T? val);

        [Intrinsic, NonVersionable]
        public static void AssumeIsType<T>(object? obj);

        [Intrinsic, NonVersionable]
        public static void AssumeIsAssignableFrom<T>(object? obj);
    }
}

I find the variant that uses a bool as parameter (Unsafe.Assume) not so nice, since the method call itself probably needs special handling (correct me if am wrong).

Alternatively, I would simply find a new C# syntax that is introduced useful.
But that would also require a new IL opcode.

public unsafe int MyUnsafeMethod(int a, int b) {
    assume a == 1;
    // or:
    __assume a == 1;
   
    return a + b;
}

[ayende]

I think that compiler intrinsic is much better option. I would also assume that having specific named methods make it easier to figure out specific scenarios. It also avoids the "any boolean expression goes here, only a few are actually recognized".

The downside is that things like:

• Has specific alignment
• Greater / less than a value

May create a lot of methods (then again, this is self documenting for what is actually supported).
Another benefit here is that you won't need to deal with avoiding constant folding, etc.