Fast-path with ValueTask slower than Task

[cincuranet]

Hi, I'm trying to understand why synchronous/non-blocking fast-path with ValueTask is slower than the same code with Task? I would expect the numbers to the opposite.

I have two variations. The "ultra fast" is just a simple chain of calls without any await and the last call is simply Task.FromResult or ValueTask.FromResult. This is like the best case scenario. The "await fast" uses again the Task.FromResult or ValueTask.FromResult, but has await (and hence state machines (and the slowdown ratio is surprisingly high)) in the call chain. Complete code is below.

BenchmarkDotNet=v0.13.0, OS=Windows 10.0.19043.1110 (21H1/May2021Update)
AMD Ryzen 7 5800X, 1 CPU, 16 logical and 8 physical cores
.NET SDK=5.0.302
  [Host]     : .NET 5.0.8 (5.0.821.31504), X64 RyuJIT
  DefaultJob : .NET 5.0.8 (5.0.821.31504), X64 RyuJIT


|             Method |       Mean |     Error |    StdDev |  Gen 0 | Gen 1 | Gen 2 | Allocated |
|------------------- |-----------:|----------:|----------:|-------:|------:|------:|----------:|
|      UltraFastTask |   6.256 ns | 0.0469 ns | 0.0439 ns | 0.0048 |     - |     - |      80 B |
| UltraFastValueTask |  13.569 ns | 0.0048 ns | 0.0043 ns |      - |     - |     - |         - |
|      AwaitFastTask |  87.562 ns | 0.4107 ns | 0.3641 ns | 0.0334 |     - |     - |     560 B |
| AwaitFastValueTask | 184.202 ns | 0.1788 ns | 0.1585 ns |      - |     - |     - |         - |

Same code, but with explicit NoInlining

|             Method |       Mean |     Error |    StdDev |  Gen 0 | Gen 1 | Gen 2 | Allocated |
|------------------- |-----------:|----------:|----------:|-------:|------:|------:|----------:|
|      UltraFastTask |   8.604 ns | 0.0474 ns | 0.0444 ns | 0.0048 |     - |     - |      80 B |
| UltraFastValueTask |  18.307 ns | 0.0174 ns | 0.0154 ns |      - |     - |     - |         - |
|      AwaitFastTask |  85.770 ns | 0.7832 ns | 0.6115 ns | 0.0334 |     - |     - |     560 B |
| AwaitFastValueTask | 186.558 ns | 0.2391 ns | 0.1997 ns |      - |     - |     - |         - |

For good measure, I tried shortening the call chain (calling method directly on _e, instead of going through _a, _b, _c, _d in the code below) makes the "await fast" faster, but the "ultra fast" is virtually the same.

BenchmarkDotNet=v0.13.0, OS=Windows 10.0.19043.1110 (21H1/May2021Update)
AMD Ryzen 7 5800X, 1 CPU, 16 logical and 8 physical cores
.NET SDK=5.0.302
  [Host]     : .NET 5.0.8 (5.0.821.31504), X64 RyuJIT
  DefaultJob : .NET 5.0.8 (5.0.821.31504), X64 RyuJIT


|             Method |      Mean |     Error |    StdDev |  Gen 0 | Gen 1 | Gen 2 | Allocated |
|------------------- |----------:|----------:|----------:|-------:|------:|------:|----------:|
|      UltraFastTask |  6.268 ns | 0.0786 ns | 0.0613 ns | 0.0048 |     - |     - |      80 B |
| UltraFastValueTask | 13.647 ns | 0.0129 ns | 0.0108 ns |      - |     - |     - |         - |
|      AwaitFastTask | 29.154 ns | 0.1350 ns | 0.1127 ns | 0.0143 |     - |     - |     240 B |
| AwaitFastValueTask | 68.568 ns | 0.0601 ns | 0.0562 ns |      - |     - |     - |         - |

The decimal is just a "random" selection. Same numbers for bool or string.

Thanks for shedding some light on that.

Benchmark code

	[MemoryDiagnoser]
	public class Test
	{
		public const decimal Result = 1234.567m;

		A _a;
		B _b;
		C _c;
		D _d;
		E _e;

		[GlobalSetup]
		public void GlobalSetup()
		{
			_a = new();
			_b = new();
			_c = new();
			_d = new();
			_e = new();
		}

		[Benchmark]
		public decimal UltraFastTask()
		{
			return _a.DoTask(_b, _c, _d, _e).GetAwaiter().GetResult();
		}

		[Benchmark]
		public decimal UltraFastValueTask()
		{
			return _a.DoValueTask(_b, _c, _d, _e).GetAwaiter().GetResult();
		}

		[Benchmark]
		public async Task<decimal> AwaitFastTask()
		{
			return await _a.DoAwaitTask(_b, _c, _d, _e);
		}

		[Benchmark]
		public async ValueTask<decimal> AwaitFastValueTask()
		{
			return await _a.DoAwaitValueTask(_b, _c, _d, _e);
		}
	}

	class A
	{
		public Task<decimal> DoTask(B b, C c, D d, E e) => b.DoTask(c, d, e);
		public async Task<decimal> DoAwaitTask(B b, C c, D d, E e) => await b.DoAwaitTask(c, d, e);
		public ValueTask<decimal> DoValueTask(B b, C c, D d, E e) => b.DoValueTask(c, d, e);
		public async ValueTask<decimal> DoAwaitValueTask(B b, C c, D d, E e) => await b.DoAwaitValueTask(c, d, e);
	}
	class B
	{
		public Task<decimal> DoTask(C c, D d, E e) => c.DoTask(d, e);
		public async Task<decimal> DoAwaitTask(C c, D d, E e) => await c.DoAwaitTask(d, e);
		public ValueTask<decimal> DoValueTask(C c, D d, E e) => c.DoValueTask(d, e);
		public async ValueTask<decimal> DoAwaitValueTask(C c, D d, E e) => await c.DoAwaitValueTask(d, e);
	}
	class C
	{
		public Task<decimal> DoTask(D d, E e) => d.DoTask(e);
		public async Task<decimal> DoAwaitTask(D d, E e) => await d.DoAwaitTask(e);
		public ValueTask<decimal> DoValueTask(D d, E e) => d.DoValueTask(e);
		public async ValueTask<decimal> DoAwaitValueTask(D d, E e) => await d.DoAwaitValueTask(e);
	}
	class D
	{
		public Task<decimal> DoTask(E e) => e.DoTask();
		public async Task<decimal> DoAwaitTask(E e) => await e.DoAwaitTask();
		public ValueTask<decimal> DoValueTask(E e) => e.DoValueTask();
		public async ValueTask<decimal> DoAwaitValueTask(E e) => await e.DoAwaitValueTask();
	}
	class E
	{
		public Task<decimal> DoTask() => Task.FromResult(Test.Result);
		public async Task<decimal> DoAwaitTask() => await Task.FromResult(Test.Result);
		public ValueTask<decimal> DoValueTask() => ValueTask.FromResult(Test.Result);
		public async ValueTask<decimal> DoAwaitValueTask() => await ValueTask.FromResult(Test.Result);
	}

[stephentoub]

Allocation and gen0 collection is relatively cheap, so the allocations occurring for the tasks here is likely not significantly negatively impacting those throughput numbers. The impact of allocation ends up being higher when there's more going on than in a microbenchmark. For a serialized benchmark, I expect Task<T> to be faster. A ValueTask<T> is a discriminated union of a Task<T> and a T, which means it has fields for both, which means copying one is more expensive than copying a Task<T> (that also means a ValueTask<T> takes up more space in a state machine for an async method for which it's awaited). A reference to a Task<T> could be passed back in a register, but a ValueTask<decimal> won't fit in one. ValueTask<T> also doesn't store a Task<T> field; it stores an object field, and it then does a type check at run time to see whether the field is null, a Task<T>, or an IValueTaskSource<T>. That type check adds overhead.

Try changing your benchmarks to actually stress things in parallel to make the GC impact more visible, e.g.

	[Benchmark]
	public int UltraFastTask()
	{
		Parallel.For(0, 1_000_000, i =>
		{
			_a.DoTask(_b, _c, _d, _e).GetAwaiter().GetResult();
		});
		return 0;
	}

	[Benchmark]
	public int UltraFastValueTask()
	{
		Parallel.For(0, 1_000_000, i =>
		{
			_a.DoValueTask(_b, _c, _d, _e).GetAwaiter().GetResult();
		});
		return 0;
	}

I expect you'll see a stark difference. This is what I got when I tried:

Method	Mean	Error	StdDev	Code Size	Gen 0	Gen 1	Gen 2	Allocated
UltraFastTask	6,198.9 us	425.92 us	1,255.83 us	0 KB	11539.0625	46.8750	-	70,325 KB
UltraFastValueTask	896.8 us	17.83 us	35.20 us	0 KB	-	-	-	4 KB

[cincuranet]

When stressing the GC I see expected numbers too.

|             Method |      Mean |     Error |    StdDev |      Gen 0 |    Gen 1 | Gen 2 |  Allocated |
|------------------- |----------:|----------:|----------:|-----------:|---------:|------:|-----------:|
|      UltraFastTask |  2.828 ms | 0.0887 ms | 0.2617 ms |  4792.9688 |  35.1563 |     - |  78,135 KB |
| UltraFastValueTask |  1.617 ms | 0.0105 ms | 0.0093 ms |          - |        - |     - |       5 KB |

I also looked at the disassembly and the type checks (together with more work on xmmX registers) for ValueTask make the code quite a bit longer.

    L0000: push rdi
    L0001: push rsi
    L0002: push rbx
    L0003: sub rsp, 0xb0
    L000a: vzeroupper
    L000d: vxorps xmm4, xmm4, xmm4
    L0011: vmovdqa [rsp+0x20], xmm4
    L0017: vmovdqa [rsp+0x30], xmm4
    L001d: vmovdqa [rsp+0x40], xmm4
    L0023: vmovdqa [rsp+0x50], xmm4
    L0029: vmovdqa [rsp+0x60], xmm4
    L002f: mov rsi, rdx
    L0032: mov rdx, [rcx+8]
    L0036: mov rax, [rcx+0x10]
    L003a: mov r8, [rcx+0x18]
    L003e: mov r9, [rcx+0x20]
    L0042: mov rcx, [rcx+0x28]
    L0046: cmp [rdx], edx
    L0048: cmp [rax], eax
    L004a: cmp [r8], r8d
    L004d: cmp [r9], r9d
    L0050: cmp [rcx], ecx
    L0052: lea rdx, [rsp+0x60]
    L0057: mov dword ptr [rdx], 0x30000
    L005d: xor ecx, ecx
    L005f: mov [rdx+4], ecx
    L0062: mov qword ptr [rdx+8], 0x12d687
    L006a: mov [rsp+0x50], rcx
    L006f: mov byte ptr [rsp+0x5a], 1
    L0074: mov word ptr [rsp+0x58], 0
    L007b: vmovdqu xmm0, [rsp+0x50]
    L0081: vmovdqu [rsp+0x90], xmm0
    L008a: vmovdqu xmm0, [rsp+0x60]
    L0090: vmovdqu [rsp+0xa0], xmm0
    L0099: vmovdqu xmm0, [rsp+0x90]
    L00a2: vmovdqu [rsp+0x30], xmm0
    L00a8: vmovdqu xmm0, [rsp+0xa0]
    L00b1: vmovdqu [rsp+0x40], xmm0
    L00b7: vmovdqu xmm0, [rsp+0x30]
    L00bd: vmovdqu [rsp+0x70], xmm0
    L00c3: vmovdqu xmm0, [rsp+0x40]
    L00c9: vmovdqu [rsp+0x80], xmm0
    L00d2: mov rdi, [rsp+0x70]
    L00d7: test rdi, rdi
    L00da: jne short L00ed
    L00dc: vmovdqu xmm0, [rsp+0x80]
    L00e5: vmovdqu [rsp+0x20], xmm0
    L00eb: jmp short L0151
    L00ed: mov rdx, rdi
    L00f0: mov rcx, 0x7ffc34225f28
    L00fa: call System.Runtime.CompilerServices.CastHelpers.IsInstanceOfClass(Void*, System.Object)
    L00ff: mov rbx, rax
    L0102: test rbx, rbx
    L0105: je short L012d
    L0107: mov ecx, [rbx+0x34]
    L010a: and ecx, 0x11000000
    L0110: cmp ecx, 0x1000000
    L0116: je short L0120
    L0118: mov rcx, rbx
    L011b: call System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(System.Threading.Tasks.Task)
    L0120: vmovdqu xmm0, [rbx+0x38]
    L0125: vmovdqu [rsp+0x20], xmm0
    L012b: jmp short L0151
    L012d: lea rdx, [rsp+0x20]
    L0132: mov rcx, rdi
    L0135: movsx r8, word ptr [rsp+0x78]
    L013b: mov r11, 0x7ffc34847020
    L0145: mov rax, 0x7ffc34847020
    L014f: call qword ptr [rax]
    L0151: vmovdqu xmm0, [rsp+0x20]
    L0157: vmovdqu [rsi], xmm0
    L015b: mov rax, rsi
    L015e: add rsp, 0xb0
    L0165: pop rbx
    L0166: pop rsi
    L0167: pop rdi
    L0168: ret

Thanks for a great answer.