Stop UTF8 decoding/reencoding in Fortunes platform

[roji]

Here's a hacky attempt to do UTF8 all the way in Fortunes; since PostgreSQL delivers text encoded in UTF8, it's silly to decode it to .NET UTF-16 strings, only to re-encode it back to UTF8 for sending to the web client. tl;dr this seems to bring a 4.19% throughput increase.

Npgsql already supports reading strings as raw binary data:

await using var command = new NpgsqlCommand("SELECT 'hello'", conn);
await using var reader = await command.ExecuteReaderAsync();

await reader.ReadAsync();

// Can just get the string as a byte array, but this allocates the buffer inside the driver:
var bytes = reader.GetFieldValue<byte[]>(0);
Console.WriteLine(Encoding.UTF8.GetString(bytes));

// Can use a stream to avoid allocating the byte array (the stream gets recycled internally):
var buf = new byte[1024];
var stream = reader.GetStream(0);
var len = stream.Read(buf);
Console.WriteLine(Encoding.UTF8.GetString(bytes, 0, len));

This all works pretty well; the one problem is the the Fortunes scenario mandates that we sort the returned message. I'm not sure if there's a built-in way to compare UTF8 strings (as byte[]), maybe it's OK to compare byte (wink wink)? IIRC other scenarios don't have sorting requirements so this may be more appropriate there.

For now I removed sorting from both the baseline and the UTF8 version for an apples-to-apples comparison. 4.19% is maybe a bit disappointing, I'll take a look with perfivew to ensure everything s what it should be.

Results (remember: both are without sortng and so not actual Fortunes results):

db	baseline	change
CPU Usage (%)	36	36	0.00%
Cores usage (%)	994	1,018	+2.41%
Working Set (MB)	45	44	-2.22%
Build Time (ms)	1,414	1,676	+18.53%
Start Time (ms)	437	405	-7.32%
Published Size (KB)	523,872	523,872	0.00%

application	baseline	change
CPU Usage (%)	96	97	+1.04%
Cores usage (%)	2,691	2,705	+0.52%
Working Set (MB)	568	569	+0.18%
Private Memory (MB)	1,318	1,319	+0.08%
Build Time (ms)	2,975	3,116	+4.74%
Start Time (ms)	1,789	260	-85.47%
Published Size (KB)	96,484	96,483	-0.00%
Symbols Size (KB)	45	45	0.00%
.NET Core SDK Version	8.0.100-preview.3.23159.6	8.0.100-preview.3.23159.6

load	baseline	change
CPU Usage (%)	40	41	+2.50%
Cores usage (%)	1,118	1,135	+1.52%
Working Set (MB)	48	49	+2.08%
Private Memory (MB)	363	363	0.00%
Start Time (ms)	0	0
First Request (ms)	101	292	+189.11%
Requests/sec	481,301	501,449	+4.19%
Requests	7,267,478	7,571,817	+4.19%
Mean latency (ms)	1.14	1.06	-7.02%
Max latency (ms)	37.09	20.74	-44.08%
Bad responses	0	0
Socket errors	0	0
Read throughput (MB/s)	624.71	650.86	+4.19%
Latency 50th (ms)	1.00	0.97	-3.00%
Latency 75th (ms)	1.19	1.13	-5.04%
Latency 90th (ms)	1.48	1.37	-7.43%
Latency 99th (ms)	3.78	3.59	-5.03%

/cc @DamianEdwards @davidfowl @ajcvickers
/cc @vonzshik @NinoFloris @Brar

[DamianEdwards]

Wow just the increase from not sorting is surprising (although I guess it shouldn't be).

As for sorting over the Utf8 bytes, I have to imagine that's something we have implemented in the framework somewhere already, or at least a utility we can call into?

[DamianEdwards]

As for the sorting, I asked internally and got an answer from @stephentoub that Message.Span.SequenceCompareTo(other.Message.Span) is what we want.

[roji]

Thanks @DamianEdwards, will do all that tomorrow, and look at the other benchmarks too.

[roji]

New Fortunes version up, with sorting and with direct HTML encoding into our output buffer (no additional copy via stackalloc). This gives us a 6.94% increase, really nice!

Will clean this up and look at the other benchmarks.

db	fortunes_baseline	fortunes_utf8
CPU Usage (%)	36	36	0.00%
Cores usage (%)	999	1,007	+0.80%
Working Set (MB)	46	44	-4.35%
Build Time (ms)	1,747	1,665	-4.69%
Start Time (ms)	428	428	0.00%
Published Size (KB)	523,880	523,880	0.00%

application	fortunes_baseline	fortunes_utf8
CPU Usage (%)	95	96	+1.05%
Cores usage (%)	2,650	2,681	+1.17%
Working Set (MB)	559	576	+3.04%
Private Memory (MB)	1,308	1,326	+1.38%
Build Time (ms)	3,389	3,086	-8.94%
Start Time (ms)	1,714	265	-84.54%
Published Size (KB)	96,481	96,480	-0.00%
Symbols Size (KB)	45	45	0.00%
.NET Core SDK Version	8.0.100-preview.3.23159.20	8.0.100-preview.3.23159.20

load	fortunes_baseline	fortunes_utf8
CPU Usage (%)	39	41	+5.13%
Cores usage (%)	1,096	1,143	+4.29%
Working Set (MB)	48	49	+2.08%
Private Memory (MB)	363	363	0.00%
Start Time (ms)	0	0
First Request (ms)	103	252	+144.66%
Requests/sec	467,291	499,716	+6.94%
Requests	7,055,662	7,545,559	+6.94%
Mean latency (ms)	1.14	1.07	-6.14%
Max latency (ms)	25.11	26.89	+7.09%
Bad responses	0	0
Socket errors	0	0
Read throughput (MB/s)	606.52	648.61	+6.94%
Latency 50th (ms)	1.03	0.96	-6.80%
Latency 75th (ms)	1.24	1.15	-7.26%
Latency 90th (ms)	1.56	1.42	-8.97%
Latency 99th (ms)	3.80	3.82	+0.53%

[roji]

Silly me, I forgot that Fortunes is the only benchmark involving strings.

Cleaned this PR up, this should be ready for merging.

[DamianEdwards]

Nice improvement! I'll rebase my Razor PR on these changes and hopefully the improvement sticks 😄

[roji]

@DamianEdwards did a bit more cleanup, holding byte[] directly rather than Memory<Span>.

FYI I'm getting lots of variance in benchmark results: I just got 523k RPS (+13.28%), whereas in a previous run before this change I got only +2%... Hopefully there isn't something screwing up our stability here /cc @sebastienros.

But in any case this is definitely an improvement, regardless of exactly how much. Will go ahead and merge.

[DamianEdwards]

FYI I'm getting lots of variance in benchmark results: I just got 523k RPS (+13.28%), whereas in a previous run before this change I got only +2%... Hopefully there isn't something screwing up our stability here /cc @sebastienros.

I've found this too for the Fortunes benchmark when doing the Razor Slices PR. I agree it would be nice to understand why it's so noisy as like you saw, sometimes you can get a really good run that is more than the value of the improvement you're trying to achieve, which makes it harder to judge if it's really better or not.