Enable SIMD for JSON API

[terrajobst]

From @hez2010 on March 9, 2019 9:19

Is your feature request related to a problem? Please describe.

Feature request. Enable SIMD for the new Json APIs in ASP.NET Core 3.

Describe the solution you'd like

See SimdJsonSharp
With SIMD support, SimdJsonSharp is faster than any existing Json library on .NET.
I think the new Json APIs in ASP.NET Core 3 should add SIMD support to improve its performance.

Copied from original issue: dotnet/aspnetcore#8366

[terrajobst]

Hey @hez2010 that's a great suggestion. We're in the middle of providing a new perf oriented set of JSON APIs for .NET Core 3.0.

@ahsonkhan, what's your take on this?

[Tornhoof]

My personal take on this:
If you take a look at @EgorBo's SimdJsonSharp and the available benchmarks the structural symbol tape concept really shines the larger the data gets, especially if it is pretty-printed (i.e. much whitespace between relevant symbols). If that's not the case, e.g. no pretty-printing, most of the time is spent in parsing of the more complex data types. That's visibile in the benchmark, a simpler, "not-as-valid" double parser is easily ten times faster than the corefx version.
Still I expect SimdJsonSharp (especially if it's further optimized) to be twice as fast (or more) for data sizes of 1kb and larger, compared to the "normal" approach of a byte-per-byte parser.

[terrajobst]

I think we'll never have the option to be forgiving in parsing in the name of perf, due to security concerns. However, I don't see a reason why we shouldn't be speeding up the implementation using SIMD.

[filipnavara]

SimdJsonSharp does full validation, including checking for valid UTF-8. However, it is designed around parsing large chunks and may need to be adjusted to provide something more "streamable" and memory efficient in addition to being fast. I recommend reading the papers about original SimdJson to figure out what it does and how it accomplished it.

[NinoFloris]

Any news on this?

[tannergooding]

I'd defer to @ahsonkhan for his stance; but I would expect we aren't going to look into this for the 3.0 timeframe. Instead, 3.0 will likely aim towards getting the new API shipped and stable and perf improvements (like this) will be investigated for a future release.

[ahsonkhan]

but I would expect we aren't going to look into this for the 3.0 timeframe. Instead, 3.0 will likely aim towards getting the new API shipped and stable and perf improvements (like this) will be investigated for a future release.

Precisely.

The SIMD work definitely looks promising and we should definitely investigate how we can incorporate it into the implementation of System.Text.Json APIs. I prefer we defer this work until after we have locked the API design and features for 3.0. If anyone wants to tackle parts of this or work on prototypes to incorporate some of the SIMD effort with the existing Utf8JsonReader, that would definitely be appreciated!

[RobertHenry6bev]

See the paper https://arxiv.org/pdf/1902.08318.pdf on ideas for using SIMD

See the git repo https://github.com/simdjson/simdjson

[eiriktsarpalis]

Next step would be to identify concrete vectorization opportunities in the codebase. STJ already uses vectorization indirectly via the encoding routines in System.Text.Encodings.Web and the Span helper methods.

[layomia]

Triage: we should also assess prior art from other serializers and how they utilize SIMD.