Implement a heuristic for using fast-path serialization in streaming JsonSerializer methods. #78646

eiriktsarpalis · 2022-11-21T17:49:54Z

This PR refactors the root-level serialization methods so that source generated fast-path serialization delegates can be utilized in more places, including object values and in the streaming serialization methods. Specifically for the streaming methods, a heuristic is implemented where fast-path serialization kicks in if the following conditions are satisfied for a given type:

No serialized value has generated a JSON payload exceeding JsonSerializerOptions.DefaultBufferSize / 2 bytes AND
At least 10 serialization operations have already been completed.

This is to prevent excessive buffering since the fast path serializer doesn't support streaming.

Performance

The change shows notable performance improvements in streaming serialization workloads:

WriteJson_LoginViewModel

Method	Job	Toolchain	Mode	Mean	Error	StdDev	Median	Min	Max	Ratio	MannWhitney(3%)	RatioSD	Gen0	Allocated	Alloc Ratio
SerializeToStream	Job-KZZOPZ	main	SourceGen	416.3 ns	14.25 ns	15.84 ns	413.8 ns	387.5 ns	443.4 ns	1.00	Base	0.00	0.0141	152 B	1.00
SerializeToStream	Job-UEGCWD	PR	SourceGen	344.4 ns	32.70 ns	37.65 ns	345.7 ns	294.5 ns	436.7 ns	0.83	Faster	0.09	0.0150	152 B	1.00

WriteJson_LargeStructWithProperties

Method	Job	Branch	Mode	Mean	Error	StdDev	Median	Min	Max	Ratio	MannWhitney(3%)	RatioSD	Gen0	Allocated	Alloc Ratio
SerializeToStream	Job-KZZOPZ	main	SourceGen	768.1 ns	42.52 ns	48.97 ns	761.1 ns	696.2 ns	864.0 ns	1.00	Base	0.00	0.0211	232 B	1.00
SerializeToStream	Job-UEGCWD	PR	SourceGen	393.0 ns	11.23 ns	11.53 ns	388.5 ns	383.7 ns	426.3 ns	0.52	Faster	0.02	0.0151	152 B	0.66

WriteJson_Dictionary

Method	Job	Toolchain	Mode	Mean	Error	StdDev	Median	Min	Max	Ratio	MannWhitney(3%)	RatioSD	Allocated	Alloc Ratio
SerializeToStream	Job-KZZOPZ	main	SourceGen	10.260 μs	1.3123 μs	1.5112 μs	9.767 μs	8.691 μs	13.380 μs	1.00	Base	0.00	152 B	1.00
SerializeToStream	Job-UEGCWD	PR	SourceGen	7.558 μs	0.6635 μs	0.7640 μs	7.557 μs	6.435 μs	9.027 μs	0.75	Faster	0.15	152 B	1.00

WriteJson_ImmutableDictionary

Method	Job	Toolchain	Mode	Mean	Error	StdDev	Median	Min	Max	Ratio	MannWhitney(3%)	RatioSD	Allocated	Alloc Ratio
SerializeToStream	Job-KZZOPZ	main	SourceGen	26.40 μs	1.519 μs	1.688 μs	26.40 μs	22.39 μs	29.12 μs	1.00	Base	0.00	304 B	1.00
SerializeToStream	Job-UEGCWD	PR	SourceGen	12.07 μs	1.120 μs	1.289 μs	11.52 μs	10.69 μs	15.26 μs	0.45	Faster	0.05	152 B	0.50

ghost · 2022-11-21T17:50:08Z

Tagging subscribers to this area: @dotnet/area-system-text-json, @gregsdennis
See info in area-owners.md if you want to be subscribed.

Issue Details

This PR refactors the root-level serialization methods so that source generated fast-path serialization delegates can be utilized in more places, including object values and in the streaming serialization methods. Specifically for the streaming methods, a heuristic is implemented where fast-path serialization kicks in if the following conditions are satisfied for a given type:

No serialized value has generated a JSON payload exceeding JsonSerializerOptions.DefaultBufferSize / 2 bytes AND
At least 10 serialization operations have already been completed.

This is to prevent excessive buffering since the fast path serializer doesn't support streaming.

Author:	eiriktsarpalis
Assignees:	-
Labels:	`area-System.Text.Json`
Milestone:	-

eiriktsarpalis · 2022-11-21T17:50:45Z

FYI @stephentoub @eerhardt @davidfowl

src/libraries/System.Text.Json/tests/System.Text.Json.Tests/Serialization/Stream.WriteTests.cs

...libraries/System.Text.Json/src/System/Text/Json/Serialization/JsonSerializer.Write.Stream.cs

layomia

Minor questions but LGTM overrall.

...ibraries/System.Text.Json/src/System/Text/Json/Serialization/JsonSerializer.Write.Helpers.cs

...System.Text.Json/src/System/Text/Json/Serialization/Metadata/JsonTypeInfoOfT.WriteHelpers.cs

eiriktsarpalis · 2022-11-28T17:22:33Z

We just merged dotnet/performance#2740 adding source gen benchmarks to dotnet/performance. I'm holding off from merging this PR for a few days, to give our performance infrastructure the chance to properly record the performance numbers before this change.

...System.Text.Json/src/System/Text/Json/Serialization/Metadata/JsonTypeInfoOfT.WriteHelpers.cs

…JsonSerializer methods.

…erialization for consistency.

...System.Text.Json/src/System/Text/Json/Serialization/Metadata/JsonTypeInfoOfT.WriteHelpers.cs

BrennanConroy · 2022-12-01T16:36:53Z

...System.Text.Json/src/System/Text/Json/Serialization/Metadata/JsonTypeInfoOfT.WriteHelpers.cs

+                    supportAsync: true);
+
+                using var bufferWriter = new PooledByteBufferWriter(Options.DefaultBufferSize);
+                using var writer = new Utf8JsonWriter(bufferWriter, Options.GetWriterOptions());


Has any thought been put into making the cache use a ConcurrentQueue instead so it can be used in async paths?
Example: https://github.com/dotnet/aspnetcore/blob/main/src/Servers/Kestrel/Transport.Sockets/src/Internal/SocketSenderPool.cs

Given that the perf benefits are fairly modest when doing thread-local caching in the sync methods, my expectation is that any mechanism requiring more elaborate synchronization would likely offer diminishing returns.

The big gain we've seen in our JSON web benchmarks is that by reducing the allocations of the Utf8JsonWriter (aka pooling them) it greatly reduces the working set of the application, from over 400 MB to ~60 MB for a test run doing millions of RPS.

The big gain we've seen in our JSON web benchmarks is that by reducing the allocations of the Utf8JsonWriter (aka pooling them) it greatly reduces the working set of the application, from over 400 MB to ~60 MB for a test run doing millions of RPS.

Do we know how much of that would also be achieved by changing GC configuration to make it more aggressive rather than being ok expanding to the memory available?

@BrennanConroy could we do a run of the JSON middleware benchmark with Workstation GC to compare?

Could you share the benchmark code? If run in .NET 7, I would expect any perf issues related to Utf8JsonWriter allocations to have dissipated because of #73338.

https://github.com/aspnet/Benchmarks/blob/main/src/Benchmarks/Middleware/JsonMiddleware.cs#L56
It's using the Stream methods which weren't affected until this PR.

What about the variant that caches Utf8JsonWriter instances?

Still streaming API, just the synchronous one
aspnet/Benchmarks#1772

The Utf8JsonWriter overload is not doing streaming serialization, it will try to write everything to the destination buffer in one go. Like I said, this is not an apples-to-apples comparison, streaming mode requires more work and calls into a "slow path" for converters that do support streaming.

Using Utf8JsonWriter instead of Stream is still a valid approach, however it does trade off performance for small vs. large JSON payloads. Hopefully the changes implemented in this PR can let you have the best of both worlds by just calling into the streaming APIs (provided you're using source gen).

The recursion would happen if a gshared type would contain a recursive reference to it. It was triggered by the JsonTypeInfo:JsonTypeInfo<Queue<T>> field added by dotnet#78646. Fixes dotnet#79279.

The recursion would happen if a gshared type would contain a recursive reference to it. It was triggered by the JsonTypeInfo:JsonTypeInfo<Queue<T>> field added by #78646. Fixes #79279.

eiriktsarpalis requested a review from steveharter November 21, 2022 17:49

dotnet-issue-labeler bot added the area-System.Text.Json label Nov 21, 2022

ghost assigned eiriktsarpalis Nov 21, 2022

eiriktsarpalis requested review from layomia and krwq November 21, 2022 17:50

eiriktsarpalis added the tenet-performance Performance related issue label Nov 21, 2022

eiriktsarpalis added this to the 8.0.0 milestone Nov 21, 2022

krwq reviewed Nov 23, 2022

View reviewed changes

src/libraries/System.Text.Json/tests/System.Text.Json.Tests/Serialization/Stream.WriteTests.cs Show resolved Hide resolved

eiriktsarpalis commented Nov 23, 2022

View reviewed changes

src/libraries/System.Text.Json/tests/System.Text.Json.Tests/Serialization/Stream.WriteTests.cs Show resolved Hide resolved

krwq reviewed Nov 23, 2022

View reviewed changes

...libraries/System.Text.Json/src/System/Text/Json/Serialization/JsonSerializer.Write.Stream.cs Show resolved Hide resolved

krwq approved these changes Nov 23, 2022

View reviewed changes

layomia reviewed Nov 24, 2022

View reviewed changes

davidfowl mentioned this pull request Nov 24, 2022

.net 6 aspnet minimal api not using the System.Text.Json source generator? dotnet/aspnetcore#45064

Closed

1 task

eiriktsarpalis mentioned this pull request Nov 28, 2022

System.Text.Json work planned for .NET 8 #77020

Closed

16 tasks

DamianEdwards mentioned this pull request Nov 28, 2022

Investigate options to allow pooling of JSON serialization infrastructure dotnet/aspnetcore#45332

Open

BrennanConroy mentioned this pull request Nov 29, 2022

Update Json middleware benchmark to use source-gen aspnet/Benchmarks#1772

Merged

eiriktsarpalis force-pushed the root-serialization-refactoring branch from 5d53577 to b77139c Compare November 29, 2022 16:14

eerhardt reviewed Nov 30, 2022

View reviewed changes

...System.Text.Json/src/System/Text/Json/Serialization/Metadata/JsonTypeInfoOfT.WriteHelpers.cs Outdated Show resolved Hide resolved

eerhardt reviewed Dec 1, 2022

View reviewed changes

...System.Text.Json/src/System/Text/Json/Serialization/Metadata/JsonTypeInfoOfT.WriteHelpers.cs Show resolved Hide resolved

eiriktsarpalis added 3 commits December 1, 2022 14:45

Implement a heuristic for using fast-path serialization in streaming …

93b659b

…JsonSerializer methods.

Add testing for polymorphic serialization

7ba5175

Apply refactorings to root-level deserialization methods similar to s…

0d4374a

…erialization for consistency.

eiriktsarpalis force-pushed the root-serialization-refactoring branch from b77139c to 7546782 Compare December 1, 2022 15:28

Use pooled Utf8JsonWriter in fast-path async serialization

7546782

eerhardt reviewed Dec 1, 2022

View reviewed changes

...System.Text.Json/src/System/Text/Json/Serialization/Metadata/JsonTypeInfoOfT.WriteHelpers.cs Show resolved Hide resolved

BrennanConroy reviewed Dec 1, 2022

View reviewed changes

This was referenced Dec 1, 2022

System.Data.OleDb.Test - timeout and hangs #74488

Open

System.IO.Pipes.Tests.NamedPipeTest_CurrentUserOnly_Windows.Connection_UnderDifferentUsers_BehavesAsExpected test failure #79052

Closed

Only record streaming serialization sizes on successful operations

2e5cda6

eiriktsarpalis merged commit 3559d33 into dotnet:main Dec 5, 2022

eiriktsarpalis deleted the root-serialization-refactoring branch December 5, 2022 16:15

dakersnar mentioned this pull request Dec 6, 2022

Regressions in System.Text.Json.Serialization.Tests.WriteJson<Int32> #79292

Closed

vargaz mentioned this pull request Dec 6, 2022

[mono][aot] Fix infinite recursion in the AOT dwarf emitter. #79310

Merged

dotnet locked as resolved and limited conversation to collaborators Jan 5, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Implement a heuristic for using fast-path serialization in streaming JsonSerializer methods. #78646

Implement a heuristic for using fast-path serialization in streaming JsonSerializer methods. #78646

eiriktsarpalis commented Nov 21, 2022 •

edited

ghost commented Nov 21, 2022

eiriktsarpalis commented Nov 21, 2022

layomia left a comment

eiriktsarpalis commented Nov 28, 2022

BrennanConroy Dec 1, 2022

eiriktsarpalis Dec 1, 2022

DamianEdwards Dec 1, 2022

stephentoub Dec 2, 2022

DamianEdwards Dec 2, 2022

eiriktsarpalis Dec 5, 2022

BrennanConroy Dec 5, 2022

eiriktsarpalis Dec 5, 2022

BrennanConroy Dec 5, 2022

eiriktsarpalis Dec 6, 2022

Implement a heuristic for using fast-path serialization in streaming JsonSerializer methods. #78646

Implement a heuristic for using fast-path serialization in streaming JsonSerializer methods. #78646

Conversation

eiriktsarpalis commented Nov 21, 2022 • edited

Performance

WriteJson_LoginViewModel

WriteJson_LargeStructWithProperties

WriteJson_Dictionary

WriteJson_ImmutableDictionary

ghost commented Nov 21, 2022

eiriktsarpalis commented Nov 21, 2022

layomia left a comment

Choose a reason for hiding this comment

eiriktsarpalis commented Nov 28, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

eiriktsarpalis commented Nov 21, 2022 •

edited