Add CopyString and ValueIsEscaped APIs to Utf8JsonReader

[eiriktsarpalis]

This PR

• Implements the Utf8JsonReader.ValueIsEscaped property. Fix Add Utf8JsonReader.ValueIsEscaped property #45167.
• Implements the UtfJsonReader.CopyString methods. Fix Support non-allocating view on string properties values of JSON in System.Text.Json.Utf8JsonReader #54410.
• Rewires internal converter implementations to use the new APIs, where applicable.
• Reworks the Unescape implementation to better take advantage of the Span.IndexOf and CopyTo methods, resulting in improved performance for certain scenaria because of vectorization.

The changes improve performance in a few applications, most notably when reading escaped strings or when deserializing converters that would until now allocate strings.

String-based enum deserialization without escaping

Method	Branch	Mean	Error	StdDev	Median	Min	Max	Ratio	MannWhitney(5%)	RatioSD	Gen 0	Allocated	Alloc Ratio
DeserializeFromUtf8Bytes	main	200.1 ns	9.58 ns	11.03 ns	193.7 ns	190.5 ns	221.4 ns	1.00	Base	0.00	0.0128	136 B	1.00
DeserializeFromUtf8Bytes	PR	172.9 ns	1.14 ns	0.95 ns	172.9 ns	171.6 ns	175.2 ns	0.86	Faster	0.05	-	-	0.00

Utf8JsonReader.GetString() call on escaped string

Method	Branch	Mean	Error	StdDev	Median	Min	Max	Ratio	MannWhitney(5%)	Gen 0	Allocated	Alloc Ratio
GetString	main	10.643 us	0.1048 us	0.0929 us	10.627 us	10.527 us	10.848 us	1.00	Base	1.1797	11.72 KB	1.00
GetString	PR	7.678 us	0.0615 us	0.0546 us	7.658 us	7.615 us	7.765 us	0.72	Faster	1.1651	11.72 KB	1.00

cc @davidfowl @bartonjs @steveharter @stephentoub

[dotnet-issue-labeler]

Note regarding the new-api-needs-documentation label:

This serves as a reminder for when your PR is modifying a ref *.cs file and adding/modifying public APIs, to please make sure the API implementation in the src *.cs file is documented with triple slash comments, so the PR reviewers can sign off that change.

[ghost]

Tagging subscribers to this area: @dotnet/area-system-text-json, @gregsdennis
See info in area-owners.md if you want to be subscribed.

Issue Details

---

This PR

• Implements the Utf8JsonReader.ValueIsEscaped property. Fix Add Utf8JsonReader.ValueIsEscaped property #45167.
• Implements the UtfJsonReader.CopyString methods. Fix Support non-allocating view on string properties values of JSON in System.Text.Json.Utf8JsonReader #54410.
• Rewires internal converter implementations to use the new APIs, where applicable.
• Reworks the Unescape implementation to better take advantage of the Span.IndexOf and CopyTo methods, resulting in improved performance for certain scenaria because of vectorization.

The changes improve performance in a few applications, most notably when reading escaped strings or when deserializing converters that would until now allocate strings.

String-based enum deserialization without escaping

Method	Branch	Mean	Error	StdDev	Median	Min	Max	Ratio	MannWhitney(5%)	RatioSD	Gen 0	Allocated	Alloc Ratio
DeserializeFromUtf8Bytes	main	200.1 ns	9.58 ns	11.03 ns	193.7 ns	190.5 ns	221.4 ns	1.00	Base	0.00	0.0128	136 B	1.00
DeserializeFromUtf8Bytes	PR	172.9 ns	1.14 ns	0.95 ns	172.9 ns	171.6 ns	175.2 ns	0.86	Faster	0.05	-	-	0.00

Utf8JsonReader.GetString() call on escaped string

Method	Branch	Mean	Error	StdDev	Median	Min	Max	Ratio	MannWhitney(5%)	Gen 0	Allocated	Alloc Ratio
GetString	main	10.643 us	0.1048 us	0.0929 us	10.627 us	10.527 us	10.848 us	1.00	Base	1.1797	11.72 KB	1.00
GetString	PR	7.678 us	0.0615 us	0.0546 us	7.658 us	7.615 us	7.765 us	0.72	Faster	1.1651	11.72 KB	1.00

Author:	eiriktsarpalis
Assignees:	-
Labels:	area-System.Text.Json, new-api-needs-documentation
Milestone:	-

[krwq]

why not just return false directly? do we need to use goto DestinationTooShort? False return means DestinationTooShort anyway. If you think we might change that in the future perhaps consider:
const bool DestinationTooShort = false; and return DestinationTooShort

[eiriktsarpalis]

It's a micro-optimization. We sometimes do this in hot path methods to minimize the number of instructions being emitted:

https://sharplab.io/#v2:C4LghgzgtgPgAgJgIwFgBQcDMACR2DC2A3utmbjnEgGzYBGA9gwDbYAqAphMABQCWAO2DYAHgBpsg4QE8AlMVLklfAGbYeI7AD5sABlmKlZEmiNncAdmwqwzCBwDchowF9nS9+VXrp2ADx6BqbmJuZGcFY2do6eZG7BZrGW2MAATgCuMQnY8R7ZWLg09EysnNwI/EKiElLYcgrZymoa2oFJoWFkAOYMwAzYAGJgfMxOjeS55knePL4B+u1JSj19g8OjSZOJ42QRKRlZSkMjIEl7UfZjSvEuQA===

It might not be super critical for the unescaper though.

[krwq]

I'd suggest filing issue on improving this rather than making code harder to read

[eiriktsarpalis]

It's fairly pervasive in the hot code paths in both STJ and STEW, so we can look at it later.

[krwq]

We should at least open the issue to track this work