[API Proposal]: JsonSerializerOptions.UseZeroByteReads

[MihaZupan]

Background and motivation

Zero-byte reads are a technique used in our networking stacks (System.Net, ASP.NET, YARP) to defer renting/allocating memory until there's data available on the socket.
This can greatly reduce memory usage in applications with a lot of concurrent operations.
In the case of YARP, we've seen a 10x reduction in process memory consumption when used with WebSockets.

With Json, this would help in cases where the whole object hasn't been read into memory yet.
An API that would benefit greatly is JsonSerializer.DeserializeAsyncEnumerable which can be long-lived and spend a lot of time just waiting on IO.

Starting with .NET 7.0, the streams returned by HttpClient are able to take advantage of this technique.
System.Net.Http.Json extensions are therefore a prime candidate where we should strongly consider enabling this option by default.

Prior art:

• StreamPipeReaderOptions.UseZeroByteReads from System.IO.Pipelines (Add ability to perform zero byte reads in StreamPipeReader #37539)
• SocketTransportOptions.WaitForDataBeforeAllocatingBuffer from Kestrel

@davidfowl I assume this wouldn't do much in ASP.NET since the HttpContext.Body already does zero-byte reads internally, right? That is unless the json implementation would be able to defer allocating its own buffers on top of using ZBRs on the underlying transport.

API Proposal

namespace System.Text.Json;

public sealed class JsonSerializerOptions
{
    // New property on existing type
    public bool UseZeroByteReads { get; set; }
}

API Usage

private static readonly JsonSerializerOptions s_jsonOptions = new() { UseZeroByteReads = true };

public static IAsyncEnumerable<Foo> DeserializeAsync(Stream stream) =>
    JsonSerializer.DeserializeAsyncEnumerable<Foo>(stream, s_jsonOptions);

Alternative Designs

If the json implementation isn't capable of deferring allocations of its own buffers while waiting for more data, the user could achieve similar performance gains by implementing their own Stream wrapper that performed zero-byte reads and passing that to JsonSerializer instead.

[ghost]

Tagging subscribers to this area: @dotnet/area-system-text-json, @gregsdennis
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Background and motivation

Zero-byte reads are a technique used in our networking stacks (System.Net, ASP.NET, YARP) to defer renting/allocating memory until there's data available on the socket.
This can greatly reduce memory usage in applications with a lot of concurrent operations.
In the case of YARP, we've seen a 10x reduction in process memory consumption when used with WebSockets.

With Json, this would help in cases where the whole object hasn't been read into memory yet.
An API that would benefit greatly is JsonSerializer.DeserializeAsyncEnumerable which can be long-lived and spend a lot of time just waiting on IO.

Starting with .NET 7.0, the streams returned by HttpClient are able to take advantage of this technique.
System.Net.Http.Json extensions are therefore a prime candidate where we should strongly consider enabling this option by default.

Prior art:

• StreamPipeReaderOptions.UseZeroByteReads from System.IO.Pipelines (Add ability to perform zero byte reads in StreamPipeReader #37539)
• SocketTransportOptions.WaitForDataBeforeAllocatingBuffer from Kestrel

@davidfowl I assume this wouldn't do much in ASP.NET since the HttpContext.Body already does zero-byte reads internally, right? That is unless the json implementation would be able to defer allocating its own buffers on top of using ZBRs on the underlying transport.

API Proposal

namespace System.Text.Json;

public sealed class JsonSerializerOptions
{
    // New property on existing type
    public bool UseZeroByteReads { get; set; }
}

API Usage

private static readonly JsonSerializerOptions s_jsonOptions = new() { UseZeroByteReads = true };

public static IAsyncEnumerable<Foo> DeserializeAsync(Stream stream) =>
    JsonSerializer.DeserializeAsyncEnumerable<Foo>(stream, s_jsonOptions);

Alternative Designs

If the json implementation isn't capable of deferring allocations of its own buffers while waiting for more data, the user could achieve similar performance gains by implementing their own Stream wrapper that performed zero-byte reads and passing that to JsonSerializer instead.

Risks

YARP always uses zero-byte reads on HTTP bodies. We've seen in the past that some implementations of Stream are not aware of the concept of zero-byte reads, causing them to report premature EOF exceptions when reading 0 bytes.

If the user runs into such a scenario, they should fix that Stream implementation, or turn off UseZeroByteReads.

Author:	MihaZupan
Assignees:	-
Labels:	api-suggestion, area-System.Text.Json
Milestone:	-

[krwq]

@MihaZupan Stream.Read/ReadAsync are documented that zero bytes means EOF (last sentence specifically):
https://learn.microsoft.com/en-us/dotnet/api/system.io.stream.read?view=net-7.0

The total number of bytes read into the buffer. This can be less than the number of bytes allocated in the buffer if that many bytes are not currently available, or zero (0) if the end of the stream has been reached.

If that is no longer true should we change the docs?

IMO we're working correctly here, I think it makes more sense for you to create stream wrapper if you prefer different to default behavior.

Possibly it makes more sense to add that flag to Stream itself?

[stephentoub]

are documented that zero bytes means

The zero here refers to the input, i.e. asking for zero bytes.

[krwq]

@MihaZupan I'm marking this as Future for now. If this is impacting your scenario significantly or if you're planning to contribute please let me know and we can bump priority. We will unlikely have time to address this in 8.0 otherwise.

[MihaZupan]

I suppose another alternative design here would be for System.Text.Json to always use zero-byte reads and there is no API to toggle that behavior.

[MihaZupan]

Any thoughts on this @krwq? We might not need an API review.

[ilmax]

I feel like this doesn't belong to the JsonSerializerOptions. It's something too low level compared to what's in the option IMO. As a user I am expecting to find details about serialization convention, case handling and what not there, not low level details about how the serializer allocates memory while reading from a stream.

My suggestion is to always do zero bytes read and eventually, if you really want to have a switch, implement it via AppContext or something alike.

Just my 2 cent :)

[MihaZupan]

Any thoughts on this @dotnet/area-system-text-json: #77935 (comment)?

[En3Tho]

I agree with @ilmax here. I think if profitable, STJ should just implement this. Zero byte reads are now used in other places in BCL by default.