System.Text.Json.JsonException: Reference '{$id}' not found.

[BrunoBlanes]

Description

Exception is thrown when trying to deserialize a Json object serialized using ReferenceHandler.Preserve on Blazor WebAssembly:

Server

services.AddMvc().AddJsonOptions(options =>
{
    options.JsonSerializerOptions.ReferenceHandler = ReferenceHandler.Preserve;
});

Json

{"$id":"1","$values":[{"$id":"2","value":0,"name":"zero","categories":{"$id":"3","$values":[]}},{"$id":"4","value":1,"name":"one","categories":{"$id":"5","$values":[]}},{"$id":"6","value":2,"name":"two","categories":{"$id":"7","$values":[{"$id":"8","name":"Prime","numbers":{"$id":"9","$values":[{"$ref":"6"},{"$id":"10","value":3,"name":"three","categories":{"$id":"11","$values":[{"$ref":"8"}]}}]}}]}},{"$ref":"10"},{"$id":"12","value":4,"name":"four","categories":{"$id":"13","$values":[]}}]}

Client

var response = await Http.GetFromJsonAsync<IList<Number>>("Number", new JsonSerializerOptions
{
    ReferenceHandler = ReferenceHandler.Preserve
});

Exception stack

System.Text.Json.JsonException: Reference '10' not found. Path: $.$values[3].$ref | LineNumber: 57 | BytePositionInLine: 18.
   at System.Text.Json.ThrowHelper.ThrowJsonException(String message)
   at System.Text.Json.ThrowHelper.ThrowJsonException_MetadataReferenceNotFound(String id)
   at System.Text.Json.Serialization.PreserveReferenceResolver.ResolveReference(String referenceId)
   at System.Text.Json.JsonSerializer.ResolveMetadataForJsonObject[Number](Utf8JsonReader& reader, ReadStack& state, JsonSerializerOptions options)
   at System.Text.Json.Serialization.Converters.ObjectDefaultConverter`1[[WebApplication2.Shared.Number, WebApplication2.Shared, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null]].OnTryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, Number& value)
   at System.Text.Json.Serialization.JsonConverter`1[[WebApplication2.Shared.Number, WebApplication2.Shared, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null]].TryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, Number& value)
   at System.Text.Json.Serialization.Converters.IEnumerableDefaultConverter`2[[System.Collections.Generic.IList`1[[WebApplication2.Shared.Number, WebApplication2.Shared, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null]], System.Private.CoreLib, Version=5.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[WebApplication2.Shared.Number, WebApplication2.Shared, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null]].OnTryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, IList`1& value)
   at System.Text.Json.Serialization.JsonConverter`1[[System.Collections.Generic.IList`1[[WebApplication2.Shared.Number, WebApplication2.Shared, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null]], System.Private.CoreLib, Version=5.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].TryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, IList`1& value)
   at System.Text.Json.Serialization.JsonConverter`1[[System.Collections.Generic.IList`1[[WebApplication2.Shared.Number, WebApplication2.Shared, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null]], System.Private.CoreLib, Version=5.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].ReadCore(Utf8JsonReader& reader, JsonSerializerOptions options, ReadStack& state)
   at System.Text.Json.JsonSerializer.ReadCore[IList`1](JsonConverter jsonConverter, Utf8JsonReader& reader, JsonSerializerOptions options, ReadStack& state)
   at System.Text.Json.JsonSerializer.ReadCore[IList`1](JsonReaderState& readerState, Boolean isFinalBlock, ReadOnlySpan`1 buffer, JsonSerializerOptions options, ReadStack& state, JsonConverter converterBase)
   at System.Text.Json.JsonSerializer.<ReadAsync>d__20`1[[System.Collections.Generic.IList`1[[WebApplication2.Shared.Number, WebApplication2.Shared, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null]], System.Private.CoreLib, Version=5.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].MoveNext()
   at System.Net.Http.Json.HttpContentJsonExtensions.<ReadFromJsonAsyncCore>d__3`1[[System.Collections.Generic.IList`1[[WebApplication2.Shared.Number, WebApplication2.Shared, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null]], System.Private.CoreLib, Version=5.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].MoveNext()
   at System.Net.Http.Json.HttpClientJsonExtensions.<GetFromJsonAsyncCore>d__9`1[[System.Collections.Generic.IList`1[[WebApplication2.Shared.Number, WebApplication2.Shared, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null]], System.Private.CoreLib, Version=5.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].MoveNext()
   at WebApplication2.Client.Pages.FetchData.OnInitializedAsync() in C:\Users\bruno\Downloads\WebApplication2\Client\Pages\FetchData.razor:line 41
   at Microsoft.AspNetCore.Components.ComponentBase.RunInitAndSetParametersAsync()
   at Microsoft.AspNetCore.Components.RenderTree.Renderer.GetErrorHandledTask(Task taskToHandle)

Remarks

• I have tried using the HttpClient extension method and the usual JsonSerializer approach with the same results.
• I noticed that regardless of the number of objects in the list, it will always throw when it encounters the first element that is out of the nested reference. (Un-minify the Json and look for the reference that threw, it will be clear what I mean).

[Symbai]

Similar issues have been reported in the past, check out this comment, maybe it solves your problem: #45325 (comment)

[BrunoBlanes]

Some more context:

public class Number
{
    public int Value { get; set; }
    public string Name { get; set; }
    public IList<Category> Categories { get; set; }
}

public class Category
{
    public string Name { get; set; }
    public IList<Number> Numbers { get; set; }
}

[layomia]

@BrunoBlanes - based on the input JSON, it looks like you need to specify either camelCase settings or case insensitive property name matching. See https://dotnetfiddle.net/rThN9S.

var response = await Http.GetFromJsonAsync<IList<Number>>("Number", new JsonSerializerOptions
{
    ReferenceHandler = ReferenceHandler.Preserve,
    PropertyNamingPolicy = JsonNamingPolicy.CamelCase // this is needed
});

Let us know if you still face issues here.

[BrunoBlanes]

I forgot about naming policies... I went with PropertyNamingPolicy = null and it worked fine, however I think a more specific error message would be in place given that I had the impression that JsonSerializer had actually read at least part of the JSON file properly, which wasn't at all the case.

[ahsonkhan]

@layomia, @Jozkee - consider this as another ref count for better exception and error messaging to help debugging mentioned here as well. If it is easy to do with minimal code changes, great. Otherwise, maybe update the exception message string being thrown when preserve is on to include potential causes for such an error generally so at least folks have a hint of what things to look out for.