JsonSerializerContext source generator ignores Converters

[johnwc]

Description

Trying to figure out why my code keeps failing, and it when looking further into the generated code, there seems to be no converter being added to the Default context property. It has a count of 0 no matter how I try to use it.

[JsonSourceGenerationOptions(
    GenerationMode = JsonSourceGenerationMode.Default,
    DefaultIgnoreCondition = JsonIgnoreCondition.WhenWritingNull,
    Converters = [
        typeof(AccountId.AccountIdSystemTextJsonConverter)
        ])]
[JsonSerializable(typeof(AccountMembership))]
[JsonSerializable(typeof(AccountId))]
public partial class SourceGenerationContext : JsonSerializerContext
{
}

Reproduction Steps

Create SourceGenerationContext using the shared code above.
View that the Default.Optons.Converters is empty.

Expected behavior

The list of converters given in the JsonSourceGenerationOptions attribute be set within the Default options. for the context

Actual behavior

No converters added.

Regression?

No response

Known Workarounds

No response

Configuration

.Net 9.0.6

Other information

No response

[dotnet-policy-service]

Tagging subscribers to this area: @dotnet/area-system-text-json, @gregsdennis
See info in area-owners.md if you want to be subscribed.

[elgonzo]

Please check your AccountId.AccountIdSystemTextJsonConverter converter declaration, particularly what base type it derives from.

The only situation i know of where the specified converter type isn't taken into the options is when that converter type doesn't derive from System.Text.Json.Serialization.JsonConverter<T> or System.Text.Json.Serialization.JsonConverterFactory. (That particular situation would still count as a bug, i guess. The STJ source generator should emit a warning or error and not silently ignore non-JsonConverter types specified in the JsonSourceGenerationOptions.Attribute.Converters parameter.) EDIT: The STJ source generator does emit a warning if a non-JsonConverter type is being specified in JsonSourceGenerationOptionsAttribute.Converters.

[johnwc]

It does inherit from System.Text.Json.Serialization.JsonConverter

public class AccountIdSystemTextJsonConverter : global::System.Text.Json.Serialization.JsonConverter<AccountId>
{
...
}

[elgonzo]

I cannot reproduce. Neither with STJ 9.0.4 nor with the newest STJ 10.0.0-preview.5.25277.114 pre-release versions.

The presence of the global:: namespace prefix there in the declaration makes me think that this declaration is generated by some other source generator. If that is true, it might be that the STJ source generator cannot see the output of that other source generator and hence being unable to verify AccountIdSystemTextJsonConverter being really a JsonConverter<T> type (see also here: dotnet/roslyn#48358).

[johnwc]

It's a public class, nothing is being hidden. It is being generated using Vogen nuget package.

[elgonzo]

Vogen a source generator, right? As i said in my previous comment: "it might be that the STJ source generator cannot see the output of that other source generator and hence being unable to verify AccountIdSystemTextJsonConverter being really a JsonConverter type (see also here: dotnet/roslyn#48358).".

It's not about types being public or not. It's about the inability of one source generator seeing the output of another source generator. It's a roslyn limitation. It has to be fixed in roslyn; nothing that the STJ source generator can fix.

Until this roslyn limitation has been solved, you might be able to workaround this by creating a partial class stub including it inheriting from JsonConverter<T> in a real .cs file in the project that the STJ source generator can see and process:

public partial class AccountIdSystemTextJsonConverter : JsonConverter<AccountId>
{ }

(whether you need some additional attribute annotation for the partial declaration or not and for anything further, i don't know as i am not experienced with Vogen in particular. Please consult the Vogen documentation about such.)

[elgonzo]

If for whatever reason Vogen does not allow you to create partial class stubs for the JsonConverters it generates, or you simply don't like creating such stubs, there is another alternative workaround.

Instead of using the SourceGenerationContext.Default and the [JsonSourceGenerationOptions] for the (de)serialization settings, you can create a SourceGenerationContext instance with a JsonSerializerOptions instance (thus not requiring the STJ source generator to see/inspect the JsonConverter types) in a similar manner to this:

[JsonSourceGenerationOptions(GenerationMode = JsonSourceGenerationMode.Default)]
[JsonSerializable(typeof(AccountMembership))]
[JsonSerializable(typeof(AccountId))]
public partial class SourceGenerationContext : JsonSerializerContext
{
    //
    // Use this instead of the Default property
    //
    public static SourceGenerationContext UseThis {get; } = CreateMyContextWithOptions();

    private static SourceGenerationContext CreateMyContextWithOptions()
    {
        var opts = new JsonSerializerOptions
        {
            DefaultIgnoreCondition = JsonIgnoreCondition.WhenWritingNull
        };

        opts.Converters.Add(new AccountId.AccountIdSystemTextJsonConverter());
        return new SourceGenerationContext(opts);
    }
}

[huoyaoyuan]

It is being generated using Vogen nuget package.

Do you mean the converter is generated? Source generators can't see each other. Is there a partial declaration in your code?

[elgonzo]

By the way, are the data models you want to de/serialize also entirely created by Vogen source-generator without a (partial) declaration in your project code? If so, the STJ source generator would also not be able to see the shape of these model types either and not be able to synthesize appropriate de/serialization code. If that's the case, you might need to create partial declarations in "real" .cs files in your project code (containing the (partial) member declarations of those members that should participate in de/serialization); or it might perhaps just be easier to use the default reflection-based serializer instead.

If all you are going to de/serialize are Vogen-generated data models which are entirely and completely handled by Vogen-generated JsonConverters, you might not really benefit from the STJ source generator and could use the default reflection-based serializer, because in that situation the de/serialization would be entirely handled by the Vogen-generated JsonConverters. Heck, since Vogen is apparently able to generate JsonConverters, it is not too far fetched to believe that Vogen might also be able to offer an already "pre-generated" (and properly configured) JsonSerializerContext-based serializer. But please consult the Vogen documentation about that and don't mistake my assumption here as fact.

[johnwc]

I see, so this is the chicken before the egg type of scenario. Where the source generators are not aware of the source created by other generators at the type of generation? Even if I was able to create a partial class in my own code, I do not think the source generator would be able to see that it is inherited from JsonConverter, as the inheritance is added via the source generator.

How about doing something like this then, where you add a JsonSourceGenerationOptionsConverter attribute to the JsonSerializerContext. Then the source generator looks for those attributes added to the class and use them to populate the Converters.

[AttributeUsage(AttributeTargets.Class, AllowMultiple = true, Inherited = false)]
public class JsonSourceGenerationOptionsConverterAttribute<T> : System.Attribute where T : JsonConverter
{
    /// <summary>
    /// Constructs a new <see cref="JsonSourceGenearationConverterOptionsAttribute{T}"/> instance.
    /// </summary>
    public JsonSourceGenerationOptionsConverterAttribute() { }

    /// <summary>
    /// Gets the type of the converter.
    /// </summary>
    public Type ConverterType => typeof(T);
}

[JsonSourceGenerationOptions(
    GenerationMode = JsonSourceGenerationMode.Default,
    DefaultIgnoreCondition = JsonIgnoreCondition.WhenWritingNull,
    UnknownTypeHandling = JsonUnknownTypeHandling.JsonNode)]
[JsonSerializable(typeof(AccountMembership))]
[JsonSerializable(typeof(AccountId))]
[JsonSourceGenerationOptionsConverter<AccountId.AccountIdSystemTextJsonConverter>()]
public partial class SourceGenerationContext : JsonSerializerContext
{
}

[johnwc]

@huoyaoyuan @elgonzo any thoughts?