Custom SourceSerializer options not effective for response data

[chaimtevel]

Elastic.Clients.Elasticsearch version: 8.13.0

Elasticsearch version: 8.9.1

.NET runtime version: 8.0

Operating system version: Windows

Description of the problem including expected versus actual behavior:
We're really were excited to see the latest release which has added support for all aggregations. As I was testing I came across this issue. When adding custom converters to DefaultSourceSerializer, they do not seem to be taking effect when reading response data.

In the snippet below we're using a custom json converter for System.Net.IPAddress. When writing the data to the index the serializations works as expected but when reading the response data to a search request we get a UnexpectedTransportException.

Expected behavior
For the custom converters to be used to deserialize response data. This does appear to work in Elastic.Clients.Elasticsearch version 8.12.1

Provide DebugInformation (if relevant):

Here is a sample app I tested using Linpad:

static Serializer searializerFactory(Serializer def, IElasticsearchClientSettings s) => new DefaultSourceSerializer(s, o =>
{
	o.Converters.Add(new IPAddressConverter());
});

const string indexName = "test-serialization";

var nodepool = new SingleNodePool(new Uri("http://localhost:9200"));

var settings = new ElasticsearchClientSettings(nodepool, searializerFactory)
	.EnableHttpCompression()
	.DefaultFieldNameInferrer(s => System.Text.Json.JsonNamingPolicy.CamelCase.ConvertName(s))
	.EnableDebugMode(cd =>
	{
		var request = System.Text.Encoding.Default.GetString(cd.RequestBodyInBytes);
		Util.SqlOutputWriter.WriteLine(cd.DebugInformation);
	})
	.DefaultMappingFor<AuditSearchItem>(m => m
		.IndexName(indexName)
		.IdProperty(i => i.ID)
	);

var client = new ElasticsearchClient(settings);

using CancellationTokenSource tokenSource = new(TimeSpan.FromSeconds(15));
var token = tokenSource.Token;

await client.IndexAsync(new AuditSearchItem(), token);

// wait for elastic index refresh
await Task.Delay(TimeSpan.FromSeconds(2), token);

var searchResponse = await client.SearchAsync<AuditSearchItem>(c => c
	.Index(indexName)
	.Query(q => q
		.MatchAll(ma => ma.QueryName("all"))	
	)
	.Size(15)
, token);

searchResponse.Documents.Dump();

public class IPAddressConverter : JsonConverter<IPAddress>
{
	public override IPAddress? Read(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options)
	{
		if (reader.TokenType == JsonTokenType.String)
		{
			var value = reader.GetString();
			return IPAddress.Parse(value!);
		}
		else
		{
			return default;
		}
	}

	public override void Write(Utf8JsonWriter writer, IPAddress value, JsonSerializerOptions options)
		=> writer.WriteStringValue(value.ToString());
}

public class AuditSearchItem
{
	public Guid ID { get; init; } = Guid.NewGuid();
	public IPAddress IpAddress { get; init; } = IPAddress.Parse("127.0.0.1");
}

The error message is The JSON value could not be converted to System.Net.IPAddress. Path: $.hits.hits[0]._source.ipAddress | LineNumber: 22 | BytePositionInLine: 35.

I've attached the debug information.
custom-serialization-debug-info.txt

Thanks!

Chaim

[flobernd]

Hi @chaimtevel, thanks for reporting! I'll have a look at this right away 🙂

[flobernd]

Thanks for the nice issue description and the reproducer!

Fixed in 8.13.1.

[chaimtevel]

Thank you so much! We rely on this library heavily so we're happy to help in any way we can.