GeoLocation deserialization failing.

[vulegend]

Elastic.Clients.Elasticsearch version: 9.0.6

Elasticsearch version: 9.0.1

.NET runtime version: 9

Operating system version: Windows 11, ES hosted on docker container using latest image

Description of the problem including expected versus actual behavior:

I can't seem to get GeoLocation deserialization to work. Serialization works fine but whenever it tries to read it from a document it throws the following exception:

Elastic.Transport.UnexpectedTransportException: Operation is not valid due to the current state of the object.
 ---> System.InvalidOperationException: Operation is not valid due to the current state of the object.
   at Elastic.Clients.Elasticsearch.GeoLocationConverter.Read(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options) in /_/src/Elastic.Clients.Elasticsearch/_Shared/Types/GeoLocation.cs:line 26
   at System.Text.Json.Serialization.JsonConverter`1.TryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value, Boolean& isPopulatedValue)
   at System.Text.Json.Serialization.Metadata.JsonPropertyInfo`1.ReadJsonAndSetMember(Object obj, ReadStack& state, Utf8JsonReader& reader)
   at System.Text.Json.Serialization.Converters.ObjectDefaultConverter`1.OnTryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value)
   at System.Text.Json.Serialization.JsonConverter`1.TryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value, Boolean& isPopulatedValue)
   at System.Text.Json.Serialization.JsonConverter`1.ReadCore(Utf8JsonReader& reader, T& value, JsonSerializerOptions options, ReadStack& state)
   at System.Text.Json.Serialization.Metadata.JsonTypeInfo`1.Deserialize(Utf8JsonReader& reader, ReadStack& state)
   at System.Text.Json.JsonSerializer.Read[TValue](Utf8JsonReader& reader, JsonTypeInfo`1 jsonTypeInfo)
   at Elastic.Transport.Extensions.TransportSerializerExtensions.Deserialize[T](Serializer serializer, Utf8JsonReader& reader, MemoryStreamFactory memoryStreamFactory)
   at Elastic.Clients.Elasticsearch.Serialization.SourceConverter`1.Read(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options) in /_/src/Elastic.Clients.Elasticsearch/_Shared/Next/SourceConverter.cs:line 84
   at Elastic.Clients.Elasticsearch.GetResponseConverter`1.<>c.<Read>b__10_2(Utf8JsonReader& r, JsonSerializerOptions o) in /_/src/Elastic.Clients.Elasticsearch/_Generated/Api/GetResponse.g.cs:line 94
   at Elastic.Clients.Elasticsearch.GetResponseConverter`1.Read(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options) in /_/src/Elastic.Clients.Elasticsearch/_Generated/Api/GetResponse.g.cs:line 41
   at System.Text.Json.Serialization.JsonConverter`1.TryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value, Boolean& isPopulatedValue)
   at System.Text.Json.Serialization.JsonConverter`1.ReadCore(Utf8JsonReader& reader, T& value, JsonSerializerOptions options, ReadStack& state)
   at System.Text.Json.Serialization.Metadata.JsonTypeInfo`1.ContinueDeserialize(ReadBufferState& bufferState, JsonReaderState& jsonReaderState, ReadStack& readStack, T& value)
   at System.Text.Json.Serialization.Metadata.JsonTypeInfo`1.DeserializeAsync(Stream utf8Json, CancellationToken cancellationToken)
   at Elastic.Transport.Products.Elasticsearch.ElasticsearchResponseBuilder.SetBodyCoreAsync[TResponse](Boolean isAsync, ApiCallDetails details, BoundConfiguration boundConfiguration, Stream responseStream, CancellationToken cancellationToken)
   at Elastic.Transport.DefaultResponseFactory.CreateCoreAsync[TResponse](Boolean isAsync, Endpoint endpoint, BoundConfiguration boundConfiguration, PostData postData, Exception ex, Nullable`1 statusCode, Dictionary`2 headers, Stream responseStream, String contentType, Int64 contentLength, IReadOnlyDictionary`2 threadPoolStats, IReadOnlyDictionary`2 tcpStats, CancellationToken cancellationToken)
   at Elastic.Transport.HttpRequestInvoker.RequestCoreAsync[TResponse](Boolean isAsync, Endpoint endpoint, BoundConfiguration boundConfiguration, PostData postData, CancellationToken cancellationToken)
   at Elastic.Transport.RequestPipeline.CallProductEndpointCoreAsync[TResponse](Boolean isAsync, Endpoint endpoint, BoundConfiguration boundConfiguration, PostData postData, Auditor auditor, CancellationToken cancellationToken)
   at Elastic.Transport.DistributedTransport`1.RequestCoreAsync[TResponse](Boolean isAsync, EndpointPath path, PostData data, Action`1 configureActivity, IRequestConfiguration localConfiguration, CancellationToken cancellationToken)
   --- End of inner exception stack trace ---
   at Elastic.Transport.DistributedTransport`1.ThrowUnexpectedTransportException[TResponse](Exception killerException, List`1 seenExceptions, Endpoint endpoint, TResponse response, IReadOnlyCollection`1 auditTrail)
   at Elastic.Transport.DistributedTransport`1.RequestCoreAsync[TResponse](Boolean isAsync, EndpointPath path, PostData data, Action`1 configureActivity, IRequestConfiguration localConfiguration, CancellationToken cancellationToken)
   at IntegrationTests.Test.Terms.TermsEmptyArrayTest.Run() in D:\Smart\SmartQueryBuilder\IntegrationTests\Test\Terms\TermsEmptyArrayTest.cs:line 26

In my index configuration and document class it's defined like this:

public GeoLocation Position { get; set; }

And mapped with:

.GeoPoint(y => y.Position)

This is how it looks like in response from Kibana (both mapping and actual result):

"position": {
            "lat": 29.313292,
            "lon": -95.427645
          }

"position": {
        "type": "geo_point"
      },

This is how i construct the GeoLocation objects in the documents before indexing:

.ForMember(x => x.Position, opt => opt.MapFrom(src => GeoLocation.LatitudeLongitude(new LatLonGeoLocation(src.Lat, src.Lng))))

I tried multiple things and nothing seems to work. This is a pretty big issue that prevents me from upgrading and using the new version.

Steps to reproduce:

1. Create a document class with GeoLocation property
2. Construct GeoLocation object using GeoLocation.LatitudeLongitude method
3. Index the documents
4. Try reading the document with simple GetAsync

Expected behavior:

The library should successfully deserialize GeoLocation object.

Provide ConnectionSettings (if relevant):

Provide DebugInformation (if relevant):

[vulegend]

While nothing out of the box works with default settings i got it to work with a custom JsonConverter. It works with the following converter:

using System.Text.Json;
using System.Text.Json.Serialization;
using Elastic.Clients.Elasticsearch;

public class CustomGeoLocationConverter : JsonConverter<GeoLocation>
{
    public override GeoLocation? Read(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options)
    {
        if (reader.TokenType == JsonTokenType.Null)
        {
            return null;
        }

        if (reader.TokenType != JsonTokenType.StartObject)
        {
            throw new JsonException("Expected StartObject token for GeoLocation.");
        }

        double? lat = null;
        double? lon = null;

        while (reader.Read())
        {
            if (reader.TokenType == JsonTokenType.EndObject)
            {
                if (lat.HasValue && lon.HasValue)
                {
                    return GeoLocation.LatitudeLongitude(new LatLonGeoLocation(lat.Value, lon.Value));
                }
                
                throw new JsonException("GeoLocation object must contain 'lat' and 'lon' properties.");
            }

            if (reader.TokenType == JsonTokenType.PropertyName)
            {
                var propertyName = reader.GetString();
                reader.Read();

                switch (propertyName)
                {
                    case "lat":
                        lat = reader.GetDouble();
                        break;
                    case "lon":
                        lon = reader.GetDouble();
                        break;
                    default:
                        reader.Skip();
                        break;
                }
            }
        }
        
        throw new JsonException("Unexpected end of JSON while parsing GeoLocation.");
    }

    public override void Write(Utf8JsonWriter writer, GeoLocation value, JsonSerializerOptions options)
    {
        JsonSerializer.Serialize(writer, value, value.GetType(), options);
    }
}

[flobernd]

Hi @vulegend ,

this is an oversight in the new 9.x and 8.19-preview versions.

The built-in converter for GeoLocation should indeed as well support deserialization (see #8513 (comment)).

Will be fixed in an upcoming release.