elastic.RequestResponseSerializer.Deserialize<CreateIndexRequestDescriptor>(stream) fails with: read too much or not enough

[dlukyanov]

Elastic.Clients.Elasticsearch version: 8.10

Elasticsearch version: 8

.NET runtime version: 6

Operating system version: windows10

Not sure if it's a problem, but my goal to externalize index mapping definition - I want to read it from a file.

Tried to use ElasticsearchClient.RequestResponseSerializer.Deserialize<CreateIndexRequestDescriptor>(stream)

However it throws exception:

System.Text.Json.JsonException: The converter 'System.Text.Json.Serialization.Converters.CastingConverter`
  2[Elastic.Clients.Elasticsearch.IndexManagement.CreateIndexRequestDescriptor,Elastic.Clients.Elasticsearch.Serialization.ISelfSerializable]' 
  read too much or not enough. Path: $ | LineNumber: 0 | BytePositionInLine: 1.
   at System.Text.Json.ThrowHelper.ThrowJsonException_SerializationConverterRead(JsonConverter converter)
   at System.Text.Json.Serialization.JsonConverter`1.VerifyRead(JsonTokenType tokenType, Int32 depth, Int64 bytesConsumed, Boolean isValueConverter, Utf8JsonReader& reader)
   at System.Text.Json.Serialization.JsonConverter`1.TryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value)
   at System.Text.Json.Serialization.JsonConverter`1.ReadCore(Utf8JsonReader& reader, JsonSerializerOptions options, ReadStack& state)
   at System.Text.Json.JsonSerializer.ContinueDeserialize[TValue](ReadBufferState& bufferState, JsonReaderState& jsonReaderState, ReadStack& readStack, JsonTypeInfo jsonTypeInfo)
   at System.Text.Json.JsonSerializer.ReadFromStream[TValue](Stream utf8Json, JsonTypeInfo jsonTypeInfo)
   at Elastic.Clients.Elasticsearch.Serialization.SystemTextJsonSerializer.Deserialize[T](Stream stream) in /_/src/Elastic.Clients.Elasticsearch/Serialization/SystemTextJsonSerializer.cs:line 65
   at Program.<Main>$(String[] args)

Code to reproduce:

        var c = new CreateIndexRequestDescriptor(conf.ElasticIndex);
        c.Mappings(m => {
            m.Properties(new Properties
            {
                { "_id", new LongNumberProperty(){ Index=false } },
                { "_access", new KeywordProperty() },
                { "_isactive", new BooleanProperty() },
                { "Type", new KeywordProperty() },
                { "Name", new TextProperty() },
                { "Description", new TextProperty() },
                { "CreateDate", new DateProperty() { Format = "yyyy-MM-dd HH:mm:ss.SSS" } },
                { "UpdateDate", new DateProperty() { Format = "yyyy-MM-dd HH:mm:ss.SSS" } }
            });
        });
        var json = elastic.RequestResponseSerializer.SerializeToString(c);
        elastic.RequestResponseSerializer.Deserialize<CreateIndexRequestDescriptor>(
                new MemoryStream(Encoding.UTF8.GetBytes(json))
            );

json contains correct mapping definition:

{
	"mappings":{
		"properties":{
			"_id":{"index":false,"type":"long"},
			"_access":{"type":"keyword"},
			"_isactive":{"type":"boolean"},
			"Type":{"type":"keyword"},
			"Name":{"type":"text"},
			"Description":{"type":"text"},
			"CreateDate":{"format":"yyyy-MM-dd HH:mm:ss.SSS","type":"date"},
			"UpdateDate":{"format":"yyyy-MM-dd HH:mm:ss.SSS","type":"date"}
		}
	}
}

However the last line throws exception i mentioned above.

[flobernd]

Hi @dlukyanov,

this behavior is kinda expected. The request descriptors and requests in general are serializable, but not (necessarily) deserializable. The same applies to responses which are only deserializable, but not serializable.

We usually recommend creating custom POCO classes and translating them to the Elasticsearch types and visa verce.

Does that answer your question?

[dlukyanov]

Maybe there a way to send a raw elastic request?

[flobernd]

@dlukyanov Yes, this is certainly possible like documented here.

However, I personally would not recommend that.

[gitdrk]

@flobernd I am having a hard time finding documentation on how to do the POCO -> Fluent Mapping approach in v8. Do you have an example somewhere? I am migrating from nest and previously using attribute mapping.

[flobernd]

@gitdrk It basically boils down to using the PutMapping API (most mappings are supported during index creation as well). I'll give you some examples:

await client.Indices
	.PutMappingAsync<Person>(
		Indices.Index("index"),
		x => x.Properties(p =>
			p.DoubleNumber(person => person.Age!, c => c.Coerce(true))));

or

await client.Indices
	.PutMappingAsync<Person>(
		Indices.Index("index"),
		x => x.Properties(p => p.Text(p => p.FirstName!, c => c.Analyzer("my_analyzer"))));

It might as well be useful to have a look at the REST documentation for now until a C# specific documentation is available. The client is designed in a way that all the structs and fields from the JSON map pretty closely to the request classes generated for the client.