Significant performance issue with the driver running under .NET Framework

[onyxmaster]

Elastic.Clients.Elasticsearch version:
8.19.12

Elasticsearch version:
Irrelevant

.NET runtime version:
4.7.2 TFM, 4.8.1 runtime

Operating system version:
Windows 11

Description of the problem including expected versus actual behavior:
The Enum.HasFlag under .NET Framework boxes its argument leading to excessive allocations on hot paths. While in modern .NET this issue is not present, the classic behavior is, unfortunately, as described.
When HasFlag is used casually (in composition code or request setup) it could be okay, but when it's used in tight loops like serialization code, it brings a lot of unnecessary allocations (we measured about 6M objects per 30s in our specific scenario since we index embedding vectors which employs floating-point serialization extensively) significantly increasing pressure for gen0 allocations/collections.

I did a search over the codebase and HasFlag is only used in three distinct source files (1, 2, 3), so it does not look like it's a common practice, hence the improvement looks low-effort for me.

I can provide a PR for the first two files, replacing HasFlag calls with generic bit checks (value & flag) == flag, but not sure what to do with the generated code.

Thanks for taking a look.

P.S. PerfView traces are available upon request

[flobernd]

Hi @onyxmaster thanks for bringing this up.

I'll improve that for the next release!

[flobernd]

The latest 8.19.13 and 9.2.2 versions contain improved code for the Single/Double converter. The codegen for the remaining file needs additional work, but I think the hot-path should be fine with this for now.

Btw.: There are some upcoming changes in regards to vector ingestion in ES 9.3 (which we plan to make available in the clients as well) which involves sending vectors as base64 encoded string instead of a large JSON array.

[onyxmaster]

Thank you, Florian!
The news about base64 serialization are also very nice, especially since we get embeddings from an OpenAI API-compatible endpoint so I probably can just transfer them as an opaque string, skipping large portion of serialization/deserialization in the middle and saving quite a few allocations and CPU cycles.

[flobernd]

@onyxmaster That indeed sounds like a very promising use case!

The only caveat is that Elasticsearch requires the individual base64 encoded floats to use big-endian byte order. If that is supported by OAI, your plan should work.

Otherwise, the .NET client encodes the arrays, but this causes at least one allocation (I'll use a rented buffer to minimize impact). It will most likely still be faster than the default JSON encoding.

[onyxmaster]

requires the individual base64 encoded floats to use big-endian byte order

No luck for us in this case, since OAI floats appear to be little-endian , thanks for mentioning this.
Thanks for the fix!