Enhance documentation around compatibility and clarify client roadmap

[andrewrimmer]

It would be useful to provide clarification on what versions of Elasticsearch are supported by which version of the .NET client.

The compatibility matrix is pretty clear that the .NET client major version should match the major version of the Elastic cluster. However it is not clear from the documentation what support there is across minor versions.

My assumption has been that there would be very few breaking changes between minor releases of Elastic and as such version 7.x of the .NET client should work against any version of the Elastic cluster that is equal or greater than the Elastic client version. This assumption has proven to be incorrect. @codebrain clarified that the 7.0 .NET client is not recommended to be used against a 7.1 or 7.2 cluster.

This brings up many different questions.

1. Should the minor version of the .NET client always match the elastic cluster? i.e. 7.0.x should be used against 7.0.x clusters? (Maybe this is implied in the compatibility matrix, but perhaps could be made clearer?)
2. When does trunk development (and automated testing) move to the next Elastic version?
3. Will the next supported version of Elastic cluster be 7.1 or 7.2?
4. Is there a roadmap for the .NET client and when current versions of Elastic cluster will be supported?
5. Are the loose targets for how long the .NET client will lag behind the Elastic cluster?
6. How dangerous is using a .NET client that is behind the minor version of the elastic cluster
7. Would it be useful to have the automated integration tests run against the latest current release of Elastic? This could give developers a better idea of what should work should they choose (have) to run a .NET client against an unsupported version of Elastic.

Current Elasticsearch 7 .net support?

Version	Release Date	.NET support?
7.0	10th April 2019	2nd July 2019
7.1	20th May 2019	❌
7.2	25th June 2019	❌

I think if you were new to Elasticsearch and started development right now there would be a good chance you would install the latest and greatest Elastic (7.2) and then install 7.0 of NEST and assume everything would be fine.

Hopefully the documentation can be fine tuned, so that choosing the right versions of Elastic and the client are more obvious, and in turn also make it easier to plan rolling upgrades to the latest supported version of Elastic.

I am happy to help, if there is anything I can do.

[codebrain]

Hi @andrewrimmer - this is a topic we can definitely add more clarity to, thanks for opening this issue. We will be talking about this very shortly and updating the main page (linking in #3663 for future reference).

My intention is to re-run the 7.2.0 integration tests and determine the exact nature of the failures. As stated in the previous ticket, the 7.1.1 tests all passed without issue.

I will update on my findings here.

[jasondaicoder]

This is a really great page, we were looking for the same clarification for a while. Having the compatibility metric table will help lot. It also would be good if we have a backwards compatibility metric such as does NEST 7 work with lower version of ES like 5.x?

[codebrain]

Hi @andrewrimmer - thank you for opening this ticket. We are going to improve the messaging on the homepage of this repo to better reflect how we think about versioning. In the meantime, please find some responses to the questions you've raised;

---

Should the minor version of the .NET client always match the elastic cluster? i.e. 7.0.x should be used against 7.0.x clusters? (Maybe this is implied in the compatibility matrix, but perhaps could be made clearer?)

This is probably best answered with an example.

Using a 7.0.0 client against a 7.0.0 server? Support for all features in 7.0.0. In some instances, we may decide to delay implementing new APIs or some features until later minor versions, but this is the exception and not the rule.

Using a 7.0.0 client against a 7.0.1 - 7.2.0 server? Support for all comparable features that are available in the 7.0.0 server. If you want to use new features introduced in 7.2.0 with a lower version client, then you will have to use the low level client DoRequest method and perform the request/response (de)serialisation yourself.

However, there are some nuances to this that need addressing.

Both the server and the client are developed according to https://semver.org/ principles, so there should be no breaking changes in a minor version - a 7.0.0 client will work against a 7.2.0 server.

However, to err is human, and sometimes a breaking change slips into a minor that renders it impossible for us to fix before we release a 7.2.0 client. I really want to stress that this is the exception, rather than the rule.

For full disclosure, we released 7.0.0 having all of the unit and integration tests passing against7.0.0 and 7.1.1 clusters. The integration tests didn't all pass against 7.2.0 and the advice I gave at the time was that using the 7.0.0 against a 7.2.0 server was "at your own risk". With hindsight, this messaging was not really as clear as it could have been and I really should have given you a better explanation. If this has caused some concern or confusion please accept my apology.

I have re-run the integration tests again against the 7.2.0 server and found the tests that failed. The tests are reflected in this PR: #3957

These failing tests are related to behavioural changes in Elasticsearch as to how it handles empty filters. We had an integration test in the client codebase which asserted the expected behaviour in Elasticsearch, which then broke in 7.2.0. It could be argued if these tests are inherently useful in the client codebase, but for the time being I have muted them for 7.2.0 and above.

Use a 7.0.0 client against any 7.x server version - if there are issues please raise them in the usual way.

We will be releasing a 7.1.1 client shortly too - all unit and integrations pass on all released 7.x versions of the server.

When does trunk development (and automated testing) move to the next Elastic version?

master reflects the latest server version (8.x snapshots)
7.x reflects the latest development of the 7.x client
7.1 is the integration branch for the next client release (7.1.0)

When we release a client (e.g. 7.0.0) we will run the integration and unit tests against the latest minor patch version of Elasticsearch that matches the major.minor of the client release (7.0.0) and then any other latest minors within that major version (7.1.1 and 7.2.0). Based on the advice given above, we then make a decision on release.

In practicality the most work is in (master) and then the current client version (7.1).

Will the next supported version of Elastic cluster be 7.1 or 7.2?

7.1, then we will work on 7.2 features.

Is there a roadmap for the .NET client and when current versions of Elastic cluster will be supported?
Are the loose targets for how long the .NET client will lag behind the Elastic cluster?

I think these are similar questions, so I will try to answer both.

The client is currently always trailing the server release. I can say that this is something we would very much like to change; we do talk about this issue.

Major versions take us longer than minor versions, primarily for two reasons.

Firstly, we see the most change in the server during a major version release. This results in more client work - an example being document type changes and the impact that has on a typed language.

Secondly, we also carry around our own technical debt between major versions, mostly to ensure binary compatibility for our users. Major version releases give us an opportunity to pay down this debt, something we very much want to do to ensure good codebase health. Sometimes the major releases give us a chance to implement far reaching changes, such as serialisation in 7.0.0.

In terms of roadmap, we are driven by both the server changes and our own technical debt.

In terms of timeframes we are doing our best to release as soon as we are able.

How dangerous is using a .NET client that is behind the minor version of the elastic cluster

I'm not quite sure how to answer this, the best I can answer is, given the previous advice above, is to use a 7.0.0 client against any 7.x server version, and if there are issues, please raise them in the usual way.

Would it be useful to have the automated integration tests run against the latest current release of Elastic? This could give developers a better idea of what should work should they choose (have) to run a .NET client against an unsupported version of Elastic.

Yes, we do.

The improvement we can make is to give clearer information on incompatibilities if there are any known at the time of release.

[codebrain]

On second thoughts, let me re-open this ticket and use it to track the homepage updates too.

[kbandrew]

@codebrain thanks so much for such a thorough and thoughtful explanation - that really does help. Everything now makes sense!

[codebrain]

Homepage has now been updated with the further clarifications - closing this ticket.