Remove support for _type in searches

[jtibshirani]

Types are no longer allowed in requests in 8.0, so we can remove support for
using the _type field within a search request.

Relates to #41059.
Closes #68311.

[elasticmachine]

Pinging @elastic/es-search (Team:Search)

[romseygeek]

LGTM, thanks @jtibshirani!

[romseygeek]

I think this means we can remove the 'type' field and constructor parameter as well?

[jtibshirani]

Oops, yep I'll remove it.

[pgomulka]

is this the only yaml test case that will be affected? Just wanted to make sure we have coverage in 7.x to bring this with compatible api

[jtibshirani]

Yes this is the only one, we had very light test coverage for this capability.

[romseygeek]

@pgomulka and I spoke about this offline, and I want to lay out my thoughts on why removing this in 8.0 is the correct solution, given our backwards-compatibility guarantees.

We're adding version compatibility to our REST layer, meaning that if you have a 7.x client you should be able to communicate with an 8.x cluster without getting parse errors. The motive for this is specifically to allow a staggered upgrade of clients and servers. Without this layer, given the changes in the structure of search responses (eg removal of the 'type' field from search hits), it is possible that a 7.x client communicating with an 8.x server would see a response as invalid and throw an exception; similarly, an 8.x client communicating with a 7.x server would also treat the response as invalid; this leaves users attempting a rolling upgrade from 7.x to 8.x with a conundrum - when should they upgrade their clients? The addition of the compatibility layer solves this by allowing you to upgrade your cluster fully while still running queries from a 7.x client, and then upgrade the clients once the cluster upgrade is complete.

Running queries or aggregations against a 'type' field is deprecated in 7.x. Whenever you do this, you will get warning headers and log messages. There is nothing inherent in the difference between a 7.x cluster and an 8.x cluster that prevents users from addressing these deprecation warnings before upgrading, and so I would argue that the compatibility layer should specifically not try and deal with this behaviour. You should fix deprecation warnings before doing an upgrade, full stop. This is a different category of problem to the one that the compatibility layer is designed to deal with, where the difference in response structure is something that cannot be dealt with by the user.

[romseygeek]

TL;DR: the compatibility layer is there to deal with things that can't issue a deprecation warning; queries against _type issue a deprecation warning; ergo the compatibility layer should not try and deal with them.

[pgomulka]

at first I thought we should support this with compatible API. From client's point of view the request shape has changed, the 'meta' field type has gone, so his 7.x client with compatible API should still work.
However after chatting @romseygeek I realised that even thought it is a meta field _type , it is part of the client's data. And since the data is incorrect he should fix it.

I wonder what is @jakelandis and @Mpdreamz view on this?

[jtibshirani]

Thanks for the review! I'm going to merge this (as we discussed), but feel free to continue the discussion here.

[pgomulka]

we discussed this with @jakelandis and we think that this breaking change does not need a compatible api implementation.
After the removal of support of _type in field search, the query will return 200 OK. So will not break the client
There will be a behaviour change though. Previously when searching for field: _type result would be Match all, now we will get no results.

[Mpdreamz]

@jakelandis @pgomulka 👍