[ML] Transition to typeless (mapping) APIs #39256
Conversation
|
Pinging @elastic/ml-core |
|
These changes are definitely great for 8.0 but for 7.x some edge cases that need testing are:
For cases 2. and 4. it could possibly make a difference if the shards of the 6.x indices are on a 6.7 or 7.x node and also whether the master node is a 6.7 or 7.x node, so actually there are 10 possibly different scenarios in the list above. |
davidkyle
force-pushed
the
remove-type
branch
2 times, most recently
from
6f451a8
to
4e6dc69
Compare
|
please run elasticsearch-ci/1 |
|
please run elasticsearch-ci/2 |
|
After removing the mapping type from the ml templates I hit a problem that caused the I reverted the template mappings back to using a type to fix this using the |
This sounds fine for master but for 7.x I think it will run into a problem in the case where we update the mappings of an existing index that was created in 6.x. In this case we need to use whatever type the index was created with. We don't care what that type is, we just need to make sure our updated mapping specifies it, otherwise the update will fail with a "more than one type" error. It makes me think that to solve this problem in 7.x will require a change to non-ML code - either I wonder if what is needed is an API where we can say, "update the mappings for whatever single type this index is already using". |
|
run elasticsearch-ci/packaging-sample |
davidkyle
force-pushed
the
remove-type
branch
from
6ce38c7
to
3dafaf1
Compare
|
run elasticsearch-ci/2 |
|
Once #39469 is fixed the BWC problems should be resolved and these changes should be good to merge. |
davidkyle
force-pushed
the
remove-type
branch
from
3512e27
to
ddadba4
Compare
|
run elasticsearch-ci/2 |
Following elastic#39256 some of these tests were generating errors when indexing documents. For consistency it's best that all of them use typeless APIs.
Following #39256 some of these tests were generating errors when indexing documents. For consistency it's best that all of them use typeless APIs.
ML has historically used doc as the single mapping type but reindex in 7.x will change the mapping to _doc. Switching to the typeless APIs handles the case where the mapping type is either doc or _doc. This change removes deprecated typed usages.
ML has historically used doc as the single mapping type but reindex in 7.x will change the mapping to _doc. Switching to the typeless APIs handles the case where the mapping type is either doc or _doc. This change removes deprecated typed usages.
…9574) Following elastic#39256 some of these tests were generating errors when indexing documents. For consistency it's best that all of them use typeless APIs.
…9574) Following elastic#39256 some of these tests were generating errors when indexing documents. For consistency it's best that all of them use typeless APIs.
Removes deprecated usage of types in the APIs and resolves a reindex issue with the upgrade assistant in 7 to wit:
docas the single mapping type by convention_docunless a target type is specifieddoctype after reindex a mapping conflict occurs as the index would have 2 typesdocand_doc. This is the cause of [ML] Need to cope with .ml-config being an alias #38796Depending on whether or not the upgrade assistant has been run the single mapping type of the ml indices may be
docor_doc, moving to the typeless APIs largely covers this situation however, there is the case of updating the results index mapping on job creation in which case the new mapping must have the same type as the old otherwise the index will have 2 types of mappingsWIP in progress as testing is required, depends on #39243