[DOCS] Simplify ML upgrade step #40006
Conversation
|
Pinging @elastic/ml-core |
There was a problem hiding this comment.
I left one comment. I guess it shows that the previous instructions were completely unclear about:
there must be no machine learning jobs running during the upgrade
When I wrote that I was thinking of "running" as meaning "actively doing stuff", and you can prevent that by setting upgrade mode. But I guess most people would read "running" as meaning the same as "opened".
Maybe this suggests a different restructuring of the whole thing...
|
|
||
| If your {ml} indices were created before {prev-major-version}, you must stop | ||
| all {dfeeds} and close all {ml} jobs then | ||
| <<reindex-upgrade,reindex the indices>>. |
There was a problem hiding this comment.
An alternative here is to temporarily halt the tasks associated with your {ml} jobs and {dfeeds} and prevent new jobs from opening by using the <<ml-set-upgrade-mode,set upgrade mode API>> during the reindex.
There was a problem hiding this comment.
Thanks, that step was added to the reindexing page in https://github.com/lcawl/elasticsearch/pull/1/files, so I've rebased and augmented the info there.
|
Thanks for the feedback, @droberts195.
Is it easy for users to know when a job is "actively doing stuff"? If not, then I think it would cause more headache than help to make that differentiation. |
If ML upgrade mode is set then jobs will not be "actively doing stuff". Upgrade mode was added as a way for users to tell jobs not to do anything during an upgrade without having to close them. One problem with closing jobs is that after upgrade you have to reopen them, and that assumes you can remember which ones were open before you started. This is a hard problem for some environments, for example Cloud. |
|
Would it be appropriate to change the lead-in sentence from: "Stop any machine learning jobs that are running. (Optional)" To something like this: Temporarily stop the tasks associated with active {ml} jobs and {dfeeds}. (Optional) |
| ifdef::include-xpack[] | ||
| If you use {ml-features} and you're migrating indices from a 6.5 or earlier | ||
| cluster, the job and {dfeed} configuration information are not stored in an | ||
| index. You must recreate your {ml} jobs in the new cluster. If you are migrating from a 6.6 or later cluster, it is a good idea to temporarily halt the tasks associated with your {ml} jobs and {dfeeds} and prevent new jobs from opening |
There was a problem hiding this comment.
The jobs won't open on the new cluster because whether jobs are open or closed is determined by persistent tasks that won't be brought across by the reindex.
So I would change "prevent new jobs from opening" to something like "prevent inconsistencies between different ML indices that are reindexed at slightly different times".
There was a problem hiding this comment.
Thanks @droberts195 does that clarification apply to the "reindex in place" section too?
There was a problem hiding this comment.
For reindex in place the persistent tasks are in the same cluster, so I think that clarification only applies to reindex from remote.
This PR contains changes suggested in #39944