[ML] Make ml internal indices hidden

[przemekwitek]

This PR makes the following indices hidden:

1. .ml-anomalies*
2. .ml-state*
3. .ml-notifications*
4. .ml-annotations*

Transform indices (.transform-notifications-* and deprecated .data-frame-notifications-*) and inference index (.ml-inference-000001) are not converted in this PR.

Relates #52420

[elasticmachine]

Pinging @elastic/ml-core (:ml)

[droberts195]

I think line 941 of MachineLearning.java also needs deleting. At the time it was added it must have been expected that .ml-state would become a system index, not a hidden index. But since we want rollover type behaviour on it we've now decided that it will be a hidden index.

[benwtrent]

Do we need to worry about things like the ExpiredForecastsRemover?
It hits AnomalyDetectorsIndex.jobResultsIndexPrefix() + "*"; for its search but does not set the expansion flags to include hidden indices.

[przemekwitek]

I think line 941 of MachineLearning.java also needs deleting. At the time it was added it must have been expected that .ml-state would become a system index, not a hidden index. But since we want rollover type behaviour on it we've now decided that it will be a hidden index.

Done.

[przemekwitek]

Do we need to worry about things like the ExpiredForecastsRemover?
It hits AnomalyDetectorsIndex.jobResultsIndexPrefix() + "*"; for its search but does not set the expansion flags to include hidden indices.

My understanding is that it will work correctly as it provides the pattern starting with a dot ('.'). The problem occurs when (like in MlDistributedFailureIT.java) no index is provided.

[droberts195]

It hits AnomalyDetectorsIndex.jobResultsIndexPrefix() + "*"; for its search but does not set the expansion flags to include hidden indices.

My understanding is that wildcard patterns still match hidden indices providing the pattern begins with a dot - see the second bullet point in the description of #50452. So we should be OK.

Once this PR is merged the whole team will need to be on the lookout for unforeseen side effects. It's good that we're not doing this close to feature freeze.

[droberts195]

LGTM

[benwtrent]

👍 cool, glad hidden are still expanded if their prefix if specified.

[przemekwitek]

run elasticsearch-ci/2

[przemekwitek]

run elasticsearch-ci/2