Skip to content

Conversation

@prwhelan
Copy link
Member

@prwhelan prwhelan commented Oct 13, 2025

If a user manually creates the legacy audit index, TransformClusterStateListener will automatically add the non-legacy alias to forward messages to it. If the transform feature had never been accessed before (for example after the feature is reset), then the next time the auditor is called, it will associate with the existing legacy index rather than the notifications index.

We cannot cleanly block users from manually creating system or hidden indices (yet), but we can associate the legacy index with the transform feature so that a feature reset will delete the legacy index (like it does with the notifications index). The next time the transform feature is used, it will automatically create the notifications index rather than reusing the legacy index.

Resolve #121241


This is only reproducible after the legacy index is manually created in testAliasCreatedforBWCIndexes, so we need to run the full suite until it runs the tests in the failing order (testAliasCreatedforBWCIndexes -> testAuditorWritesAudits)

This is not needed for 9.x because the code to alias the deprecated index had been removed in 9.0: d839205

If a user manually creates the legacy audit index,
TransformClusterStateListener will automatically add the non-legacy
alias to forward messages to it. If the transform feature had never been
accessed before (for example after the feature is reset), then the next
time the auditor is called, it will associate with the existing legacy
index rather than the notifications index.

We cannot cleanly block users from manually creating system or hidden
indices (yet), but we can associate the legacy index with the transform
feature so that a feature reset will delete the legacy index (like it
does with the notifications index). The next time the transform feature
is used, it will automatically create the notifications index rather
than reusing the legacy index.

Resolve elastic#121241
@prwhelan prwhelan added >bug :ml/Transform Transform Team:ML Meta label for the ML team auto-backport Automatically create backport pull requests when merged v8.19.6 v8.18.9 labels Oct 13, 2025
@elasticsearchmachine
Copy link
Collaborator

Hi @prwhelan, I've created a changelog YAML for you.

@prwhelan prwhelan marked this pull request as ready for review October 14, 2025 14:00
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/ml-core (Team:ML)

@prwhelan prwhelan enabled auto-merge (squash) October 14, 2025 15:24
@prwhelan prwhelan merged commit 98e02b2 into elastic:8.19 Oct 14, 2025
23 checks passed
@elasticsearchmachine
Copy link
Collaborator

💔 Backport failed

Status Branch Result
8.18 Commit could not be cherrypicked due to conflicts

You can use sqren/backport to manually backport by running backport --upstream elastic/elasticsearch --pr 136508

prwhelan added a commit to prwhelan/elasticsearch that referenced this pull request Oct 15, 2025
If a user manually creates the legacy audit index,
TransformClusterStateListener will automatically add the non-legacy
alias to forward messages to it. If the transform feature had never been
accessed before (for example after the feature is reset), then the next
time the auditor is called, it will associate with the existing legacy
index rather than the notifications index.

We cannot cleanly block users from manually creating system or hidden
indices (yet), but we can associate the legacy index with the transform
feature so that a feature reset will delete the legacy index (like it
does with the notifications index). The next time the transform feature
is used, it will automatically create the notifications index rather
than reusing the legacy index.

Resolve elastic#121241
elasticsearchmachine pushed a commit that referenced this pull request Oct 15, 2025
If a user manually creates the legacy audit index,
TransformClusterStateListener will automatically add the non-legacy
alias to forward messages to it. If the transform feature had never been
accessed before (for example after the feature is reset), then the next
time the auditor is called, it will associate with the existing legacy
index rather than the notifications index.

We cannot cleanly block users from manually creating system or hidden
indices (yet), but we can associate the legacy index with the transform
feature so that a feature reset will delete the legacy index (like it
does with the notifications index). The next time the transform feature
is used, it will automatically create the notifications index rather
than reusing the legacy index.

Resolve #121241
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

auto-backport Automatically create backport pull requests when merged backport pending >bug :ml/Transform Transform Team:ML Meta label for the ML team v8.18.9 v8.19.6

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants