[DSL] Introduce data stream global retention - Part 3

[gmarouli]

In this PR we introduce the API that will expose the global retention configuration and will allow users to take advantage of it.

These APIs are protected by the dedicated introduced privileges:

• manage_data_stream_global_retention or higher, which allows all operations on the global retention configuration
• monitor_data_stream_retention or higher, which allows the retrieval of the global retention configuration

This PR is not delivering the complete feature since the global retention is not used yet.

Relevant documentation:

• Retention Tutorial
• Data stream lifecycle page
• Global retention APIs

[github-actions]

Documentation preview:

• ✨ Changed pages

[elasticsearchmachine]

Pinging @elastic/es-data-management (Team:Data Management)

[gmarouli]

Investigating the bwc failures

[gmarouli]

The bwc failures, happened because of the missing feature health.extended_repository_indicator. It was fixed as soon as I updated the branch with main. I expect the tests will pass now.

[parkertimmins]

This is cool, much nicer than than specifying a numeric version. I think I need to go change a yaml test I just wrote ...

[gmarouli]

Yes, I also liked it a lot, but I couldn't make it work in one test. I do not know yet why, it's in my todo list to see if it has a conflict when there is a skip with a version in the setup step :(. It looks like it works in most tests though.

[gmarouli]

I am leaning towards that test failure MixedClusterClientYamlTestSuiteIT » test {p0=mlt/30_unlike/Basic mlt query is not related to this PR. Because:

• It is not using any lifecycle related features, it just creates an index and searches on it, so I can't imagine how this test can fail because of the global retention feature.
• All the global retention tests are skipped because this test is using 7.17.19, so I do not see any test interference.
• I found the same failure back in January.

[parkertimmins]

LGTM, great tutorial btw!

[elasticsearchmachine]

Hi @gmarouli, I've updated the changelog YAML for you. Note that since this PR is labelled release highlight, you need to update the changelog YAML to fill out the extended information sections.

[masseyke]

LGTM

[elasticsearchmachine]

@gmarouli according to this PR's labels, I need to update the changelog YAML, but I can't because the PR is closed. Please either update the changelog yourself on the appropriate branch, or adjust the labels. Specifically:

• The PR is not labelled release highlight but the changelog has a highlight section