Move kibana reporting data stream settings into component template #107581
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Previously these were contained in the index template, however, Kibana needs to be able to make overrides to only the settings, so factoring these out would allow them to do this (in such a way that they can be overridden by the `kibana-reporting@custom` component template as well). Relates to elastic#97765
dakrone
added
>non-issue
:Data Management/Data streams
elasticsearchmachine
added
the
Team:Data Management
|
Pinging @elastic/es-data-management (Team:Data Management) |
nielsbauman
approved these changes
May 2, 2024
parkertimmins
approved these changes
May 2, 2024
|
@elasticmachine update branch |
dakrone
added
the
auto-merge-without-approval
markjhoy
pushed a commit
to markjhoy/elasticsearch
that referenced
this pull request
May 9, 2024
…lastic#107581) Previously these were contained in the index template, however, Kibana needs to be able to make overrides to only the settings, so factoring these out would allow them to do this (in such a way that they can be overridden by the `kibana-reporting@custom` component template as well). Relates to elastic#97765
7 tasks
tsullivan
added a commit
to elastic/kibana
that referenced
this pull request
May 21, 2024
## Summary Closes #161608 * [X] Depends on elastic/elasticsearch#97765 * [x] Depends on elastic/elasticsearch#107581 * [x] Add create a new report job and check the details of the templated data stream. * [x] Run Discover tests in Flaky Test Runner: https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/5999 ## Release Note Reporting internal storage has been changed from using regular indices to a data stream configuration for a more efficient sharding strategy. This change is not expected to have any impact to users. ## Screenshots ### Upgrade test (manual process) Using a report generated before this change, and a report generated after "upgrading":  Even though the two reports are in different types storage, they are still managed by the same policy:  Looking at the details of the policy shows how the different types of storage are used:  ### Log lines Initial startup in clean environment ``` [2024-05-13T13:22:49.138-07:00][INFO ][plugins.reporting.store] Creating ILM policy for reporting data stream: kibana-reporting [2024-05-13T13:22:53.337-07:00][INFO ][plugins.reporting.store] Linking ILM policy to reporting data stream: .kibana-reporting, component template: kibana-reporting@custom ``` Kibana restart with ES running continuously ``` [2024-05-13T13:24:32.733-07:00][DEBUG][plugins.reporting.store] Found ILM policy kibana-reporting; skipping creation. [2024-05-13T13:24:32.733-07:00][INFO ][plugins.reporting.store] Linking ILM policy to reporting data stream: .kibana-reporting, component template: kibana-reporting@custom ``` ### Checklist - [x] [Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html) was added for features that require explanation or tutorials - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios - [ ] [Flaky Test Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was used on any tests changed ~~See https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/5302 (internal link)~~
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Previously these were contained in the index template, however, Kibana needs to be able to make overrides to only the settings, so factoring these out would allow them to do this (in such a way that they can be overridden by the
kibana-reporting@customcomponent template as well).Relates to #97765