Skip to content

Document namespace index templates and overlap resolution - #7705

Open
vishaangelova wants to merge 2 commits into
mainfrom
6545-integration-namespace-index-templates
Open

Document namespace index templates and overlap resolution#7705
vishaangelova wants to merge 2 commits into
mainfrom
6545-integration-namespace-index-templates

Conversation

@vishaangelova

@vishaangelova vishaangelova commented Aug 3, 2026

Copy link
Copy Markdown
Member

Summary

This PR documents the Fleet integration namespace index templates feature, which lets you apply shared settings and mappings across all of an integration's data streams in a single namespace, instead of customizing each data stream individually. The feature is GA in Serverless and targets Stack 9.5+.

It also adds a companion page for resolving overlap warnings Fleet surfaces when a user's pre-existing (higher-priority) index templates conflict with the new Fleet-managed namespace index templates.

Implementation PRs:

Resolves #6545 and #7683

Changes

  • reference/fleet/data-streams-namespace-custom.md (new): How-to for the feature — what namespace index templates are, the <type>-<dataset>@namespace.<namespace> naming and priority (250) scheme, the user-managed <namespace>@custom component template, the composed_of precedence order, UI opt-in (integration Settings tab and policy editor Advanced options), the opt-in APIs (namespace_customization_enabled_for and the _bulk_namespace_customization endpoint), and how to turn the feature off.
  • reference/fleet/data-streams-template-conflicts.md (new): Troubleshooting page for overlapping index templates — when overlaps arise, how Fleet detects and reports them (preflight _simulate_index check, priority-based outcomes), and three resolution paths (use Fleet-managed templates, keep existing templates, or combine both).
  • reference/fleet/data-streams.md: Adds the namespace-level @custom tier to the Edit the Elasticsearch index template section and clarifies the @custom precedence chain (data stream > namespace > package > type).
  • Pointers and cross-links: Adds references to the new pages from the ILM tutorial landing page, Scenario 3, and the integration assets best practices page, and wires both new pages into toc.yml.

Scope

The ILM policy picker is intentionally out of scope and will be documented separately as part of #7701.

Generative AI disclosure

  1. Did you use a generative AI (GenAI) tool to assist in creating this contribution?
  • Yes
  • No
  1. If you answered "Yes" to the previous question, please specify the tool(s) and model(s) used (e.g., Google Gemini, OpenAI ChatGPT-4, etc.).

Tool(s) and model(s) used: Cursor / Claude Opus 4.8

Co-authored-by: Cursor <cursoragent@cursor.com>
@github-actions

github-actions Bot commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

Elastic Docs AI PR menu

Check the box to run an AI review for this pull request.

  • Review docs changes (docs-review). Status: not started.

Powered by GitHub Agentic Workflows and docs-actions. For more information, reach out to the docs team.

@github-actions

github-actions Bot commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

@github-actions

github-actions Bot commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

Elastic Docs Style Checker (Vale)

Summary: 1 suggestion found

💡 Suggestions (1): Optional style improvements. Apply when helpful.
File Line Rule Message
reference/fleet/data-streams-scenario3.md 84 Elastic.Clone Use Cloning only when referring to cloning a GitHub repository or creating a copy that is linked to the original. Often confused with 'copy' and 'duplicate'.

The Vale linter checks documentation changes against the Elastic Docs style guide. To use Vale locally or report issues, refer to Elastic style guide for Vale.

Comment thread reference/fleet/data-streams-namespace-custom.md Outdated

The namespaces you opt in for an integration are the same across the whole cluster. The index templates and component templates {{fleet}} creates are also shared across the cluster.

If your {{kib}} space restricts which namespaces are allowed, you can only enable namespace index templates for namespaces that match those restrictions.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Side note: this feature seems pretty hidden. It is not exposed in the UI and the only reference I found is https://github.com/elastic/kibana/blob/main/x-pack/platform/plugins/shared/fleet/dev_docs/space_awareness.md. Even the two API endpoints are not surfaced in Kibana APIs reference since they lack an OAS tag. I'm not sure if this was intentional (implementation PR for reference), maybe something to look at.

Comment thread reference/fleet/data-streams-namespace-custom.md Outdated
Comment thread reference/fleet/data-streams-namespace-custom.md Outdated

Use this path only when you understand which template should win for each data stream.

1. Review every overlapping template's priority against `250`. You can't change the priority of the {{fleet}}-managed namespace index template; it's always `250`.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ A user with manage_index_templates cluster privilege can change the priority of namespace index templates. These are created with a fixed priority which is the base template priority + 50, i.e.:

  • 250 for standard data streams (base 200)
  • 200 for dataset_is_prefix data streams (base 150)

However an importantly, Fleet will reset the priority on the next package reinstall or upgrade, so it is not advised to change the priority manually unless it's specifically necessary.

I would suggest something along these lines:
Fleet sets the priority of namespace index templates to 250 for most integrations (200 for integrations with dataset_is_prefix: true). While you can change the priority directly using the Elasticsearch Index Template API, Fleet will restore the default value whenever the integration's templates are reinstalled or resynchronized.

@vishaangelova vishaangelova Aug 4, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in bcf8289 - what do you think?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great, thanks for the changes!

@jillguyonnet

Copy link
Copy Markdown
Member

Thanks so much @vishaangelova this looks great and it should be easy to add #7701 on top of it.

I spotted one statement that needs correcting: #7705 (comment)

I realise this didn't come up in our discussion, apologies about that. Please let me know if I can clarify anything.

My other comments are minor nits.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

2 participants