[9.4](backport #50674) [Metricbeat] Add elasticsearch/security_stats metricset

[mergify]

Type of change: Enhancement

Proposed commit message

[Metricbeat] Add elasticsearch/security_stats metricset

WHAT:

• Adds a new security_stats metricset to the Elasticsearch module
  that scrapes the per-node GET /_security/stats endpoint
  introduced in Elasticsearch 9.2.
• The first metric exposed is the Document Level Security (DLS)
  cache: entries, memory, hits, misses, evictions, hit/miss latency.
• Each event is enriched with node.{name,roles,version} via a
  single filter-path-scoped /_nodes call per scrape, exposed as a
  reusable NodeEnrichment helper on the module's MetricSet so
  future per-node metricsets can adopt it.
• Wires the metricset into xpackEnabledMetricSets so monitoring
  deployments using xpack.enabled: true route events to
  .monitoring-* indices.
• Gates on ES 9.2.0 (in-memory version compare) and on
  xpack.Features.Security.Enabled (proactive GET /_xpack probe),
  mirroring the conditional-availability pattern in ccr and
  ml_job. Aggregates /_nodes enrichment failures via
  errors.Join rather than logging Debug, so they surface in
  self-monitoring (matches node_stats).

WHY:

• Gives Stack Monitoring fleet-wide visibility into DLS cache
  health (cache thrash, oversized working sets, unhealthy hit/miss
  ratios) — currently invisible across the fleet.
• Lays the foundation for additional security subsystems the
  endpoint may expose in future ES releases (additive sibling
  paths under roles.*).

Checklist

• My code follows the style guidelines of this project
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• I have made corresponding change to the default configuration files
• I have added tests that prove my fix is effective or that my feature works.
• I have added an entry in ./changelog/fragments using the changelog tool.

Disruptive User Impact

None for end users — the metricset is opt-in via configuration.

Shared test infrastructure (metricbeat/docker-compose.yml and
x-pack/metricbeat/docker-compose.yml) is unchanged from main: the
elasticsearch test service still runs with xpack.security.enabled=false,
so /_security/stats is not registered in CI and TestFetch/security_stats
is skipped unconditionally with a TODO pointing at a follow-up PR that
migrates the metricbeat compose stack to an x-pack-security-enabled posture
(file-realm users, Kibana credentials, test fixture auth). At that point
the skip becomes vacuous and the metricset is exercised against a real
/_security/stats response.

In production, the metricset checks GET /_xpack on each scrape — the same
proactive feature-availability pattern used by ccr and ml_job — and
short-circuits with a throttled debug log when features.security.enabled
is false, so clusters running without security get no events and no error
spam.

How to test this PR locally

1. From metricbeat/, run the unit tests:

   go test -v -race -count=1 ./module/elasticsearch/security_stats/...
   

   Expect TestMapper, TestEmpty, TestSkipsNodesWithoutDLSStats,
   TestRejectsMalformedResponse, and TestEnrichmentMissingForNode
   to pass.

2. Spin up ES 9.2+ via the module's docker-compose and confirm the
   integration suite still passes — TestFetch/security_stats skips
   under the unmodified compose stack, all other elasticsearch
   metricsets pass:

   cd metricbeat
   mage docker:composeUp
   go test -v -tags=integration -run 'TestFetch$' ./module/elasticsearch/...
   

   Expect every TestFetch/* PASS, with TestFetch/security_stats
   SKIP and the message
   /_security/stats requires xpack.security.enabled=true on the test cluster (deferred to a follow-up compose change).

   To exercise the metricset end-to-end against a real
   /_security/stats payload, point a metricbeat instance at any
   local ES 9.2+ cluster that has xpack.security.enabled: true and
   configure it with metricsets: [security_stats]. The next
   iteration of this work (the compose-migration follow-up) will
   fold this into the standard integration suite.

3. Confirm version gating: point the metricset at an ES < 9.2 host
   and verify Fetch returns nil with a (throttled) debug log.

Related issues

• Relates [elasticsearch] Add security_stats data stream (DLS bitset cache, ES 9.2+) integrations#18680

---

This is an automatic backport of pull request #50674 done by [Mergify](https://mergify.com).

[github-actions]

🤖 GitHub comments

Just comment with:

• run docs-build : Re-trigger the docs validation. (use unformatted text in the comment!)
• /test : Run the Buildkite pipeline.

[github-actions]

🔍 Preview links for changed docs

• docs/reference/metricbeat/exported-fields-elasticsearch.md
• docs/reference/metricbeat/metricbeat-metricset-elasticsearch-security_stats.md
• docs/reference/metricbeat/metricbeat-module-elasticsearch.md
• docs/reference/metricbeat/metricbeat-modules.md
• metricbeat/module/elasticsearch/security_stats/_meta/docs.md

[github-actions]

Vale Linting Results

Summary: 1 warning, 2 suggestions found

⚠️ Warnings (1)

File	Line	Rule	Message
docs/reference/metricbeat/metricbeat-metricset-elasticsearch-security_stats.md	2	Elastic.MappedPages	mapped_pages should only be added or updated in rare scenarios. Talk with your local technical writer before pushing changes to this key.

💡 Suggestions (2)

File	Line	Rule	Message
docs/reference/metricbeat/metricbeat-metricset-elasticsearch-security_stats.md	13	Elastic.WordChoice	Consider using 'select, press, visits' instead of 'hit', unless the term is in the UI.
docs/reference/metricbeat/metricbeat-metricset-elasticsearch-security_stats.md	17	Elastic.WordChoice	Consider using 'deactivated, deselected, hidden, turned off, unavailable' instead of 'disabled', unless the term is in the UI.

---

The Vale linter checks documentation changes against the Elastic Docs style guide.

To use Vale locally or report issues, refer to Elastic style guide for Vale.