enhancement(tag_cardinality_limit transform): add exclude_tags option to bypass cardinality limiting

[kaarolch]

Summary

Adds a new exclude_tags option to the tag_cardinality_limit transform. Tag keys listed in exclude_tags bypass cardinality limiting entirely: they are passed through unchanged on every event, are not counted against metric
value_limit, and never enter the cache.

The option can be set globally and/or per-metric (via per_metric_limits.<name>); when both are set, the
effective exclusion list is the union of the two. This is useful for tags whose high cardinality is intentional and must be preserved, such as kube_pod_name or tenant_id, while still bounding the cardinality of other tags on the same metric.

Why

Closes #20972. Similar to @btkostner we have multiple scenarios when most of metric tags should be protected by cardinality tags transform but there are bunch of tags what should be always bypassed.

Vector configuration

api:
  enabled: true
sources:
  http_metrics:
    type: http_server
    address: "0.0.0.0:8080"
    decoding:
      codec: influxdb
    path: "/api/v1/write"
transforms:
  cardinality_limit:
    type: tag_cardinality_limit
    inputs:
      - http_metrics
    mode: exact
    value_limit: 3
    limit_exceeded_action: drop_tag
    # New option: these tag keys bypass cardinality limiting entirely.
    exclude_tags:
      - kube_pod_name
sinks:
  console_debug:
    type: console
    inputs:
      - cardinality_limit
    encoding:
      codec: json

Per metric name example:

transforms:
  cardinality_limit:
    type: tag_cardinality_limit
    inputs: [http_metrics]
    mode: exact
    value_limit: 3
    limit_exceeded_action: drop_tag
    exclude_tags: [kube_pod_name]
    per_metric_limits:
      http_requests_total:
        value_limit: 5
        exclude_tags: [tenant_id]

How did you test this PR?

Unit tests (added in src/transforms/tag_cardinality_limit/tests.rs):

• exclude_tags_drop_tag_passthrough_hashset
• exclude_tags_drop_tag_passthrough_bloom
• exclude_tags_drop_event_passthrough_hashset
• exclude_tags_drop_event_passthrough_bloom
• exclude_tags_merge_global_and_per_metric

These cover both Exact and Probabilistic modes, both limit_exceeded_action values (drop_tag and drop_event), and the global + per-metric union semantics.

cargo test -p vector --lib transforms::tag_cardinality_limit

Local verification commands run from the repo root:

Local verification commands:

cargo fmt --all -- --check
cargo check -p vector
cargo clippy -p vector --tests -- -D warnings
cargo test  -p vector --lib transforms::tag_cardinality_limit
make check-generated-docs 

E2E test with the config above and a small Python sender (stdlib only) that publishes one measurement carrying six tag keys per request — service, env (constant), kube_pod_name (excluded, varies), and region / instance / version (not excluded, vary):

{"name":"http_requests_total_count","tags":{"env":"staging","instance":"node-1","kube_pod_name":"pod-1","region":"region-1","service":"api","version":"v1.1"},"timestamp":"2026-04-28T10:30:05.665264Z","kind":"absolute","gauge":{"value":1.0}}
{"name":"http_requests_total_latency_ms","tags":{"env":"staging","instance":"node-1","kube_pod_name":"pod-1","region":"region-1","service":"api","version":"v1.1"},"timestamp":"2026-04-28T10:30:05.665264Z","kind":"absolute","gauge":{"value":11.0}}
{"name":"http_requests_total_count","tags":{"env":"staging","instance":"node-2","kube_pod_name":"pod-2","region":"region-2","service":"api","version":"v1.2"},"timestamp":"2026-04-28T10:30:05.689172Z","kind":"absolute","gauge":{"value":2.0}}
{"name":"http_requests_total_latency_ms","tags":{"env":"staging","instance":"node-2","kube_pod_name":"pod-2","region":"region-2","service":"api","version":"v1.2"},"timestamp":"2026-04-28T10:30:05.689172Z","kind":"absolute","gauge":{"value":12.0}}
{"name":"http_requests_total_count","tags":{"env":"staging","instance":"node-3","kube_pod_name":"pod-3","region":"region-3","service":"api","version":"v1.3"},"timestamp":"2026-04-28T10:30:05.690437Z","kind":"absolute","gauge":{"value":3.0}}
{"name":"http_requests_total_latency_ms","tags":{"env":"staging","instance":"node-3","kube_pod_name":"pod-3","region":"region-3","service":"api","version":"v1.3"},"timestamp":"2026-04-28T10:30:05.690437Z","kind":"absolute","gauge":{"value":13.0}}
{"name":"http_requests_total_count","tags":{"env":"staging","kube_pod_name":"pod-4","service":"api"},"timestamp":"2026-04-28T10:30:05.691647Z","kind":"absolute","gauge":{"value":4.0}}
{"name":"http_requests_total_latency_ms","tags":{"env":"staging","kube_pod_name":"pod-4","service":"api"},"timestamp":"2026-04-28T10:30:05.691647Z","kind":"absolute","gauge":{"value":14.0}}
{"name":"http_requests_total_count","tags":{"env":"staging","kube_pod_name":"pod-5","service":"api"},"timestamp":"2026-04-28T10:30:05.692986Z","kind":"absolute","gauge":{"value":5.0}}
{"name":"http_requests_total_latency_ms","tags":{"env":"staging","kube_pod_name":"pod-5","service":"api"},"timestamp":"2026-04-28T10:30:05.692986Z","kind":"absolute","gauge":{"value":15.0}}
{"name":"http_requests_total_count","tags":{"env":"staging","kube_pod_name":"pod-6","service":"api"},"timestamp":"2026-04-28T10:30:05.694419Z","kind":"absolute","gauge":{"value":6.0}}
{"name":"http_requests_total_latency_ms","tags":{"env":"staging","kube_pod_name":"pod-6","service":"api"},"timestamp":"2026-04-28T10:30:05.694419Z","kind":"absolute","gauge":{"value":16.0}}
{"name":"http_requests_total_count","tags":{"env":"staging","kube_pod_name":"pod-7","service":"api"},"timestamp":"2026-04-28T10:30:05.695741Z","kind":"absolute","gauge":{"value":7.0}}
{"name":"http_requests_total_latency_ms","tags":{"env":"staging","kube_pod_name":"pod-7","service":"api"},"timestamp":"2026-04-28T10:30:05.695741Z","kind":"absolute","gauge":{"value":17.0}}
{"name":"http_requests_total_count","tags":{"env":"staging","kube_pod_name":"pod-8","service":"api"},"timestamp":"2026-04-28T10:30:05.696966Z","kind":"absolute","gauge":{"value":8.0}}

Change Type

• Bug fix
• New feature
• Dependencies
• Non-functional (chore, refactoring, docs)
• Performance

Is this a breaking change?

• Yes
• No

The new field defaults to an empty list (#[serde(default)]), so existing configurations behave exactly as before.

Does this PR include user facing changes?

• Yes. Please add a changelog fragment based on our guidelines.
• No. A maintainer will apply the no-changelog label to this PR.

References

• Related:
  • tag_cardinality_limit tag exclusion #20972
  • Request Tag Cardinality Limit with tags exclusion option OR Best way to handle unexpected high cardinality tag #22438

Notes

• Please read our Vector contributor resources.
• Do not hesitate to use @vectordotdev/vector to reach out to us regarding this PR.
• Some CI checks run only after we manually approve them.
  • We recommend adding a pre-push hook, please see this template.
  • Alternatively, we recommend running the following locally before pushing to the remote branch:
    • make fmt
    • make check-clippy (if there are failures it's possible some of them can be fixed with make clippy-fix)
    • make test
• After a review is requested, please avoid force pushes to help us review incrementally.
  • Feel free to push as many commits as you want. They will be squashed into one before merging.
  • For example, you can run git merge origin master and git push.
• If this PR introduces changes Vector dependencies (modifies Cargo.lock), please
  run make build-licenses to regenerate the license inventory and commit the changes (if any). More details on the dd-rust-license-tool.

[kaarolch]

btw. we can optimized redundant scans of per_metric_limits per event transform_one now scans per_metric_limits three times per event:

• once to compute has_per_metric_config (mod.rs:152-155)
• once inside effective_exclude_tags (mod.rs:60-62)
• once inside get_config_for_metric when picking the action (mod.rs:172)
  but as @pront mentioned in other PR current vector backlog is quite full so smaller PR better. I can address this scan in separate PR.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: da116e6bac

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".