Skip to content

Fix Kueue metric metadata and add log collection - #24700

Merged
lucia-sb merged 40 commits into
masterfrom
lucia-sb/kueue
Aug 4, 2026
Merged

Fix Kueue metric metadata and add log collection#24700
lucia-sb merged 40 commits into
masterfrom
lucia-sb/kueue

Conversation

@lucia-sb

@lucia-sb lucia-sb commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

What does this PR do?

Fixes the Kueue integration's metric metadata and adds log-collection support.

  • Metadata fix (dashboards): metadata.csv listed ~30 counter/histogram metrics under their base names as gauge (e.g. kueue.controller.runtime.reconcile, kueue.controller.runtime.reconcile_time.seconds, kueue.process.cpu.seconds, the workqueue.* family). The check actually submits them as monotonic counters/histograms — .count/.bucket/.sum — so dashboards querying the base names showed no data. Corrected the entries to the real submitted names and count type.
  • Dual-form metrics (native_dynamic): Kueue exposes a few metrics as both a gauge and a _total counter (go.memstats.alloc_bytes, kueue.finished_workloads). The standard metric map collapsed the counter onto the gauge's name, so the counter was submitted as a gauge — polluting the gauge and never emitting a .count. Mapped these with the native_dynamic type so both the gauge and its .count are submitted correctly, and added the missing .count metadata rows.
  • Metric-existence audit: cross-checked every metric in metadata.csv against Kueue v0.18 source (pkg/metrics/metrics.go) and the live endpoint. Fixed admission_cycle.preemption_skips (it is a Gauge, not a counter) and removed resource_flavor.quota_reserved_workloads, which Kueue v0.18 does not define.
  • Fixture + tests (so it can't regress): rebuilt tests/fixtures/metrics.txt so the affected families use the real endpoint format, test_check now runs the check twice (so OpenMetrics monotonic counters flush their .count) under the strict two-way metadata assertion, and test_e2e uses rate=True so the e2e metadata assertion validates counters/histograms against the live endpoint.
  • Log collection: added the logs template to spec.yaml (regenerated conf.yaml.example) and a "Log collection" section to the README.
  • E2E metric coverage: expanded the kind environment to actually emit the previously-uncovered metrics — cohort / fair sharing, GPU, a custom (other) resource, a not-active ClusterQueue, and preemption / eviction.
  • E2E robustness: the env now picks non-colliding kind service/pod subnets (avoids host/VPN route collisions) and disables Kueue's visibility server, whose self-signed cert bootstrap crashloops the controller in some clusters.

Motivation

Several Kueue dashboard widgets (reconciles, reconcile time, process CPU, workqueue, …) showed no data because the metadata metric names/types didn't match what the Agent submits. The metadata and the test fixture were a matched-but-wrong pair, and the existing metadata tests didn't catch it (the unit test stripped .count/.bucket/.sum suffixes; the e2e never flushed counters). This corrects the metadata, closes the test gap, and rounds out the integration's metric and log coverage.

Review checklist (to be filled by reviewers)

  • Feature or bugfix MUST have appropriate tests (unit, integration, e2e)
  • Add qa/required if this PR needs QA validation, or qa/skip-qa if it does not. Exactly one of the two is required.
  • If you need to backport this PR to another branch, you can add the backport/<branch-name> label to the PR and it will automatically open a backport PR once this one is merged

🤖 Generated with Claude Code

- Correct metadata.csv for counter/histogram metrics: they are submitted as
  .count/.bucket/.sum (monotonic_count), not base-name gauges, so dashboards
  querying the base names showed no data. Corrects ~30 entries.
- Rebuild tests/fixtures/metrics.txt to the real endpoint format for the
  affected families so unit tests reflect what Kueue actually exposes.
- Strengthen tests so this can't regress: test_check runs the check twice so
  monotonic counters flush and the strict metadata assertion validates them;
  test_e2e uses rate=True so the e2e metadata check validates counters too.
- Add Kueue log collection support (logs template in spec.yaml, regenerated
  conf.yaml.example, README section).
- Expand the E2E environment to exercise cohort/fair-sharing, GPU, a custom
  resource, a not-active ClusterQueue, and preemption/eviction metrics.
- Make the E2E env robust locally: choose non-colliding kind subnets and
  disable Kueue's visibility server, whose cert bootstrap crashloops some
  clusters.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@lucia-sb lucia-sb added the qa/skip-qa Automatically skip this PR for the next QA label Jul 28, 2026
@datadog-prod-us1-3

datadog-prod-us1-3 Bot commented Jul 28, 2026

Copy link
Copy Markdown

Tests  Code Coverage

🎉 All green!

🧪 All tests passed
❄️ No new flaky tests detected

🎯 Code Coverage (details)
Patch Coverage: 41.48%
Overall Coverage: 74.40% (-12.00%)

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 92dddc7 | Docs | Datadog PR Page | Give us feedback!

lucia-sb and others added 24 commits July 28, 2026 12:59
…adata

Verified every metric in metadata.csv against Kueue v0.18 source
(pkg/metrics/metrics.go) and the live endpoint:

- kueue.resource_flavor.quota_reserved_workloads does not exist in Kueue
  v0.18 (no resource_flavor metric is defined). Removed it from the
  metadata, metric map, test fixture, and expected-metrics list.
- kueue.admission_cycle.preemption_skips is a Gauge, not a counter.
  Reverted it to the base gauge name.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
kueue_finished_workloads is a dual-form metric: Kueue exposes both a
gauge (kueue_finished_workloads) and a counter
(kueue_finished_workloads_total). The standard metric map collapsed the
counter onto the gauge's name, so the counter was submitted as a gauge —
polluting kueue.finished_workloads and never emitting a .count.

Map it as native_dynamic (the same mechanism used for
go.memstats.alloc_bytes and the local_queue transformer) so the gauge and
the cumulative .count are both submitted correctly. Added the
kueue.finished_workloads.count metadata row and fixture coverage.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
41 rows carried over from master differed only by having the surrounding
double quotes stripped from the description column, with no change to the
description text. Restoring the original quoting shrinks the diff for this
file from 135 lines to 53, leaving only the substantive changes: the 22
base-name gauge rows replaced by 31 .count/.bucket/.sum count rows and the
resource_flavor.quota_reserved_workloads removal.

Parsing the file before and after with csv.reader yields identical rows, so
this is a formatting-only change.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The parser strips '_total' from counter families, so the
kueue_local_queue_finished_workloads_total key was never looked up. Both
forms already resolve correctly because the local-queue custom transformer
wraps get_native_dynamic_transformer; add the resulting .count metric to
the fixture assertions so that stays covered.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The four families converted from gauges to histograms had no le="+Inf"
bucket and had bucket lines on only the first series, a shape no real
endpoint produces. Emit the full bucket set plus +Inf for every series so
the +Inf submission path is exercised.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Derive the prefix from the octet count so macOS netstat's abbreviated
network routes (10 meaning 10.0.0.0/8) are not narrowed to /32 and missed.
Log the chosen pair, including the fallback, since the subnets are the
first thing to check when the controller crashloops. Turn the builder into
a context manager so the rendered config is unlinked on teardown.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Patching containers/0 blind succeeds even if the flag lands on a sidecar,
turning an upstream manifest change into a crashloop with an unrelated
error. Resolve the index by container name and fail loudly if it is absent.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Both preempt Jobs asked for a full core against a 1-core quota, so the
admitted pod oversubscribed a 2-vCPU runner on top of kube-system and the
existing workloads. Preemption is quota accounting, so the metrics are
identical at a fifth of the resources and no pods are left Pending.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The ddev config path and kind cluster name both encoded py3.13-v0.18.0 as
a literal, so bumping python or KUEUE_VERSION broke the workload-events
test with FileNotFoundError. Build both from get_active_env().

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Side-load the workload image once instead of pulling it from Docker Hub for
every Job pod, explain why min_collection_interval dropped to 30, narrow
the enabled integration frameworks to the one the tests exercise, and drive
the workload-events check through run() so the test stops calling a private
initialization directly.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Only derive `kueue_preempted_by` from evictions whose reason is
`Preempted`. A `FlavorMigration` eviction carries a lookalike message
("Evicted to accommodate a workload (UID: ...)") that was being mined
for a preemptor UID that does not exist.

Also correct the log collection section of the README, extract the
kubectl helpers into tests/kube.py, and extend the e2e coverage.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`assert_metric_has_tags` covers every metric in `EXPECTED_METRIC_TAGS`. The
custom helper is now reserved for the status families, whose tags carry no
information on their own: a series exists for every state of every queue, and
only the value says which state is current.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The version literal lived in both hatch.toml and a conftest fallback, and the
workload image lived in a constant kept in sync with the Job manifests by a
comment. Read both from their single source instead, so a bump has one place to
change and a manifest using a different image is still preloaded.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The status families needed a value paired with tags, which the custom helper
provided by matching a tag subset. `assert_metric` covers it by matching the
exact tag set, with the per-run endpoint tag taken from the saved instance
config, so the tests now use only built-in assertions.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

@datadog-prod-us1-3 datadog-prod-us1-3 Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Datadog Autotest: PASS

More details

The changed Kueue metric mappings and workload-event paths behaved correctly across the exercised counter, histogram, duplicate, namespace, eviction, and fallback scenarios. The repository kind E2E harness exited without producing a result file in this sandbox, so live-controller validation remains the only gap.

Was this helpful? React 👍 or 👎

📊 Validated against 8 scenarios · Open Bits AI session

🤖 Datadog Autotest · Commit 7a23490 · What is Autotest? · @DataDog review to ask questions · Any feedback? Reach out in #autotest

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 7a234909b0

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread kueue/tests/common.py
Comment on lines +64 to +68
- template: logs
example:
- type: docker
source: kueue
service: <SERVICE>

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I assume this has no effect on k8s deployments right?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, this doesn’t affect Kubernetes deployments. I used type: docker because that is the standard log example defined in other integrations too, including Kubernetes-based ones like Kuma.

The README has a more detailed example of how to set them up for Kueue.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good, although I'm not sure if Kueue can work outside of k8s. In any case, not blocking the PR on this.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I could change it to type: file; the issue is that the template requires a type.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can leave it like this.

gjulianm
gjulianm previously approved these changes Aug 3, 2026
@buraizu buraizu self-assigned this Aug 3, 2026

@buraizu buraizu left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good overall, just requesting a few minor updates

Comment thread kueue/README.md Outdated
Comment thread kueue/README.md Outdated
Comment thread kueue/README.md Outdated
Co-authored-by: Bryce Eadie <bryce.eadie@datadoghq.com>
@temporal-github-worker-1
temporal-github-worker-1 Bot dismissed gjulianm’s stale review August 4, 2026 08:13

Review from gjulianm is dismissed. Related teams and files:

  • gpu-monitoring-agent
    • kueue/README.md
Co-authored-by: Bryce Eadie <bryce.eadie@datadoghq.com>
@lucia-sb
lucia-sb requested review from buraizu and gjulianm August 4, 2026 09:06
gjulianm
gjulianm previously approved these changes Aug 4, 2026
Co-authored-by: Bryce Eadie <bryce.eadie@datadoghq.com>
@temporal-github-worker-1
temporal-github-worker-1 Bot dismissed gjulianm’s stale review August 4, 2026 16:31

Review from gjulianm is dismissed. Related teams and files:

  • gpu-monitoring-agent
    • kueue/README.md
lucia-sb and others added 2 commits August 4, 2026 18:47
PR #24760 added a third `_get_metric_tags` call site on master while this
branch renamed the helper to `get_metric_tags`. The two changes touched
different regions, so the merge was textually clean but left the new call
site referencing the old name, failing lint with F821.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@dd-octo-sts

dd-octo-sts Bot commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

Validation Report

All 21 validations passed.

Show details
Validation Description Status
agent-reqs Verify check versions match the Agent requirements file
ci Validate CI configuration and code coverage settings
codeowners Validate every integration has a CODEOWNERS entry
config Validate default configuration files against spec.yaml
dep Verify dependency pins are consistent and Agent-compatible
http Validate integrations use the HTTP wrapper correctly
imports Validate check imports do not use deprecated modules
integration-style Validate check code style conventions
jmx-metrics Validate JMX metrics definition files and config
labeler Validate PR labeler config matches integration directories
legacy-signature Validate no integration uses the legacy Agent check signature
license-headers Validate Python files have proper license headers
licenses Validate third-party license attribution list
metadata Validate metadata.csv metric definitions
models Validate configuration data models match spec.yaml
openmetrics Validate OpenMetrics integrations disable the metric limit
package Validate Python package metadata and naming
qa-label Validate the pull request declares whether it needs QA for the next Agent release
readmes Validate README files have required sections
saved-views Validate saved view JSON file structure and fields
version Validate version consistency between package and changelog

View full run

@lucia-sb
lucia-sb requested a review from gjulianm August 4, 2026 16:56
@lucia-sb
lucia-sb added this pull request to the merge queue Aug 4, 2026
Merged via the queue into master with commit f0987e3 Aug 4, 2026
58 checks passed
@lucia-sb
lucia-sb deleted the lucia-sb/kueue branch August 4, 2026 16:59
@dd-octo-sts dd-octo-sts Bot added this to the 7.83.0 milestone Aug 4, 2026
github-actions Bot pushed a commit that referenced this pull request Aug 4, 2026
* Fix Kueue metric metadata and add log collection

- Correct metadata.csv for counter/histogram metrics: they are submitted as
  .count/.bucket/.sum (monotonic_count), not base-name gauges, so dashboards
  querying the base names showed no data. Corrects ~30 entries.
- Rebuild tests/fixtures/metrics.txt to the real endpoint format for the
  affected families so unit tests reflect what Kueue actually exposes.
- Strengthen tests so this can't regress: test_check runs the check twice so
  monotonic counters flush and the strict metadata assertion validates them;
  test_e2e uses rate=True so the e2e metadata check validates counters too.
- Add Kueue log collection support (logs template in spec.yaml, regenerated
  conf.yaml.example, README section).
- Expand the E2E environment to exercise cohort/fair-sharing, GPU, a custom
  resource, a not-active ClusterQueue, and preemption/eviction metrics.
- Make the E2E env robust locally: choose non-colliding kind subnets and
  disable Kueue's visibility server, whose cert bootstrap crashloops some
  clusters.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* Add changelog entries for #24700

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* Remove non-existent metric and fix preemption_skips type in Kueue metadata

Verified every metric in metadata.csv against Kueue v0.18 source
(pkg/metrics/metrics.go) and the live endpoint:

- kueue.resource_flavor.quota_reserved_workloads does not exist in Kueue
  v0.18 (no resource_flavor metric is defined). Removed it from the
  metadata, metric map, test fixture, and expected-metrics list.
- kueue.admission_cycle.preemption_skips is a Gauge, not a counter.
  Reverted it to the base gauge name.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* Collect kueue.finished_workloads.count via native_dynamic

kueue_finished_workloads is a dual-form metric: Kueue exposes both a
gauge (kueue_finished_workloads) and a counter
(kueue_finished_workloads_total). The standard metric map collapsed the
counter onto the gauge's name, so the counter was submitted as a gauge —
polluting kueue.finished_workloads and never emitting a .count.

Map it as native_dynamic (the same mechanism used for
go.memstats.alloc_bytes and the local_queue transformer) so the gauge and
the cumulative .count are both submitted correctly. Added the
kueue.finished_workloads.count metadata row and fixture coverage.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* Restore original description quoting in Kueue metadata.csv

41 rows carried over from master differed only by having the surrounding
double quotes stripped from the description column, with no change to the
description text. Restoring the original quoting shrinks the diff for this
file from 135 lines to 53, leaving only the substantive changes: the 22
base-name gauge rows replaced by 31 .count/.bucket/.sum count rows and the
resource_flavor.quota_reserved_workloads removal.

Parsing the file before and after with csv.reader yields identical rows, so
this is a formatting-only change.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* Drop dead local_queue finished_workloads mapping and cover its counter

The parser strips '_total' from counter families, so the
kueue_local_queue_finished_workloads_total key was never looked up. Both
forms already resolve correctly because the local-queue custom transformer
wraps get_native_dynamic_transformer; add the resulting .count metric to
the fixture assertions so that stays covered.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* Request a 24h metrics-reader token so --dev sessions survive

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* Document why both Kueue cert-bootstrap mitigations are needed

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* Correct the stale reason for excluding counters from e2e assertions

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* Make the fixture histogram families well-formed

The four families converted from gauges to histograms had no le="+Inf"
bucket and had bucket lines on only the first series, a shape no real
endpoint produces. Emit the full bucket set plus +Inf for every series so
the +Inf submission path is exercised.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* Harden kind subnet selection and clean up its temp file

Derive the prefix from the octet count so macOS netstat's abbreviated
network routes (10 meaning 10.0.0.0/8) are not narrowed to /32 and missed.
Log the chosen pair, including the fallback, since the subnets are the
first thing to check when the controller crashloops. Turn the builder into
a context manager so the rendered config is unlinked on teardown.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* Select the Kueue manager container by name when patching feature gates

Patching containers/0 blind succeeds even if the flag lands on a sidecar,
turning an upstream manifest change into a crashloop with an unrelated
error. Resolve the index by container name and fail loudly if it is absent.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* Right-size the preemption scenario to 200m of quota

Both preempt Jobs asked for a full core against a 1-core quota, so the
admitted pod oversubscribed a 2-vCPU runner on top of kube-system and the
existing workloads. Preemption is quota accounting, so the metrics are
identical at a fifth of the resources and no pods are left Pending.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* Derive the e2e env coordinates instead of hardcoding them

The ddev config path and kind cluster name both encoded py3.13-v0.18.0 as
a literal, so bumping python or KUEUE_VERSION broke the workload-events
test with FileNotFoundError. Build both from get_active_env().

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* Tighten remaining Kueue env hygiene

Side-load the workload image once instead of pulling it from Docker Hub for
every Job pod, explain why min_collection_interval dropped to 30, narrow
the enabled integration frameworks to the one the tests exercise, and drive
the workload-events check through run() so the test stops calling a private
initialization directly.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* Fix preempted_by tagging and rework the Kueue e2e test env

Only derive `kueue_preempted_by` from evictions whose reason is
`Preempted`. A `FlavorMigration` eviction carries a lookalike message
("Evicted to accommodate a workload (UID: ...)") that was being mined
for a preemptor UID that does not exist.

Also correct the log collection section of the README, extract the
kubectl helpers into tests/kube.py, and extend the e2e coverage.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* Remove stale comment on the local queue finished_workloads mapping

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* Remove comment on the preempting workload UID pattern

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* Remove comment on the preemption reason gate

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* Remove explanatory comments from the Kueue test metric lists

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* Remove explanatory comments from the Kueue e2e tests

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* Remove explanatory comments from the Kueue unit tests

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* Use the built-in tag assertion for the e2e metric tags

`assert_metric_has_tags` covers every metric in `EXPECTED_METRIC_TAGS`. The
custom helper is now reserved for the status families, whose tags carry no
information on their own: a series exists for every state of every queue, and
only the value says which state is current.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* Derive the Kueue version and workload images instead of duplicating them

The version literal lived in both hatch.toml and a conftest fallback, and the
workload image lived in a constant kept in sync with the Job manifests by a
comment. Read both from their single source instead, so a bump has one place to
change and a manifest using a different image is still preloaded.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* Drop the custom series assertion in favour of assert_metric

The status families needed a value paired with tags, which the custom helper
provided by matching a tag subset. `assert_metric` covers it by matching the
exact tag set, with the per-run endpoint tag taken from the saved instance
config, so the tests now use only built-in assertions.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* Remove explanatory comments from the Kueue test constants

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* Gate the Kueue workload-events e2e test through the dd_agent_check fixture

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* Remove explanatory comments from the Kueue test conftest

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* Simplify kind subnet selection in the Kueue test conftest

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* Let the Kueue Workload discovery loop retry instead of raising on an empty result

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* Restore Kueue metric-surface changelog coverage and harden the test env

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* Clarify the two kueue_preempted_by changelog entries

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* Make kind image side-loading best effort in the Kueue test env

kind load docker-image fails against node images whose containerd config
version the installed kind does not support. The Job pods pull the image
themselves in that case, so a failed side-load must not abort env start.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* Tolerate a missing cluster when minting the Kueue metrics reader token

The dd_environment body re-runs on the tear-down pass, where set_up_env is
false so no cluster is available. A strict token mint aborted teardown.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* Update kueue/README.md

Co-authored-by: Bryce Eadie <bryce.eadie@datadoghq.com>

* Update kueue/README.md

Co-authored-by: Bryce Eadie <bryce.eadie@datadoghq.com>

* Update kueue/README.md

Co-authored-by: Bryce Eadie <bryce.eadie@datadoghq.com>

* Reconcile the metric tag helper rename after merging master

PR #24760 added a third `_get_metric_tags` call site on master while this
branch renamed the helper to `get_metric_tags`. The two changes touched
different regions, so the merge was textually clean but left the new call
site referencing the old name, failing lint with F821.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Bryce Eadie <bryce.eadie@datadoghq.com> f0987e3
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants