Skip to content

test(debugger): ensure span error budgets are reasonable#6224

Merged
tylfin merged 1 commit into
mainfrom
tyler.finethy/DEBUG-3797
Mar 4, 2026
Merged

test(debugger): ensure span error budgets are reasonable#6224
tylfin merged 1 commit into
mainfrom
tyler.finethy/DEBUG-3797

Conversation

@tylfin
Copy link
Copy Markdown
Member

@tylfin tylfin commented Feb 5, 2026

Motivation

Add a test to verify that span decoration probes with improper expressions (e.g., "Cannot dereference field") have their error logs rate-limited to at most once per second. Currently marked as a bug for Java since span decoration probes lack probe budgets, producing 150 error snapshots for 150 requests instead of the expected ~1 per second.

Changes

  • Added a new test method test_span_probe_expression_budgets in tests/debugger/test_debugger_probe_budgets.py to verify that error logs from span probe expression failures are rate-limited to at most 1 per second, with appropriate assertions and handling for language-specific limitations.
  • Introduced a helper method setup_span_probe_expression_budgets to configure and trigger the span probe expression budget scenario in the same test file.
  • Added a new probe configuration file probe_span_method_budgets_expression.json under tests/debugger/utils/probes/ to define a span probe with an intentionally failing expression, used for testing error logging and rate limiting.
  • Imported the bug decorator from utils in tests/debugger/test_debugger_probe_budgets.py to allow marking tests with known bugs, specifically for Java.

Workflow

  1. ⚠️ Create your PR as draft ⚠️
  2. Work on you PR until the CI passes
  3. Mark it as ready for review
    • Test logic is modified? -> Get a review from RFC owner.
    • Framework is modified, or non obvious usage of it -> get a review from R&P team

🚀 Once your PR is reviewed and the CI green, you can merge it!

🛟 #apm-shared-testing 🛟

Reviewer checklist

  • Anything but tests/ or manifests/ is modified ? I have the approval from R&P team
  • A docker base image is modified?
    • the relevant build-XXX-image label is present
  • A scenario is added, removed or renamed?

@tylfin tylfin requested review from a team as code owners February 5, 2026 19:35
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Feb 5, 2026

CODEOWNERS have been resolved as:

tests/debugger/utils/probes/probe_span_method_budgets_expression.json   @DataDog/debugger @DataDog/system-tests-core
manifests/dotnet.yml                                                    @DataDog/apm-dotnet @DataDog/asm-dotnet
manifests/java.yml                                                      @DataDog/asm-java @DataDog/apm-java
manifests/php.yml                                                       @DataDog/apm-php @DataDog/asm-php
manifests/python.yml                                                    @DataDog/apm-python @DataDog/asm-python
tests/debugger/test_debugger_probe_budgets.py                           @DataDog/debugger @DataDog/system-tests-core

@tylfin tylfin force-pushed the tyler.finethy/DEBUG-3797 branch from d3870b0 to fe883a9 Compare February 5, 2026 19:40
@tylfin tylfin requested review from a team as code owners February 5, 2026 19:40
@tylfin tylfin force-pushed the tyler.finethy/DEBUG-3797 branch from fe883a9 to a87e67b Compare February 5, 2026 19:43
@tylfin tylfin changed the title test(debugger): span probe budgets (#DEBUG-3797) test(debugger): ensure span error budgets are reasonable Feb 5, 2026
@tylfin tylfin requested a review from jpbempel February 5, 2026 20:30
@tylfin tylfin enabled auto-merge (squash) February 5, 2026 21:56
Copy link
Copy Markdown
Collaborator

@nccatoni nccatoni left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM for @DataDog/system-tests-core, you should also get a review from someone familiar with the feature

@tylfin tylfin force-pushed the tyler.finethy/DEBUG-3797 branch from 7ddbed9 to 2b6dc51 Compare February 10, 2026 14:17
@datadog-official
Copy link
Copy Markdown

datadog-official Bot commented Feb 10, 2026

✅ Tests

🎉 All green!

❄️ No new flaky tests detected
🧪 All tests passed

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 7f3bc19 | Docs | Datadog PR Page | Was this helpful? React with 👍/👎 or give us feedback!

@tylfin tylfin force-pushed the tyler.finethy/DEBUG-3797 branch 3 times, most recently from df28155 to 49505ee Compare February 12, 2026 15:27
@tylfin tylfin requested review from a team as code owners February 12, 2026 15:27
@tylfin tylfin requested review from avara1986 and florentinl and removed request for a team February 12, 2026 15:27
@tylfin tylfin force-pushed the tyler.finethy/DEBUG-3797 branch from 49505ee to 949efe2 Compare February 12, 2026 16:39
@tylfin tylfin force-pushed the tyler.finethy/DEBUG-3797 branch from 323016a to 643c88c Compare February 18, 2026 15:32
Copy link
Copy Markdown
Contributor

@daniel-romano-DD daniel-romano-DD left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Mark test_span_probe_expression_budgets as irrelevant for Python, Dotnet, and PHP
@tylfin tylfin force-pushed the tyler.finethy/DEBUG-3797 branch from 3ce8a2a to 7f3bc19 Compare March 3, 2026 19:14
@tylfin tylfin disabled auto-merge March 3, 2026 19:20
@tylfin tylfin merged commit 310b787 into main Mar 4, 2026
431 of 433 checks passed
@tylfin tylfin deleted the tyler.finethy/DEBUG-3797 branch March 4, 2026 14:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants