fix(diagnostics): never send diagnostics from test environments#34
Merged
Conversation
Internal SDK diagnostics were enabled by default with no test-environment awareness, so any test suite that called track() could ship real OTLP metadata (host/OS/runtime info) to the diagnostics collector. Staying silent in tests is now the default, enforced at two layers: - SDK-level guard (protects every consumer): diagnostics auto-disable whenever a test environment is detected (PYTEST_CURRENT_TEST or PYTEST_VERSION set). No configuration needed; consumer CI sends nothing. - Repo-level guard (protects this repo's CI): a top-level tests/conftest.py sets DISABLE_DIAGNOSTICS=true before any test runs, so no test ordering or future change to the detection logic can leak traffic from our own suite. DISABLE_DIAGNOSTICS parsing is tightened: explicit falsy values (false/0/no/off) are now a deliberate opt-in that overrides test detection (the escape hatch to re-enable diagnostics under pytest), and whitespace-only values are treated as unset. Diagnostics-specific tests opt back in per-file with mocked HTTP. README and the disable_diagnostics docstring document the new behavior. Counterpart of the TypeScript fix: agentcathq/agentcat-typescript-sdk#44.
naji247
force-pushed
the
fix/disable-diagnostics-in-test-envs
branch
from
July 2, 2026 19:50
5188b67 to
2e6c402
Compare
kashishhora
approved these changes
Jul 2, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
MCPCat's internal SDK diagnostics exist to catch broken installs — they should never fire from a test run, neither ours nor yours. Before this change, diagnostics were enabled by default with no test-environment awareness, so any test suite that called
track()could ship real OTLP metadata (host/OS/runtime info) to the diagnostics collector. This PR makes staying silent in tests the default, enforced at two layers:PYTEST_CURRENT_TESTorPYTEST_VERSIONset. No configuration needed; your CI sends nothing. Anyone who genuinely needs diagnostics active in a test context can opt back in explicitly withDISABLE_DIAGNOSTICS=false.tests/conftest.pysetsDISABLE_DIAGNOSTICS=truebefore any test runs, so no test ordering or future change to the detection logic can leak traffic from our own suite. Diagnostics-specific tests opt back in per-file with mocked HTTP, keeping them network-safe.Also tightens
DISABLE_DIAGNOSTICSparsing: explicit falsy values (false/0/no/off) are now a deliberate opt-in that overrides test detection, and whitespace-only values are treated as unset. Thedisable_diagnosticsdocstring documents the new behavior.Counterpart of the TypeScript fix: agentcathq/agentcat-typescript-sdk#44.
Test plan
tests/test_diagnostics_test_env.py::test_track_does_not_enable_diagnostics_in_pytest— before the SDK guard,track()under pytest latched diagnostics on (assert True is False); after the guard it stays disabled andflush_diagnostics()posts nothing.tests/test_diagnostics_optout.py): enabled by default in a simulated non-test env; disabled by default whenPYTEST_CURRENT_TESTset; disabled by default whenPYTEST_VERSIONset;DISABLE_DIAGNOSTICS=falseforce-enables even under pytest; whitespace-only treated as unset; falsy values force-enable.uv run pytest→ 479 passed, 4 skipped, 3 xfailed.DIAGNOSTICS_ENDPOINT=http://127.0.0.1:9 uv run pytest→ 479 passed, 4 skipped, 3 xfailed (no hangs, no leaked traffic).uv run ruff check .— no new findings (new files clean; pre-existing repo findings unchanged).uv run mypy src/mcpcat --strict— error count unchanged from baseline (130), no new errors introduced.