Skip to content

refactor(tests): more test stability fixes and improvements#1035

Merged
solnic merged 7 commits intomasterfrom
refactor/more-test-stability-fixes
Apr 16, 2026
Merged

refactor(tests): more test stability fixes and improvements#1035
solnic merged 7 commits intomasterfrom
refactor/more-test-stability-fixes

Conversation

@solnic
Copy link
Copy Markdown
Collaborator

@solnic solnic commented Apr 14, 2026

I had to make more tweaks for simplicity and better stability. There are cases where it simply must not be allowed to use async: true as tests are relying on changing configuration of globals that we don't own and control.

It's worth mentioning that our end-goal here should be to introduce a proper client/transport split like we have in other SDKs. This would be a cleaner architecture with easier testability being a nice side-effect. Leaving this for 14.0.

This PR can be reviewed commit-by-commit.

Summary

  • Rework Sentry.Test isolation around an explicit Scope abstraction. Replace the flat :persistent_term layout (one key per {pid, config_key} plus separate scope-marker, allow-map, and counter keys) with a Sentry.Test.Scope value struct (one entry per test) and a Sentry.Test.Scope.Registry that owns lifecycle and resolution. Sentry.Test.Config becomes a thin adapter — namespace/1, put/1, allow/2 keep their signatures.
  • Drop the single-active-scope fallback. Master's resolve_from_active_scopes/1 routed any unclaimed pid to whichever test happened to be active, which caused stray events (e.g. OTP's async callback-crashed meta-event) to leak across tests. The new resolution tries three precise strategies and stops: walk [pid | $callers], walk $ancestors against each scope's owner_pid, walk [pid | $ancestors] against each scope's allowed_pids. Unmatched pids fall through to :default.
  • Transparent routing for globally-supervised pids. Sentry.Test.Config.put/1 auto-soft-allows :logger, :logger_sup, and Sentry.Supervisor onto the calling scope so downstream suites get per-test routing with zero opt-in. Soft-allow silently skips when a live peer scope already owns a global (async-safe); explicit allow/2 keeps strict conflict detection via Scope.AllowConflictError.
  • O(1) resolution via a reverse-index :persistent_term key. Lookups no longer iterate active scopes; lookup_allow_owner/1 and fetch/1 are direct gets. This keeps Sentry config reads cheap under async load.
  • Ancestor-lookup improvements. Warn on ambiguous overrides, dedupe collect_ancestors/3 across siblings, tighten ancestors_of/1, gate the OTP 25+ parent-pid probe behind a tag.
  • Longer default receive_timeout in Sentry.Case so async suites don't flake under load.
  • :enable_logs handler-config hardening. Reject non-boolean overrides at attach time; add positive-path coverage for handler-level enable_logs: false overriding a global enable_logs: true.
  • DSN handling in Sentry.Test.Registry. Make the Bypass DSN override unconditional so a leaked SENTRY_DSN can't ship synthetic events to a real endpoint; warn when overriding; harden with a catch-all and regression tests.

@solnic solnic force-pushed the refactor/more-test-stability-fixes branch from f1defe3 to 0e1c0b9 Compare April 14, 2026 10:49
@solnic solnic changed the title Refactor/more test stability fixes refactor(tests): more test stability fixes and improvements Apr 14, 2026
@solnic solnic force-pushed the refactor/more-test-stability-fixes branch 4 times, most recently from f21c4fb to 931dd1c Compare April 15, 2026 14:40
@solnic solnic marked this pull request as ready for review April 15, 2026 14:40
Comment thread lib/sentry/test/config.ex Outdated
Comment thread lib/sentry/test/config.ex Outdated
Comment thread lib/sentry/application.ex Outdated
@solnic solnic force-pushed the refactor/more-test-stability-fixes branch 2 times, most recently from 18f0729 to 2f6bc7e Compare April 16, 2026 07:38
Comment thread mix.exs Outdated
@solnic solnic force-pushed the refactor/more-test-stability-fixes branch from 7b1481c to 712c7d0 Compare April 16, 2026 08:10
Comment thread lib/sentry/config.ex
Comment thread test/sentry/test/config_isolation_test.exs
@solnic solnic force-pushed the refactor/more-test-stability-fixes branch from 712c7d0 to dd59cec Compare April 16, 2026 08:32
Comment thread lib/sentry/test/config.ex Outdated
@solnic solnic force-pushed the refactor/more-test-stability-fixes branch from dd59cec to aef3eb5 Compare April 16, 2026 08:43
solnic and others added 2 commits April 16, 2026 10:21
Make the Bypass DSN override in Sentry.Test.Registry unconditional so
an externally-configured DSN (e.g. leaking from SENTRY_DSN in the
environment) can never cause the test suite to ship synthetic events
to a real Sentry endpoint. Emit a Logger.warning when an override
replaces a previously-configured DSN so the developer sees exactly
what is being replaced and why.

Add a catch-all clause to maybe_warn_about_dsn_override/1 so an
unexpected DSN shape cannot crash registry init and take down the
test harness. Expose the function (@doc false) so the warning path
can be covered directly, and add a regression test asserting both
the warning-emitted and silent cases. Document the unconditional
DSN override in the CHANGELOG since test_mode: true suites that
intentionally point at a local Sentry/mock collector are affected.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@solnic solnic force-pushed the refactor/more-test-stability-fixes branch from aef3eb5 to acfbe2d Compare April 16, 2026 11:04
Comment thread lib/sentry/test/scope/registry.ex
Comment thread lib/sentry/config.ex Outdated
Comment thread lib/sentry/config.ex Outdated
Comment thread lib/sentry/test/config.ex
Comment thread lib/sentry/logger_handler.ex
cursor[bot]

This comment was marked as resolved.

@solnic solnic force-pushed the refactor/more-test-stability-fixes branch from 0a2d87a to 0a5eee2 Compare April 16, 2026 11:35
Copy link
Copy Markdown

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 99019f2. Configure here.

Comment thread lib/sentry/test/scope/registry.ex
solnic and others added 2 commits April 16, 2026 11:55
Tighten the ancestor walk used by Sentry.Test.Config so per-test
overrides resolve predictably under async tests and edge OTP
scenarios.

find_via_ancestors/2 previously returned :default both when no
ancestor had set a key and when multiple ancestors disagreed on one.
The latter is a load-bearing safety property of per-test isolation,
but a silent fallback to global config produced passing-but-wrong
assertions with no hint that ambiguity was detected. Distinguish
zero from ambiguous and emit a Logger.warning on the ambiguous
branch.

collect_ancestors/3 threaded the `seen` set on the outer call, but
each recursive branch then walked independently, allowing siblings
to re-enter each other's subtrees. Thread `seen` through the reduce
so each ancestor is visited at most once across the entire walk, and
build the collected list with Enum.reverse/++ for idiomatic list
construction.

ancestors_of/1 matched a wildcard on Process.info/2 that swallowed
any unexpected return shape. Match nil explicitly so future OTP
changes surface instead of silently regressing, and gate the
Process.info(pid, :parent) path behind an :otp_25_plus tag since
that return is only guaranteed on OTP 25+ (the project declares
elixir ~> 1.13 which is compatible back to OTP 22).

Add regression coverage: conflicting overrides on two ancestors of a
manually-spawned descendant, and an orphan-process leak test for a
pid whose chain has neither \$callers nor \$ancestors linking back
to the test — the exact scenario that motivated dropping the old
resolve_from_active_scopes/1 path.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
solnic and others added 3 commits April 16, 2026 11:55
Raise the shared receive_timeout used by Sentry.Case so async test
suites don't flake when an envelope takes longer than the old
default to reach the collecting test process under load.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The handler-level :enable_logs override added alongside the
Sentry.Test config-isolation rework previously accepted any value and
silently coerced non-booleans to the global default. Reject non-boolean
overrides with a clear error so misconfigured handler configs surface
at attach time, not as missing logs in production.

Add positive-path coverage for the inverse case the rejection test
doesn't cover: global config has enable_logs: true and the handler-level
:enable_logs: false override wins. That's the regression most likely to
resurface silently if precedence between the handler override and the
global Config.enable_logs?() is reordered.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@solnic solnic force-pushed the refactor/more-test-stability-fixes branch from 99019f2 to fd7dd65 Compare April 16, 2026 11:55
@solnic solnic merged commit 2642d93 into master Apr 16, 2026
11 checks passed
@solnic solnic deleted the refactor/more-test-stability-fixes branch April 16, 2026 15:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants