Skip to content

fix(config): isolate global state in loader tests to fix order-dependent failures#15

Merged
jkyberneees merged 1 commit into
mainfrom
fix/config-test-isolation
Jun 6, 2026
Merged

fix(config): isolate global state in loader tests to fix order-dependent failures#15
jkyberneees merged 1 commit into
mainfrom
fix/config-test-isolation

Conversation

@jkyberneees
Copy link
Copy Markdown
Contributor

Summary

go test ./internal/config/... -count=1 failed, but the same tests passed in isolation. This was a pre-existing test-isolation defect on main (reproduced via a clean worktree of main), unrelated to any recent config change.

go test ./internal/config/... -count=1                                   # FAILED
go test ./internal/config/... -run TestLoadConfig_InvalidJSON -count=1    # PASSED

Example failure: loader_test.go: Model = "deepseek-v4-flash", want empty — a value that only appears as the TestLoadConfig_EnvVars fixture.

Root cause

Two compounding sources of leaked process-global state:

  1. Test-side leaks — tests mutated os.Setenv("ODEK_*"/"HOME", …) and os.Chdir(…) with hand-written defer cleanup that is ordering-fragile.
  2. Loader-side leak (the real culprit)LoadConfigloadSecretsEnv() reads the developer's real ~/.odek/secrets.env and os.Setenvs its contents (e.g. ODEK_MODEL=deepseek-v4-flash) into the process for any test that did not isolate $HOME. Those loader-side os.Setenv calls are not covered by any test defer, so they leaked into later tests, where the documented env > file precedence overrode file/default config and broke assertions.

Fix (test-only — loader.go behavior unchanged)

  • Replace os.Setenv + defer os.Unsetenv/os.Setenv with t.Setenv (auto-restored, leak-proof, panics under t.Parallel).
  • Replace os.Chdir + defer os.Chdir(cwd) with t.Chdir.
  • Point $HOME at an isolated t.TempDir() in every test that calls LoadConfig, so loadSecretsEnv never reads the real secrets.env and never injects un-restorable env vars.

Net: 1 file changed, 59 insertions(+), 110 deletions(-).

Verification

Gate Result
gofmt -l internal/config/loader_test.go empty ✅
go vet ./internal/config/... clean ✅
go test ./internal/config/... -count=1 PASS ✅
go test ./internal/config/... -count=1 -shuffle=on PASS ✅
-shuffle= seeds 1 / 42 / 99 / 7777 PASS ✅
go build ./... ok ✅

🤖 Generated with Claude Code

…ent failures

The internal/config tests failed under a full-package run but passed in
isolation. Tests mutated process-global state — os.Setenv("ODEK_*"/"HOME"),
os.Chdir — with hand-written defer cleanup. The deeper leak: LoadConfig calls
loadSecretsEnv(), which reads the real ~/.odek/secrets.env and os.Setenv's its
contents (e.g. ODEK_MODEL) into the process for any test that did not isolate
$HOME. Those loader-side os.Setenv calls are not restored by test defers and
leaked into later tests, where the env > file precedence overrode file/default
config and broke assertions (e.g. Model = "deepseek-v4-flash").

Test-only fix (loader.go behavior unchanged):
- Replace os.Setenv + defer os.Unsetenv/os.Setenv with t.Setenv (auto-restored,
  leak-proof, panics under t.Parallel).
- Replace os.Chdir + defer os.Chdir(cwd) with t.Chdir.
- Point $HOME at an isolated t.TempDir() in every LoadConfig test so
  loadSecretsEnv never reads the developer's real secrets.env.

Verified: gofmt clean, go vet clean, go test ./internal/config/... -count=1
passes, and passes under -shuffle=on across seeds 1/42/99/7777.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@jkyberneees
Copy link
Copy Markdown
Contributor Author

Verification Certificate

Protocol: AI Verification Protocol v5.2.7 (single-reviewer application — no multi-agent pipeline; diversity_ok: false, certificate is advisory, not auto-approving)
PR: #15 — fix(config): isolate global state in loader tests
SHA: dc1c0d439f2f4b6eb9bfe3871f8b35e4c657ec50
Classification: KnownGroundTruth (test-isolation fix; oracle = the test suite itself, run under shuffled order)
LOC_filtered: ~169 (test-only; 0 production LOC)

Axes Summary

Axis Status Key finding
2.1 Semantic Correctness t.Setenv/t.Chdir semantically equal prior manual save-restore, but leak-proof and ordering-independent. Behavior of loader.go untouched.
2.2 Behavioral Contract Contract = "config tests pass deterministically in any order". Verified by -shuffle=on across 7 seeds + -race.
2.3 Security Surface Strictly reduces secret exposure: tests no longer read the developer's real ~/.odek/secrets.env (/Users/kyberneees isolated to a tempdir).
2.4 Structural Integrity Net −51 LOC; removed fragile hand-written defer cleanup.
2.5 Behavioral Exploration Completeness check: all 30 LoadConfig call sites confirmed to isolate /Users/kyberneees (programmatic audit). No t.Parallel present, so t.Setenv cannot panic.
2.6 Dependency Integrity t.Chdir requires Go 1.24+ (task brief said 1.20 — incorrect). Verified safe: go.mod requires 1.25.0, CI pins 1.24. No new deps.
2.7 Generator Provenance Authored by Claude Opus 4.8 via Claude Code; recorded in commit trailer.
2.8 Adversarial Surface No untrusted-input sinks; no eval/shell/SQL paths.
2.9 Documentation Coverage No public API surface changed (test-only). d = 1.

η (informal, single-reviewer)

Oracle is strong and independent of the change: the same production loader.go is exercised, and the fix is proven by deterministic pass under -shuffle=on (seeds 1/3/42/99/314/271828/7777) and -race. η ≈ high; ρ ≈ 0 (no generated tautological oracle — assertions are unchanged from main).

Verification Debt

  • ΔDebt: ~0h. The change pays down existing verification debt (flaky-by-order suite → deterministic).

Verdict

HumanReviewRecommended — mechanically clean and fully reproduced, but capped below AutoApprove because this is a single-reviewer certificate, not the protocol's required multi-provider pipeline. Recommended for merge.

Rationale: Root cause (loader-side os.Setenv from real secrets.env leaking into order-dependent tests) identified and eliminated test-side; loader.go behavior unchanged; all protocol verification gates green.

🤖 Generated with Claude Code

@jkyberneees jkyberneees merged commit ad0be26 into main Jun 6, 2026
6 checks passed
@jkyberneees jkyberneees deleted the fix/config-test-isolation branch June 6, 2026 11:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant