test(design-import): isolate network-asset and knobclean cases from shared state#6382
Merged
Conversation
…hared state Harden the two design-import test-isolation weaknesses behind the rare full-run flake where "DesignIR asset manifest reports missing fetcher and empty network downloads" misses its asset-empty diagnostic (first seen on the Shipyard mac lane 2026-05-21, mitigated then by timeout bumps + the design-import-network ctest resource lock): - Unique per-invocation asset URLs in the missing-fetcher/empty-download case. The on-disk asset cache is keyed by absolute URL, so fixed URLs rely on every test and every concurrent process keeping a private cache_directory to avoid a cache hit that would skip the download and its diagnostic. Deriving the URL path from the unique TempDir name makes that collision structurally impossible. - Align network_timeout_ms with the other two network smoke cases (30s): the fetch path reclassifies a slow child spawn as asset-fetch-timeout, replacing the asserted diagnostic under co-load. The fake curl never sleeps, so the wide timeout adds no wall time. - Capture the diagnostic codes actually present (new diagnostic_codes helper) so a future flake reports which diagnostic replaced the expected one instead of just "missed asset-empty". - Give the knobclean durable-cache case a unique per-invocation home directory: the fixed cwd/knobclean-home name is shared state between concurrent runs of the binary — one process's remove_all races another's fs::exists(emitted) check (reproduced under 8-way concurrent full-suite runs; 48/48 green after the fix). No production code changes; observable download/cache behavior for real imports is untouched. Full suite green in declaration order and across 10 randomized seeds with identical counts (2556 assertions in 367 test cases). Hotspot-Grow: test/test_design_import_codegen.cpp reason="isolation comments + diagnostics capture on the flake-prone network case" Hotspot-Grow: test/test_design_import_ir.cpp reason="unique per-invocation knobclean home dir + rationale comment" Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014e8kbMn8Sf4mF8ozsy1Jot
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Harden the two design-import test-isolation weaknesses behind the rare
full-run flake where "DesignIR asset manifest reports missing fetcher
and empty network downloads" misses its asset-empty diagnostic
(first seen on the Shipyard mac lane 2026-05-21, mitigated then by
timeout bumps + the design-import-network ctest resource lock):
case. The on-disk asset cache is keyed by absolute URL, so fixed URLs
rely on every test and every concurrent process keeping a private
cache_directory to avoid a cache hit that would skip the download and
its diagnostic. Deriving the URL path from the unique TempDir name
makes that collision structurally impossible.
(30s): the fetch path reclassifies a slow child spawn as
asset-fetch-timeout, replacing the asserted diagnostic under co-load.
The fake curl never sleeps, so the wide timeout adds no wall time.
helper) so a future flake reports which diagnostic replaced the
expected one instead of just "missed asset-empty".
directory: the fixed cwd/knobclean-home name is shared state between
concurrent runs of the binary — one process's remove_all races
another's fs::exists(emitted) check (reproduced under 8-way
concurrent full-suite runs; 48/48 green after the fix).
No production code changes; observable download/cache behavior for
real imports is untouched. Full suite green in declaration order and
across 10 randomized seeds with identical counts (2556 assertions in
367 test cases).
Hotspot-Grow: test/test_design_import_codegen.cpp reason="isolation comments + diagnostics capture on the flake-prone network case"
Hotspot-Grow: test/test_design_import_ir.cpp reason="unique per-invocation knobclean home dir + rationale comment"
Co-Authored-By: Claude Opus 4.8 (1M context) noreply@anthropic.com
Claude-Session: https://claude.ai/code/session_014e8kbMn8Sf4mF8ozsy1Jot
🔎 Provenance
claudem5w1~/Code/fig-fixfd0e374f-d5dc-4e2a-bb33-6fa8b66124a0Resume
Jump to this tab
Relaunch (any agent)
Restore URL — https://claude.ai/code/session_014e8kbMn8Sf4mF8ozsy1Jot
stamped 2026-07-19 17:36 UTC
Summary by cubic
Hardened design-import tests to eliminate a rare CI flake by isolating shared state, aligning network timeouts, and improving diagnostics. No production code changes.
network_timeout_msto 30s to match other network smoke tests and avoid misclassified timeouts.diagnostic_codeshelper to surface actual diagnostics when assertions fail.Written for commit 4dc54d1. Summary will update on new commits.