Skip to content

test(design-import): isolate network-asset and knobclean cases from shared state#6382

Merged
shipyard-local[bot] merged 1 commit into
mainfrom
harden/asset-flake
Jul 19, 2026
Merged

test(design-import): isolate network-asset and knobclean cases from shared state#6382
shipyard-local[bot] merged 1 commit into
mainfrom
harden/asset-flake

Conversation

@shipyard-local

@shipyard-local shipyard-local Bot commented Jul 19, 2026

Copy link
Copy Markdown
Contributor

Harden the two design-import test-isolation weaknesses behind the rare
full-run flake where "DesignIR asset manifest reports missing fetcher
and empty network downloads" misses its asset-empty diagnostic
(first seen on the Shipyard mac lane 2026-05-21, mitigated then by
timeout bumps + the design-import-network ctest resource lock):

  • Unique per-invocation asset URLs in the missing-fetcher/empty-download
    case. The on-disk asset cache is keyed by absolute URL, so fixed URLs
    rely on every test and every concurrent process keeping a private
    cache_directory to avoid a cache hit that would skip the download and
    its diagnostic. Deriving the URL path from the unique TempDir name
    makes that collision structurally impossible.
  • Align network_timeout_ms with the other two network smoke cases
    (30s): the fetch path reclassifies a slow child spawn as
    asset-fetch-timeout, replacing the asserted diagnostic under co-load.
    The fake curl never sleeps, so the wide timeout adds no wall time.
  • Capture the diagnostic codes actually present (new diagnostic_codes
    helper) so a future flake reports which diagnostic replaced the
    expected one instead of just "missed asset-empty".
  • Give the knobclean durable-cache case a unique per-invocation home
    directory: the fixed cwd/knobclean-home name is shared state between
    concurrent runs of the binary — one process's remove_all races
    another's fs::exists(emitted) check (reproduced under 8-way
    concurrent full-suite runs; 48/48 green after the fix).

No production code changes; observable download/cache behavior for
real imports is untouched. Full suite green in declaration order and
across 10 randomized seeds with identical counts (2556 assertions in
367 test cases).

Hotspot-Grow: test/test_design_import_codegen.cpp reason="isolation comments + diagnostics capture on the flake-prone network case"
Hotspot-Grow: test/test_design_import_ir.cpp reason="unique per-invocation knobclean home dir + rationale comment"
Co-Authored-By: Claude Opus 4.8 (1M context) noreply@anthropic.com
Claude-Session: https://claude.ai/code/session_014e8kbMn8Sf4mF8ozsy1Jot


🔎 Provenance

Agent claude
Machine m5
Workspace w1
Directory ~/Code/fig-fix
Session fd0e374f-d5dc-4e2a-bb33-6fa8b66124a0

Resume

claude --resume fd0e374f-d5dc-4e2a-bb33-6fa8b66124a0

Jump to this tab

cmux surface focus B1B434D3-85F6-4937-8806-96D59886F574

Relaunch (any agent)

cmux surface resume get --surface B1B434D3-85F6-4937-8806-96D59886F574

Restore URLhttps://claude.ai/code/session_014e8kbMn8Sf4mF8ozsy1Jot

stamped 2026-07-19 17:36 UTC


Summary by cubic

Hardened design-import tests to eliminate a rare CI flake by isolating shared state, aligning network timeouts, and improving diagnostics. No production code changes.

  • Bug Fixes
    • Use unique per-run asset URLs to prevent cache hits across tests/processes.
    • Set network_timeout_ms to 30s to match other network smoke tests and avoid misclassified timeouts.
    • Add diagnostic_codes helper to surface actual diagnostics when assertions fail.
    • Give the knobclean durable-cache test a unique per-run home directory to avoid concurrent fs races.

Written for commit 4dc54d1. Summary will update on new commits.

Review in cubic

…hared state

Harden the two design-import test-isolation weaknesses behind the rare
full-run flake where "DesignIR asset manifest reports missing fetcher
and empty network downloads" misses its asset-empty diagnostic
(first seen on the Shipyard mac lane 2026-05-21, mitigated then by
timeout bumps + the design-import-network ctest resource lock):

- Unique per-invocation asset URLs in the missing-fetcher/empty-download
  case. The on-disk asset cache is keyed by absolute URL, so fixed URLs
  rely on every test and every concurrent process keeping a private
  cache_directory to avoid a cache hit that would skip the download and
  its diagnostic. Deriving the URL path from the unique TempDir name
  makes that collision structurally impossible.
- Align network_timeout_ms with the other two network smoke cases
  (30s): the fetch path reclassifies a slow child spawn as
  asset-fetch-timeout, replacing the asserted diagnostic under co-load.
  The fake curl never sleeps, so the wide timeout adds no wall time.
- Capture the diagnostic codes actually present (new diagnostic_codes
  helper) so a future flake reports which diagnostic replaced the
  expected one instead of just "missed asset-empty".
- Give the knobclean durable-cache case a unique per-invocation home
  directory: the fixed cwd/knobclean-home name is shared state between
  concurrent runs of the binary — one process's remove_all races
  another's fs::exists(emitted) check (reproduced under 8-way
  concurrent full-suite runs; 48/48 green after the fix).

No production code changes; observable download/cache behavior for
real imports is untouched. Full suite green in declaration order and
across 10 randomized seeds with identical counts (2556 assertions in
367 test cases).

Hotspot-Grow: test/test_design_import_codegen.cpp reason="isolation comments + diagnostics capture on the flake-prone network case"
Hotspot-Grow: test/test_design_import_ir.cpp reason="unique per-invocation knobclean home dir + rationale comment"
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014e8kbMn8Sf4mF8ozsy1Jot
@shipyard-local
shipyard-local Bot merged commit 52024f8 into main Jul 19, 2026
18 of 23 checks passed
@danielraffel
danielraffel deleted the harden/asset-flake branch July 19, 2026 17:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant