Skip to content

E2E gate degrades silently: clang #if-hollowing, silent local skip, and sibling-checkout drift (verified J2 unload regression vs vector-solver) #140

Description

@petlenz

From the 2026-08-02 full-repo review (finding F-13). Drift verified live.

Three ways the strongest test layer silently weakens:

  1. clang hollowingtests/generated/numsim_material_check_driver.cpp:34 compiles out 11 of 17 e2e tests under clang (#if defined(__GNUC__) && !defined(__clang__)) with no skip report. Per tests/CMakeLists.txt:299 the numsim-core pin now includes the numsim-core#16/Open-set extensibility: user-registered emitters for custom cas nodes #17 history_property fix, so the gate is likely stale. Replace with GTEST_SKIP() (visible in ctest output) or remove after re-testing under clang-19.
  2. Silent local skip — with no local checkout and NUMSIM_CODEGEN_FETCH_MATERIALS=OFF (the local default), the whole Phase-B gate SKIPs with only a configure STATUS line (tests/CMakeLists.txt:386-390). Local green proves much less than CI green, invisibly.
  3. Sibling-checkout drift — local checkouts silently override the CI pins. Verified concretely: against numsim-materials feature/vector-solver (0ea3801, of which the pin aaa6614 is an ancestor), NumSimMaterialEndToEnd.J2PathDependentRetainsPlasticStrainOnUnload FAILS (plastic strain grows 0.0077→0.0120 during elastic unloading), while the pinned SHAs pass 17/17. So (a) local red/green and CI red/green are different claims, and (b) the next pin bump imports a real behavioral regression unless the vector-solver unload semantics are resolved upstream first.

Proposals: GTEST_SKIP() instead of #if; a loud configure warning when a local sibling checkout's git rev-parse HEAD differs from the pin; raise the unload behavior with numsim-materials before any pin bump; optionally a CI step asserting the expected e2e test count so a fetch-flag regression can't silently drop the gate.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions