What
A scalable golden-file test suite that validates every codegen'd CalculiX material against CalculiX's OWN built-in reference materials, run through a real ccx. Generalises the ad-hoc validation currently in examples/calculix/ (linear elasticity only) into a per-material, loop-over-all harness.
This is a follow-up to the CalculiX external .so target (PR #127); it is its own branch/issue by design (the harness needs a real ccx build and is separate from the target itself).
Folder structure
tests/calculix/
run_harness.sh # loops over every material dir, generates + runs + diffs
compare_dat.py # field-by-field .dat diff (moved from examples/calculix)
<material_type>/ # e.g. linear_elastic/, j2_plasticity/, ...
recipe.cpp # program that emits the material on the fly (CalculiXExternalTarget)
test.inp # deck using the @-codegen material
gold/
<case>.dat # COMMITTED reference output from CalculiX's built-in material
gold/*.dat is checked in — produced once from CalculiX's built-in material (e.g. *ELASTIC for linear_elastic, *PLASTIC for J2) on the same geometry/BCs. The generating built-in deck is kept alongside (e.g. gold/gen_gold.inp) so the gold is reproducible.
recipe.cpp (one folder above gold/) is the "generate the material on the fly" program: it builds the recipe and emits <Model>_ext.cpp via CalculiXExternalTarget.
How the harness works
For each tests/calculix/<material>/:
- build + run
recipe.cpp (against numsim::codegen) → emit <Model>_ext.cpp;
g++ -shared -fPIC it → lib<MODEL>.so;
- run
test.inp through a pre-built external-enabled ccx (path via $CCX), LD_LIBRARY_PATH at the .so;
compare_dat.py <material>/gold/<case>.dat <run>.dat — pass if every field matches within tolerance.
The harness loops over all material dirs and reports per-material PASS/FAIL, exit nonzero on any failure. ccx itself is built once (see examples/calculix/build_and_run_external.sh); the harness consumes it via $CCX rather than rebuilding.
Design questions to settle on the branch
- Generator wiring: compile each
recipe.cpp via a tests/calculix/CMakeLists.txt (knows the numsim::codegen/cas/tmech include+lib paths) vs. a standalone build in the shell harness. CMake is cleaner.
- CI: the harness needs a real
ccx, too heavy for default CI. Gate it behind a $CCX env / opt-in ctest (skip when absent), like the existing manual scripts.
- Gold provenance: commit the built-in deck that produced each gold so it's regenerable; document the moduli mapping (e.g.
E,ν ⇔ λ,μ).
First material
linear_elastic/ — migrate the PR #127 decks: test.inp (@-material), gold/ from CalculiX *ELASTIC (E=1.855, ν=0.325 ⇔ λ=1.3, μ=0.7), which already matches our .so bit-for-bit. Then add j2_plasticity/ once the stateful path lands.
How to test
tests/calculix/run_harness.sh returns 0 with all materials PASS against their gold.
- A negative control: perturb a material's constants and confirm the harness FAILS (proves the diff discriminates).
- Adding a new material = a new folder with
recipe.cpp + test.inp + gold/; no harness changes.
What
A scalable golden-file test suite that validates every codegen'd CalculiX material against CalculiX's OWN built-in reference materials, run through a real
ccx. Generalises the ad-hoc validation currently inexamples/calculix/(linear elasticity only) into a per-material, loop-over-all harness.This is a follow-up to the CalculiX external
.sotarget (PR #127); it is its own branch/issue by design (the harness needs a realccxbuild and is separate from the target itself).Folder structure
gold/*.datis checked in — produced once from CalculiX's built-in material (e.g.*ELASTICfor linear_elastic,*PLASTICfor J2) on the same geometry/BCs. The generating built-in deck is kept alongside (e.g.gold/gen_gold.inp) so the gold is reproducible.recipe.cpp(one folder abovegold/) is the "generate the material on the fly" program: it builds the recipe and emits<Model>_ext.cppviaCalculiXExternalTarget.How the harness works
For each
tests/calculix/<material>/:recipe.cpp(againstnumsim::codegen) → emit<Model>_ext.cpp;g++ -shared -fPICit →lib<MODEL>.so;test.inpthrough a pre-built external-enabledccx(path via$CCX),LD_LIBRARY_PATHat the.so;compare_dat.py <material>/gold/<case>.dat <run>.dat— pass if every field matches within tolerance.The harness loops over all material dirs and reports per-material PASS/FAIL, exit nonzero on any failure.
ccxitself is built once (seeexamples/calculix/build_and_run_external.sh); the harness consumes it via$CCXrather than rebuilding.Design questions to settle on the branch
recipe.cppvia atests/calculix/CMakeLists.txt(knows the numsim::codegen/cas/tmech include+lib paths) vs. a standalone build in the shell harness. CMake is cleaner.ccx, too heavy for default CI. Gate it behind a$CCXenv / opt-in ctest (skip when absent), like the existing manual scripts.E,ν ⇔ λ,μ).First material
linear_elastic/— migrate the PR #127 decks:test.inp(@-material),gold/from CalculiX*ELASTIC(E=1.855, ν=0.325 ⇔ λ=1.3, μ=0.7), which already matches our.sobit-for-bit. Then addj2_plasticity/once the stateful path lands.How to test
tests/calculix/run_harness.shreturns 0 with all materials PASS against their gold.recipe.cpp+test.inp+gold/; no harness changes.