Repository navigation
Releases: DavidWallstructurallaw/structdet-bench
Release list
StructDet-Bench v0.1.0
StructDet-Bench v0.1.0 measures which solution structures appear, how concentrated they are, and how they change across recorded LLM outputs. Its current reference workflow uses sorting mechanisms. Users supply structural assignments and supporting evidence; unresolved cases remain visible.
M: observed support, structural concentration (SCI) and related diversity measures.
V: registered comparisons under compatible frames, sensitivity and retained sampling identities.
L: trajectories, finite-family support assays, local structural half-life, conditional recovery and revision invalidation.
Study workflow: passive evidence inspection, blind review exports, correction tracking, compatible analysis compilation and readiness disclosures.
Replay checks input, method, software, registered order and stored draw identities. Missing or corrupt requested replay information fails explicitly. Existing output destinations are refused. Model calls and candidate execution occur outside the toolkit.
Use Linux with CPython 3.13.5 and Unicode 15.1.0. There are no third-party runtime dependencies. Download both structdet_bench-0.1.0-py3-none-any.whl and structdet-bench-v0.1.0-source.zip from the assets below into a new working directory. The wheel installs the package; the complete source ZIP supplies examples and documentation. Open a terminal there and run:
python3.13 -m venv .venv
.venv/bin/python -m pip install --no-index --no-deps ./structdet_bench-0.1.0-py3-none-any.whl
unzip structdet-bench-v0.1.0-source.zip
.venv/bin/python -m structdet_bench validate --bundle structdet-bench/examples/hero_hf00/bundle.json
.venv/bin/python -m structdet_bench analyze --bundle structdet-bench/examples/hero_hf00/bundle.json --output-dir hf00-outputOpen hf00-output/report.md; structured results and run identity are in report.json and run_manifest.json. HF-00 stipulates twenty observations: support is 5, SCI is 0.285, and support at the inclusive 0.10 threshold is 4. This is fixture arithmetic, with zero model calls.
The README explains input preparation, source execution, complete study examples and report interpretation. The attached RELEASE_CHECKS.md identifies the qualification results and tested artifacts; assets include source, wheel, sdist and SHA-256 checksums.
A separate limited trial covered five screenshot-derived programs and 44,975 candidate-input executions. Proposed mechanism labels remain provisional, and formal SCI was undefined for the empty admitted classification population. Full registered comparisons and independent scientific validation remain pending. Software verification and fixture results do not establish universal model capacity or theory validation.
Software uses Apache-2.0; project documentation uses CC BY 4.0.