Labwright — a verifiable wet-lab design assistant. The LLM proposes raw inputs; deterministic calculators compute every derived number; a verifier re-proves each one (hard gate). A number you type is not trusted.
Install: pip install labwright — now on PyPI (pypi.org/project/labwright). Extras: [agent] (LLM loop), [extract] (offline audit), [ui] (Gradio app).
What's new since v0.2.0 (30 commits)
- LabMath-Bench full 610-entry set, benchmarked end-to-end for
deepseek-v4-flashanddeepseek-v4-pro: usable 93.4 % (flash, hallucination 0.000) and 91.8 % (pro, 0.003) vs bare-LLM 5.1 % / 7.2 %; newTBA(target-before-attempt) metric and fail-safe adversarial section. - Weak-model ablation:
glm-4-flash(free tier) × 610 — Labwright still separates from memory systems ~100× (0.3 % → 30.5 % usable) but degrades: an honest data point on how much the framework depends on NLU quality. - Gate-variant ablation (verifier on vs off, 102-entry subset): the gate guarantees consistency, it does not raise usable rate.
- CLI:
labwright audit "<goal>"— offline fast path via the fine-tuned extractor (fixed to use the multi-block format the current adapter was trained on);labwright ui— the same two-tab Gradio app locally. README now documents the tested walkthrough. - 10-dimension evaluation framework (
evaluation/): falsifiable experiment specs + four free deterministic runners over committed results. - Hugging Face: deploy guide (
hf_space/PUBLISH.md) + refreshed static showcase on seed-pooled numbers. results/reorganized into 12 experiment-family directories with a manifest README.- Determinism:
residence_timebyte-stable across Python versions; benchmark resume-from-checkpoint; GLM tool-serialization fix. - 599 tests pass; clean-venv one-shot reproduction (
reproduce_all.sh, Dockerfile).
Honest boundary
The gate stops fabricated numbers but cannot supply physiology the model does not know: on blind goals usable drops to 44–49 %, and cold recall-only goals stay 3/8 = 38 %. Seed intervals, the scoring definitions, and the extractor register disclosure live in the repo (eval/README.md, evaluation/).