Labwright v0.3.0

Labwright — a verifiable wet-lab design assistant. The LLM proposes raw inputs; deterministic calculators compute every derived number; a verifier re-proves each one (hard gate). A number you type is not trusted.

Install: pip install labwright — now on PyPI (pypi.org/project/labwright). Extras: [agent] (LLM loop), [extract] (offline audit), [ui] (Gradio app).

What's new since v0.2.0 (30 commits)

• LabMath-Bench full 610-entry set, benchmarked end-to-end for deepseek-v4-flash and deepseek-v4-pro: usable 93.4 % (flash, hallucination 0.000) and 91.8 % (pro, 0.003) vs bare-LLM 5.1 % / 7.2 %; new TBA (target-before-attempt) metric and fail-safe adversarial section.
• Weak-model ablation: glm-4-flash (free tier) × 610 — Labwright still separates from memory systems ~100× (0.3 % → 30.5 % usable) but degrades: an honest data point on how much the framework depends on NLU quality.
• Gate-variant ablation (verifier on vs off, 102-entry subset): the gate guarantees consistency, it does not raise usable rate.
• CLI: labwright audit "<goal>" — offline fast path via the fine-tuned extractor (fixed to use the multi-block format the current adapter was trained on); labwright ui — the same two-tab Gradio app locally. README now documents the tested walkthrough.
• 10-dimension evaluation framework (evaluation/): falsifiable experiment specs + four free deterministic runners over committed results.
• Hugging Face: deploy guide (hf_space/PUBLISH.md) + refreshed static showcase on seed-pooled numbers.
• results/ reorganized into 12 experiment-family directories with a manifest README.
• Determinism: residence_time byte-stable across Python versions; benchmark resume-from-checkpoint; GLM tool-serialization fix.
• 599 tests pass; clean-venv one-shot reproduction (reproduce_all.sh, Dockerfile).

Honest boundary

The gate stops fabricated numbers but cannot supply physiology the model does not know: on blind goals usable drops to 44–49 %, and cold recall-only goals stay 3/8 = 38 %. Seed intervals, the scoring definitions, and the extractor register disclosure live in the repo (eval/README.md, evaluation/).

Labwright v0.2.0

Labwright: a verifiable wet-lab AI copilot. The LLM proposes raw inputs; deterministic calculators compute every derived number; a verifier re-proves each one (hard gate). A number you type is not trusted.

What's in v0.2.0

• Hard-gate architecture: goal → LLM proposes raw inputs → deterministic calculators (microfluidics + plate cell culture + spheroid + PK + dosing + stats) → four-layer verifier (arithmetic / units & sanity / safety / prose) → SOP + design JSON with provenance. Naive alternatives (type-the-numbers, soft self-check, LLM-as-verifier) rejected.
• Benchmark, 6 sets × 96 source-pinned goals: reading 24, blind 15, culture 14, spheroid 15, PK 14, new-domains 14. Labwright 88–100% usable, 0.000 hallucination on most sets vs bare LLM 0–12% usable, ~1.0 hallucination.
• Cold-blind honesty: a 2026-08-16 expansion grows the cold set to 12 goals (7/12 = 58% both models), with 4 calculator-derived scaling goals disclosed separately: recall-only 8 stay 3/8 = 38%.
• Seed-level stability: reading 5-seed (flash 92.5% [0.864, 0.960] / pro 95.8% [0.906, 0.982]), new-domains 5-seed (flash 98.6% [0.923, 0.997], the one set where flash beats pro by a wide margin), Wilson 95% CIs in the README.
• Scoring section in the README: self-consistent / usable / hallucination, ±5% tolerance, no-submit = 1.0, unit-misread and target-selection failure classes.
• Fast-path extractor (fine-tuned Qwen2.5-1.5B, v6): reading 96% usable, fully regenerable from committed training data.
• Reverse-verification: the same calculators check published protocols and catch unit mix-ups and order-of-magnitude errors.
• CLI: labwright design, labwright verify-protocol, labwright audit, labwright tools, labwright benchmark.
• 516 passing tests; Python API + Gradio demo; README EN/ZH.

Honest boundary

On blind goals the gate cannot supply physiology the model does not know: usable drops to 40–47%, hallucination stays 0.000, and every number in the README is recomputable from committed results/*.json. Full docs, figures, and the graphical abstract in the README.

Labwright v0.1.0

Labwright — a verifiable wet-lab AI copilot. The LLM proposes raw inputs; deterministic calculators compute every derived number; a verifier re-proves each one (hard gate). A number you type is not trusted.

What's in v0.1.0

• Hard-gate architecture: goal → LLM proposes raw inputs → deterministic calculators (microfluidics + plate cell culture + dosing) → verifier re-proves every derived number → SOP + design JSON. Naive alternatives (type-the-numbers, soft self-check) rejected.
• Benchmark: on 24 reading goals — Labwright 88–100 % usable designs, 0.000 hallucination vs bare LLM 0–12 % usable, ~1.0 hallucination.
• Reverse-verification: same calculators check published protocols (kidney/lung) and catch unit mix-ups, order-of-magnitude errors.
• Extractor: fine-tuned Qwen2.5-1.5B goal → raw-inputs model (json_parse_rate 1.0, consistency 0.998) + deterministic eval.
• CLI: labwright audit "…", labwright tools, labwright benchmark.
• Python API + Gradio space demo, CI (165 tests), issue/PR templates.

Honest boundary

On blind goals the gate can't supply physiology the model doesn't know — usable drops, hallucination stays 0.000. The reported numbers come from the calculators' real physics/biology constants, each pinned to a citable source.

Full docs, figures, and the graphical abstract in the README.