Skip to content

Labwright v0.2.0

Choose a tag to compare

View all tags
@qgeng1465 qgeng1465 released this 16 Aug 15:59
· 35 commits to main since this release
v0.2.0
eb4fa6e

Labwright: a verifiable wet-lab AI copilot. The LLM proposes raw inputs; deterministic calculators compute every derived number; a verifier re-proves each one (hard gate). A number you type is not trusted.

What's in v0.2.0

  • Hard-gate architecture: goal → LLM proposes raw inputs → deterministic calculators (microfluidics + plate cell culture + spheroid + PK + dosing + stats) → four-layer verifier (arithmetic / units & sanity / safety / prose) → SOP + design JSON with provenance. Naive alternatives (type-the-numbers, soft self-check, LLM-as-verifier) rejected.
  • Benchmark, 6 sets × 96 source-pinned goals: reading 24, blind 15, culture 14, spheroid 15, PK 14, new-domains 14. Labwright 88–100% usable, 0.000 hallucination on most sets vs bare LLM 0–12% usable, ~1.0 hallucination.
  • Cold-blind honesty: a 2026-08-16 expansion grows the cold set to 12 goals (7/12 = 58% both models), with 4 calculator-derived scaling goals disclosed separately: recall-only 8 stay 3/8 = 38%.
  • Seed-level stability: reading 5-seed (flash 92.5% [0.864, 0.960] / pro 95.8% [0.906, 0.982]), new-domains 5-seed (flash 98.6% [0.923, 0.997], the one set where flash beats pro by a wide margin), Wilson 95% CIs in the README.
  • Scoring section in the README: self-consistent / usable / hallucination, ±5% tolerance, no-submit = 1.0, unit-misread and target-selection failure classes.
  • Fast-path extractor (fine-tuned Qwen2.5-1.5B, v6): reading 96% usable, fully regenerable from committed training data.
  • Reverse-verification: the same calculators check published protocols and catch unit mix-ups and order-of-magnitude errors.
  • CLI: labwright design, labwright verify-protocol, labwright audit, labwright tools, labwright benchmark.
  • 516 passing tests; Python API + Gradio demo; README EN/ZH.

Honest boundary

On blind goals the gate cannot supply physiology the model does not know: usable drops to 40–47%, hallucination stays 0.000, and every number in the README is recomputable from committed results/*.json. Full docs, figures, and the graphical abstract in the README.

Assets 2
Loading