Skip to content

Labwright v0.3.0

Latest

Choose a tag to compare

@qgeng1465 qgeng1465 released this 29 Aug 04:43
· 4 commits to main since this release

Labwright — a verifiable wet-lab design assistant. The LLM proposes raw inputs; deterministic calculators compute every derived number; a verifier re-proves each one (hard gate). A number you type is not trusted.

Install: pip install labwright — now on PyPI (pypi.org/project/labwright). Extras: [agent] (LLM loop), [extract] (offline audit), [ui] (Gradio app).

What's new since v0.2.0 (30 commits)

  • LabMath-Bench full 610-entry set, benchmarked end-to-end for deepseek-v4-flash and deepseek-v4-pro: usable 93.4 % (flash, hallucination 0.000) and 91.8 % (pro, 0.003) vs bare-LLM 5.1 % / 7.2 %; new TBA (target-before-attempt) metric and fail-safe adversarial section.
  • Weak-model ablation: glm-4-flash (free tier) × 610 — Labwright still separates from memory systems ~100× (0.3 % → 30.5 % usable) but degrades: an honest data point on how much the framework depends on NLU quality.
  • Gate-variant ablation (verifier on vs off, 102-entry subset): the gate guarantees consistency, it does not raise usable rate.
  • CLI: labwright audit "<goal>" — offline fast path via the fine-tuned extractor (fixed to use the multi-block format the current adapter was trained on); labwright ui — the same two-tab Gradio app locally. README now documents the tested walkthrough.
  • 10-dimension evaluation framework (evaluation/): falsifiable experiment specs + four free deterministic runners over committed results.
  • Hugging Face: deploy guide (hf_space/PUBLISH.md) + refreshed static showcase on seed-pooled numbers.
  • results/ reorganized into 12 experiment-family directories with a manifest README.
  • Determinism: residence_time byte-stable across Python versions; benchmark resume-from-checkpoint; GLM tool-serialization fix.
  • 599 tests pass; clean-venv one-shot reproduction (reproduce_all.sh, Dockerfile).

Honest boundary

The gate stops fabricated numbers but cannot supply physiology the model does not know: on blind goals usable drops to 44–49 %, and cold recall-only goals stay 3/8 = 38 %. Seed intervals, the scoring definitions, and the extractor register disclosure live in the repo (eval/README.md, evaluation/).