Lab Book

A coding agent is a program that runs a language model in your terminal and lets it read files, write files, and run commands. This package turns one into a research lab. The model in your session becomes the Director: it writes tasks for other model sessions, sends them off, checks what they bring back, and keeps a notebook you can trust. You are the Investigator: you set the questions and decide what matters. Nothing in the lab counts as a fact until it is a claim with a status and evidence on record.

What you actually do

• Say open the lab — or join the lab at <address> for a lab a colleague started. The first time, you answer a few questions about the problem and about how you want to be talked to, and confirm what the Director found on your machine, in the literature, and in public data. Every time after, the Director tells you what changed since you were last there and proposes the next move.
• Read STATUS.md for a problem: one page, plain words, what the lab currently believes and what it has ruled out.
• Say what matters: which question next, what counts as evidence, when to stop.

That is the whole job. The Director runs the scripts; you never have to.

Your first hour

1. Get a coding agent. Any tool in which a language model can open a folder, read and write files there, and run commands will do. For instance: Claude Code (in a terminal, as a desktop app, or on the web at claude.ai/code), the Codex CLI, ChatGPT's work mode with a connected folder, Cursor, Copilot, Gemini CLI — and others; the list grows. Follow the tool's own install instructions. What it needs: your lab folder, python3, and git. Make an empty folder for the lab and start the agent in it.

2. Install this skill. Inside the agent, say:

   install the skill from mattrobball/lab-book
   

   or, from a shell, npx skills add mattrobball/lab-book. (By hand: copy skills/open-lab/ into your agent's skill folder — see the table at the end.)

3. Say "open the lab". The Director asks its questions one at a time — what the problem is, what would convince you a result is true, what is already known, what the limits are, and how you want to be talked to. Answer in plain sentences. It then looks for the tools on your machine, the papers, and the public data on your exact question, and shows you each list to confirm or strike. Then it sets up the folder, puts you on your own line of the record (even alone — a colleague can join later without anything changing), and proposes the first task.

4. Watch the first task go round. The Director writes a brief (the task, in full), sends a worker to do it, and when the worker returns the result is checked by machine, filed, and summarised for you in a few sentences. Anything the worker claims appears as a numbered claim (C-ann-001, C-ann-002, … — the middle part is your name) marked proposed, not yet believed. A second worker, on a different model, is sent to attack it before it is believed. The worker's whole transcript is kept with the run, so anyone can later see how the result was reached.

Joining a lab that already exists

Someone sends you the repository's address. Install the skill as above, start the agent in an empty folder, and say:

join the lab at <address>

The Director clones the lab into that folder and first tells you where it stands — that needs nothing from you. Then it asks whether you want to contribute or only read. If you contribute, it tells you what this lab counts as evidence and the rules the group works under, and asks you to agree to them; if you disagree with one, it writes that down for the next meeting rather than arguing or quietly changing it. Then it checks that git knows your name (your work is labelled with it), puts you on your own line of the record, looks at what tools your own machine has, and asks how you want to be talked to. You cite the others' work; you never edit it. What you record reaches the group at the next meeting.

Every day after

Say open the lab. The Director reads what happened since you last looked, tells you where things stand in plain sentences, and proposes the next move. You say yes, no, or something else. When you want the whole picture, read the problem's STATUS.md.

The words

The Director defines each of these the first time it uses one in a message, and GLOSSARY.md in your lab has the full list, each with an example and the file that owns the rule.

• Brief — a task written for a worker: the goal, what it may rely on, what counts as success. "Prove or refute: the largest cap in dimension 4 has 20 points."
• Run — one worker sent on one brief, numbered R-ann-001, R-ann-002, … with your name in the middle, so two people's runs never collide. One counter for the whole lab, never per problem: an ID names one thing.
• Packet — what the worker brings back: a verdict (PASS, FAIL, or UNDECIDED), what it did, what it does not claim, and how to check it.
• Ingest — the gate: the packet is checked, its evidence is re-run by machine, and only then is the run filed. An unfiled run does not exist.
• Claim — one statement with an ID and a status. Proposed means stated; verified means checked here, on record, by someone other than whoever found it; refuted means shown false.
• Replay / review — the two ways a result is validated: a machine re-runs the exact command and checks the exact output, or a different worker checks the argument by hand and the record says who and what.
• Catchup — the report at the start of every session: runs, verdicts, claim changes, what the others recorded, and anything that needs a decision.
• Meeting — how a group agrees: everyone on a call, one at the keyboard, the Director reading out each disagreement, the room deciding, the decision written down before the next item.

When it refuses

Six things are enforced by the scripts. The Director will not work around them, and neither should you; each refusal tells you what to do.

1. A claim's status changes only through the script. You will see the Director decline to write "verified" into a page by hand. It is right; ask it to set the status with the evidence run named.
2. A run enters the record only through ingest. A worker's result that was never filed is not evidence, however good it looked. Ask for it to be ingested, or closed with a reason.
3. No task goes out without a named model. The Director must say which model does the work; a default is fine, silence is not.
4. The scripts and the group's shared settings change only with your explicit agreement. The Director will ask you before changing a rule, record your answer, and run one small test task afterwards. (Your own machine's settings — which models you can run, how they start — are yours to change freely.)
5. A PASS that does not replay is not a PASS. If the machine cannot reproduce what the worker said it printed, the run is filed UNDECIDED with the worker's PASS kept beside it.
6. When a claim falls, everything standing on it falls too. Demote or supersede a claim and every verified claim resting on it, directly or through others, becomes conditional on its return; a referee that finds unproved steps under a proof files them, and the claim rests on them from that moment. The dependency graph cannot loop.

Two more things come to you as decisions, never taken alone: a worker that has gone over its time or memory budget (kill it, or let it run), and a session that has run long enough to degrade (start a fresh one, or carry on).

Working as a group

Several people can share one lab, each with their own copy of it.

• A colleague joins by saying join the lab at <address>. Everyone has their own line of the record from the day they open or join: their runs, claims and notes carry their name, so two people can never label two different things the same way.
• Every morning the Director tells you what the others recorded since you last looked — their tasks, their results, their claims — without mixing their record into yours. An experiment belongs to whoever ran it: you cite someone else's, you never edit it.
• What the group has agreed lives in one place, and it changes only at a meeting. You get on a call, one person shares their screen, and the Director reads out the list of things the records disagree about: the same result claimed twice, one page rewritten two different ways, a task left open for a week. The room decides each one and the decision is written down before the next.
• Afterwards everyone is back on the same page, literally, and the minutes of the meeting are in the notebook.
• Everyone runs the same version of the tooling, because the lab keeps its own copy of it: the Director says when a newer one is installed and asks before bringing it in.
• What your own machine can run — which models, how they start, what is installed — stays on your machine and is never shared. What the group agreed is shared. Nobody has to edit anybody else's setup.

This prevents the two failures that end shared work: two people quietly believing different things until it matters, and a day spent untangling whose copy of a file is right.

The design principle

Every mechanism here names the failure it prevents. If it cannot name one, it should not exist.

• One notebook file per entry, plus a generated index — prevents the single notebook file that grows until nobody reads it and two versions of the bottom line drift apart inside it.
• Claim status only through a script — prevents a status that was hand-edited into a file, believed for weeks, and traceable to nothing.
• The script commits its own writes — prevents evidence that exists on someone's disk but not in the history, which is a rumour.
• Replay with exact marker strings — prevents a worker's report of success standing in for success.
• The discoverer never promotes its own result — prevents a run grading its own homework.
• Catchup — prevents "what happened last week?" being answered by grepping a hundred run directories.
• A commit guard on open runs — prevents a tired Director sweeping a live worker's half-written files into the history.
• A watched wait on every worker — prevents a run running for hours, or thirty times over its memory budget, with nobody told.
• One branch and one set of numbers per person — prevents two people labelling different results the same way, and a day of untangling whose copy of a file is right.
• The worker's transcript kept with its run — prevents a result whose reasoning lives in a session store on one machine until it is pruned.
• Nothing over 50 MB enters the history; cached papers and data stay on disk — prevents a record that can no longer be pushed anywhere, and a repository full of files that were never ours to redistribute.
• One version of the tooling per lab, upgraded on your word — prevents two people's Directors following different rules over one record.

Install, in full

Claude Code can also install it as a plugin:

/plugin marketplace add mattrobball/lab-book
/plugin install lab-book@lab-book

If your tool has no skill folder, or you would rather not find it: copy skills/open-lab/ into the lab folder itself and say "read skills/open-lab/SKILL.md, then open the lab". That works anywhere.

Skill folders differ by tool and move over time; as of this writing:

CLI	Path
Claude Code	~/.claude/skills/
Codex	~/.codex/skills/
Cursor	~/.cursor/skills/
Gemini	~/.gemini/skills/
Copilot	~/.copilot/skills/
pi	~/.pi/agent/skills/
shared, where supported	~/.agents/skills/

Some tools pick up a new skill at once, others after a reload command or a restart; if "open the lab" is not recognised, restart the tool.

Status

Version 2.1.0. The charter, the three references, the templates, the glossary, and both scripts with their test suite (187 tests: python3 -m unittest discover -s tests from skills/open-lab/scripts/). Exercised end to end by a scripted cold start and a simulated two-person meeting, and revised after reading the records of three live labs (about 1,500 runs) against what their Directors had to work around.

What 2.0.0 changed, and why each is not compatible with 1.4.0:

• The record is what is committed. Every status is read from the last commit, so a stray checkout or stash cannot roll a run back to open or a claim back a status. An ingest is one transaction, one commit.
• Run and claim IDs are numbered across the whole lab, with a marker per ID naming its problem.
• A brief opens with a header the harness reads: kind, the run it checks, the claims it carries, where the worker may write, its budget. Carried claims are written into the prompt with their live status; ranges expand.
• Every claim records the model that discovered it; the Director's model is set once in lab.local.json, and verification never says "unknown".
• Workers get an allowlisted environment, never the Director's keys.
• Transcript rules are derived from a real run and kept with an example; token usage is read from the transcript where its shape is known.
• Identity comes from the lab's own settings, asked once, not from git.

Contributing

Issues welcome. For proposed mechanisms, the kit's own rule applies: name the failure it prevents, or it should not exist. Name it in general words — what went wrong, never where: no run IDs, no lab, problem, vendor or model names. The kit's reference run is the one worked example it may cite. Facts about a particular lab live in that lab.