Writing an orchestrator

Writing an orchestrator

The reference for tools/argus_orchestra.py. If you have not seen one run yet, start with Orchestrating several agents — this page is the surface, not the tour.

Three layers, and which one you want

The API	HTTP, documented in The API and live at Settings → Go to → The API, live. Everything else here is a client of it. Use it directly from a language that is not Python, or from curl.
tools/argus_client.py	the transport: who, launch, relay, ring, worktree, prefs, prompts, bells. It reads the config, prefers the agent key, and knows the four awkward things about the bell stream. One call in, one answer out.
tools/argus_orchestra.py	the framework: several agents, in an order, each waiting for something. It owns the contract in each prompt, the waiting, the names, the worktrees and the report.

Reach for the client when what you are writing is one call — a hook that rings, a cron job that starts a thing. Reach for the framework the moment there is a second agent and a wait between them, because that is where the two hundred lines of plumbing used to go.

Both are standard library only, one file each, importable or copyable. Neither is on PyPI on purpose: this project says its API can change between commits, and a versioned client would be promising a surface it cannot hold still yet.

The client, in full

Everything argus_client.py offers. Every method is one HTTP call unless it says otherwise, and every one of them raises ArgusError on a refusal rather than returning something empty.

from argus_client import Argus, ArgusError, TooFast, credentials, config_path

a = Argus()                       # reads the config, prefers the agent key
a = Argus(base="http://…", token="…")   # or say so yourself

Looking

a.who()	sessions, who is in each, in which folder, and which are waiting for a person — the four questions the browser asks separately, in one answer. The first call an agent should make
a.sessions()	the raw session list, when who() is more than you need
a.launchers(versions=False)	what this machine can start, and whether each is really installed. versions=True also asks each one what version it is, which costs about two seconds
a.prefs()	what the browser remembers, as the machine has it: the desks, the windows, and — the reason a script wants it — the prompt library and the placeholder sets
a.prompts()	the prompt library alone, or [] if no browser has synced yet
a.worktrees(path)	the repository that path is in, and every checkout of it

Doing

a.launch(launcher, name, where=".", prompt="", run=False, worktree=None, wait=True, wait_seconds=None, desk=False)	start something from the launcher list. run=False types the prompt in and leaves the return to a person; worktree="branch" cuts a checkout first and starts in it; desk=True puts a window for it on whoever has Argus open. The answer says whether it settled before the prompt went in, which is the failure this is designed around
a.relay(to, text, run=False)	a sentence into a session that is already running — what dragging a prompt onto a terminal does, offered to a program
a.ring(text="", why="asking", session="")	call the person. asking waits for one; done and failed only report. session defaults to $ARGUS_SESSION
a.worktree(repo, branch, to=None)	a second checkout on its own branch. Refuses a path that already exists, and never removes one

Waiting

a.bells(until, since=0)	bells as they ring, over one open connection, until until (a time.monotonic() value). A generator
a.wait_for(paths, until, on_bell=None)	wait for files to appear, woken by bells. Returns (arrived, missing)

bells takes a deadline rather than letting you check the clock between yields, and that is the whole reason it is worth a library: the stream sends a heartbeat every twenty-five seconds and never ends, so a for loop that checks the time between bells never gets the chance. Measured, before it was fixed: an orchestrator asked to wait ninety seconds was still waiting after five minutes. The socket timeout tracks what is left, too — forty seconds of patience when ten remain is thirty seconds late.

The awkward corners

a.call(method, path, body=None, timeout=60)	any route at all, with the token attached and the errors unwrapped. Use it for anything the methods above do not cover — nothing is hidden behind them
credentials()	(base, token) read out of ~/.config/argus/config.yaml. Prefers the agents: key over the master one, and $ARGUS_TOKEN over both
config_path()	where it looked. $ARGUS_CONFIG overrides it, which is how you point a script at a second Argus on the same machine
ArgusError	something Argus refused, carrying .status and the server's own sentence
TooFast	a subclass, for 429 only, so a script can back off rather than die. The message names the config key to raise

main() is the command-line front door, and tools/argus-say is three lines over it.

Both tables above are checked. scripts/surface.py enumerates every public name in tools/ from the code and the test suite fails if one has no docstring — or, with a checkout of this wiki, if one is named nowhere on it. CI clones the wiki so that second check is real. It exists because this page was once eight methods out of fourteen.

Orchestra

from argus_orchestra import Orchestra

o = Orchestra("~/work/api", launcher="Claude Code", prefix="", minutes=30,
              run=True, watch=False, on_lost="continue", name="")

where	the folder everything defaults to, and the repository worktrees are cut from
launcher	a name from argus-say who. Checked before anything starts; a wrong one exits with the list
prefix	goes in front of every session name and on the last segment of every branch, so two runs of the same thing live side by side
minutes	the default patience for every wait; each call can override it
run	False types every prompt in and leaves the return to a person. Do this first
watch	puts each session on the desk as it starts, and posts the run's shape so a browser can draw it live
on_lost	"continue" (default) carries on with what came back; "stop" refuses to go past a step somebody did not finish
name	what the run is called on the board. Defaults to the script's own filename
argus	an Argus instance, if you have a reason not to let it build its own

Starting things

group = o.fan_out(items, say, until=None, worktree=None, minutes=None, wait=True)
agent = o.step(say, name="step", until=None, where=None, worktree=None, minutes=None)
agent = o.start(name, say="", until=None, where=None, worktree=None)

fan_out — the same job, N ways, one agent each. items is a list of strings, or a dict when each one needs a short name and a long description: the key fills {each} and the value fills {value}, which is what makes ref-{each} and "you are reading it for {value}" two halves of one line. Blocking: it returns when they have all finished or the clock runs out. wait=False returns as soon as they are started.

step — one agent, and wait for it. The ordinary case: a judge, an editor, a summariser.

start — one agent, left to it. Use this when the agent should be alive for the whole run and told things later, which is what fullstack.py does with its three roles.

The placeholders, in say, until and worktree: {each}, {value}, {i}, {n}.

Waiting, and telling

o.wait(*agents, minutes=None)                                # until each writes its `until`
result = o.wait_for(path, minutes=None, fresh=False, where=None)
o.tell(who, text, run=True)                                  # a sentence into a running session
for turn in o.rounds(3): ...                                 # a loop that says which round it is
o.report()                                                   # what happened, and where to look

wait_for takes a path rather than an agent. fresh=True waits for it to change rather than to exist, which is the round-trip case: a tester rewrites the same FAILURES.md every round and a stale one would end the loop early with the wrong answer.

tell accepts an Agent or a bare name, and applies the prefix either way — a session name is an address, and frontend in another run is somebody else's agent.

rounds is bounded on purpose. An orchestration that can go round for ever will, on the night you are not watching; reaching the cap prints that it is a cap and not a verdict.

What comes back

group.done      # the agents that wrote their file
group.lost      # the ones that did not
group.paths     # their files, as Paths
group.files     # "- a cache: /path/RESULT.md\n- an index: …"  — for the next prompt

agent.done      # did it write it
agent.file      # what it was asked to write
agent.where     # the folder or worktree it is in
agent.text()    # what it wrote

result.says("ALL GREEN")
result.split_by("backend:", "frontend:")   # {"backend:": [...], "frontend:": [...], "": [...]}

fan_out gives you a Group, start/step give you an Agent, and wait_for gives you a Result — or None when nothing wrote the file.

for agent in group: ...        # a Group iterates, and has a len()
len(group)

agent.name      # the tmux session, prefix and all — the address `tell` writes to
agent.label     # what you called it: the idea, the lens, the role. What the diagram shows
agent.branch    # the worktree's branch, or None
agent.state     # "waiting" · "working" · "asking" · "done" · "lost" — what colours the diagram
agent.arrived() # is the file there *now*, regardless of what was decided earlier

result.path     # where it was
bool(result)    # false when the file is empty, so `if said:` means what it looks like

group.files is the method that earns its class. Every orchestrator ends up interpolating "here is what the others produced" into the next agent's instructions, and doing it by hand is a loop, a join and a formatting decision in the middle of the interesting part.

split_by keeps the lines that matched nothing under "" — a failure nobody owns is still a failure, and dropping it is how it never gets fixed.

The rest of it

Small, and here because a reference that lists twelve of twenty-one methods is a reference you stop trusting.

o.say(line="")	print through the framework, so your own lines sit in the same stream as its report rather than racing it
o.named(name)	the session name this run would use for name — the prefix rule, in case you need to build one yourself
o.free(names)	refuse now if any of those names is already running. Called for you by start and fan_out; call it yourself before doing something expensive
slug(text, n=24)	the sentence-to-name rule, trimmed at both ends and after the cut
o.argus	the client underneath. Nothing is hidden behind the framework: when it does not cover something, o.argus.call(...) is right there, holding the same token
o.started	every Agent this run has started, in order
o.stages	the shape the diagram is drawn from: [{"name": …, "agents": [Agent, …]}, …]
o.here	the answer to who() from when the run began — the machine, its sessions, its launchers
o.id, o.state, o.name	how the run appears on the noticeboard
Orchestra.LOOK_EVERY	10 seconds — how often the waiting loop looks at the disk regardless of bells
Orchestra.BEAT_EVERY	60 seconds — how often a watched run says "still here", so five minutes of silence can mean something

Both constants are class attributes: change them on the class, or on one instance, if you have a reason. The defaults are chosen so that a file appearing with no bell costs you ten seconds and a killed script is noticed in five minutes.

The prompt an agent actually receives

Yours, plus two additions, both printed by a dry run because a framework that quietly edits what you told an agent is a framework you cannot debug.

Try this approach: a cache in front of the query.

Work only in /home/you/work/api-try-a-cache.          ← only with a worktree

When you are finished, write /home/you/…/RESULT.md.    ← only with `until`
When it is written, run:  argus-say ring --why done --session a-cache

That last line is the reason the framework exists more than any other: it has to name the right session, in every prompt, every time, and it is the thing everybody gets wrong.

It also means argus-say must be on the PATH on the machine the agents run on. install.sh puts it there; from a clone it is one symlink.

The three constraints you will meet

Properties of the substrate, not shortcuts, and anything you write will run into them.

You cannot read what an agent said. Argus types into a session; it cannot read one back. Reading a pane means capture-pane, which is scraping a text user interface and — on at least one machine this was tested against — a way to take the whole tmux server down. So coordination goes through the filesystem: agents write files, the framework reads files.

There is no implicit "finished". An agent is done when it says so, which is why until and the contract exist. An agent that writes its file and forgets to ring is explicitly not a failure: the bell is the signal, the file is the fact, and the loop checks the disk every ten seconds regardless.

Nothing retries, supervises or recovers. A timeout fires and you are told which agent never finished, with the path to its worktree. A real supervisor is another hundred lines of the same kind, and it should be yours — the framework is blocking Python precisely so that while, if and try still mean what they normally mean.

What it costs to be wrong

Two brakes, and they are not security — the launcher list and the agent key are that.

• launches_a_minute, twelve. A deliberate fan-out of four referees and an editor is five in as many seconds; a runaway loop is hundreds.
• relay_a_minute, thirty. A pokes B, B pokes A, and by morning there are nine hundred lines of two robots talking.

Both are config keys, and the 429 names the one to raise. ArgusError carries the status and the server's own sentence; TooFast is its own class so a script can back off rather than die.