Repository navigation
Design Decisions
Six decisions shaped this package, and each one carried a cost. The sources are the code, its docstrings, and the README's design notes. Where a reason below is an inference from the code rather than a recorded statement, the text says so.
Problem. An orchestrator that dispatches work often runs in another process, where a traceback is unreadable and a single non-zero exit code cannot say which of many things went wrong.
Options. Raise exceptions and let callers catch them; use exit codes alone; or return a result packet for every outcome.
Choice. run_task_packet() never raises. Every refusal is a valid result packet with its own reason, and eleven reason codes exist, from task_packet_unreadable to result_packet_invalid. _refusal() builds the packet from literals of known type, so the refusal cannot fail its own validation.
Trade-off. The gateway has to catch Exception around each handler, which means a genuine bug in a handler arrives as handler_raised rather than as a crash a developer would notice at once. The rule also stops at the gateway boundary, because choose_orchestrator() still raises RuntimeError when no checkpoint exists. That inconsistency is real, although it is defensible: the router runs in process, inside code that must not continue.
What would change it. A second out-of-process caller of the router would justify giving it a result shape of its own.
Problem. A capability can be missing for two opposite reasons, and a single "not available" answer would hide the difference. One is a plan, and the other is a defect.
Options. Treat every unresolved handler as an error; ignore unresolved handlers until someone calls them; or declare capabilities ahead of their handlers and name each state.
Choice. A capability with no handler is capability_unbuilt, which is a normal state, while a handler that will not import is handler_unresolvable, which is a fault in the config. agent-gateway gateway capabilities exits 1 only for the second. On the example config it prints 2 declared, 1 built, 1 unbuilt, 0 unresolvable and exits 0.
Trade-off. The vocabulary grows, and every caller has to learn it. However, the alternative fails silently: a config with a typo in a handler path would read exactly like a feature nobody has written yet.
What would change it. Nothing short of removing declarations altogether, because the distinction follows from having them.
Problem. A release gate that evaluates a misspelled field as false can pass a change that nothing has checked, and that failure leaves no trace.
Options. Python eval over rule strings; two-valued logic with a default of false; or a closed grammar with a third truth value.
Choice. governance/rule_eval.py parses a closed grammar by hand, with no eval, and resolves any reference the context does not supply to UNKNOWN. A rule whose condition is UNKNOWN does not fire, and the decision records it, as the walkthrough in How It Works shows.
Trade-off. The parser and evaluator run to 598 lines, where eval would need a handful, and the grammar covers only what the rules have needed. There is also a gap the design has not closed: the command line exits 1 only when a decision blocks, so an empty context exits 0 while reporting four unknowns. The report is honest and the exit code is not, which makes this the first item on the next steps list.
What would change it. A rule language needing arithmetic or user functions would argue for an existing expression library in a sandbox, and not for eval.
Problem. When the primary model fails, a run needs another orchestrator, yet a fallback model has no memory of the run, and two models that alternate can undo each other's work.
Options. Retry the primary indefinitely; allow free switching; or allow a single switch, on a stated reason, from saved state.
Choice. choose_orchestrator() accepts five eligible reasons, such as primary_rate_limit and operator_takeover, and requires a checkpoint. Once active, the fallback stays active. A reason outside the list, such as an ordinary test failure, keeps the current orchestrator, since a failed test is the agent's work to fix and not a reason to replace it.
Trade-off. A run cannot return to the primary model after a short rate limit clears, so it may finish on a weaker model than it needed. Predictability wins over peak quality here, because an unattended run has nobody present to judge which model should hold the pen.
What would change it. Measured evidence that fallback runs lose quality would justify a guarded return path with a second checkpoint.
Problem. An identical tool call can be a stuck loop, and it can also be legitimate work.
Options. Block every repeat past the limit; warn only; or offer both and let the caller choose.
Choice. governance check-loop exits 1 past the limit, and the caller decides what that means. governance check-loop-hook always returns permissionDecision: allow and adds a warning as additionalContext. In a real run with three calls and a limit of 2, the third produced Repeated tool call detected (3x with identical arguments).
Trade-off. A warning depends on the agent reading it, so the hook cannot stop an agent that ignores it. The repository does not record the reason for this choice. The code suggests one, because the hook uses session_id as the task state, which means running pytest again after an edit produces the same fingerprint as running it before. A hard block there would stop correct behavior.
What would change it. A fingerprint that included a hash of the working tree would separate those two cases, and then a blocking hook would become safe.
Problem. The optional pool sends work to local models, whose replies arrive as text that a model controls in full.
Options. Trust a structured reply once it parses; or let the caller own every field that carries authority.
Choice. normalize_result() takes the task id, the model tag, and the role from the caller and sets requires_review to true, so a model cannot relabel its own result or clear the review requirement. tests/test_pool.py covers this directly in test_a_model_cannot_relabel_its_own_result. The registry also has the last word on location: a task that declares local for a model that runs in the cloud fails with execution_mismatch, which closes a route for sending confidential work off the machine.
Trade-off. Pool results can never act on their own, so a human or a primary model must always spend attention on them. For advisory work that is the intended price.
What would change it. Nothing in the current scope, since write authority for local models lies outside it.
Designed and authored by Daniel Boles; implemented with coding agents. Apache-2.0.