feat: task routing — choose which tasks the blind gate engages on#1
Merged
Conversation
…draft) Replace the earlier "When the Author Grades the Exam" technical draft with the stronger, audience-targeted "Backpressure" version (policy-vs-mechanism thesis, loop-engineering framing, integrated prior art, ImpossibleBench policy-ceiling hook, cost/quadrant + drift framing). Ported in from the prior draft: n=21/cell + confidence-interval candor, the "recomputable from the 84-run ledger" provenance line, the 4/7 Bash-escalation texture that justifies the hash layer, and the one-harness/model-drift self-objection. Fixed two staleness issues: the mutation pass is now described as built-but-advisory (not "planned"), and the unverifiable "92%" prompt figure is stated as ">85%" (what the paper's prose supports). Still draft: two figure placeholders (architecture diagram, cost quadrant) and the [link to narrative essay] / [link to repo] / [link] placeholders. The narrative essay is authored separately. lesswrong-post.md is retained, unused. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Tighten "Backpressure" from ~3740 to ~1350 words: drop the documentation aside, collapse the triple-stated mirror idea to one statement, fold the middle-ground layers into the thesis, and cut the cost-quadrant, drift, and extended self-objection passages (full version preserved at 49bd6a1). Name the cited researchers in full — Zhong/Raghunathan/Carlini (ImpossibleBench), Pan/ He/Bowman/Feng, McKee-Reid, and Pan/Jones/Jagadeesan/Steinhardt. Local-only by request; not pushed. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add a human-set routing policy (gate.blind_tdd.routing) that reserves the expensive multi-agent gate for high-stakes work and lets an ordinary in-loop verifier cover the rest. Four OR-combined predicates: path globs (matched against operator-authored files/public_surface — the most evasion-resistant), tags, min_severity, and spec keywords. mode:"all" (the default when no routing block is present) keeps the legacy gate-everything behavior, so this is fully backward compatible. - blind_tdd/routing.py: policy normalization + evaluation, with a trust note that the metadata source must be outside the worker's reach. - gate_integration.py: consult routing after task resolution; a non-matching task returns phase="skipped", reason="routing_excluded" before any spawn. - blind_tdd/init.py: setup wizard (python -m blind_tdd.init) — interactive interview and non-interactive flags, merging into themis.config.json without clobbering other keys. - plugin command /blind-tdd:setup drives the same interview (auto-discovered; no manifest changes). - docs: adoption-guide routing section + config row; README pointer. - Also tightens the README 7/4 adversarial-arm phrasing to match the essay. The trigger is human-set on purpose: it must be a rule the implementing agent can't choose to skip, or it would skip it exactly when the pressure to cheat is highest. Routing decides coverage, not integrity — a gated task still gets the full sealed, blind treatment. 24 new tests (test_routing.py, test_init.py); full suite green (360 passed). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds a human-set routing policy (
gate.blind_tdd.routing) that decides which tasks the blind gate engages on, so a project can reserve the expensive multi-agent gate for high-stakes work and let an ordinary in-loop verifier cover the rest. This brings the stakes-based routing that previously lived only in the external RALPH loop into Themis itself.How
mode: "selective"gates a task if it matches any of four OR-combined predicates;mode: "all"(the default when there's noroutingblock) keeps the legacy gate-everything behavior, so this is fully backward compatible.**/*/?, matched against operator-authoredfiles/public_surface(the most evasion-resistant)low<medium<high<critical), keywords (spec-text substring)A non-matching task returns
phase="skipped", reason="routing_excluded"before any agent spawn.Setup (both surfaces)
python -m blind_tdd.init— interactive interview and non-interactive flags (--path-glob,--tag,--min-severity,--keyword,--enable/--disable,--print,--yes); merges intothemis.config.jsonwithout clobbering other keys./blind-tdd:setup— new auto-discovered plugin command driving the same interview (no manifest changes).Honest caveats (baked into code + docs)
tags/severitycould self-label "low-stakes" to dodge the gate.path_globsis the resistant predicate. The wizard exists so a human sets the policy once — the trigger must be a rule the agent can't choose to skip.Tests
24 new tests (
test_routing.py,test_init.py); full suite green (360 passed). New files are ruff-clean.🤖 Generated with Claude Code