Repository navigation
v0.4.0
v0.4.0
Performance
- Selecting a target is 41× faster on a large belief state: 1200 nodes went from
4 494 ms to 110 ms per dispatch, and the curve from ~3.8× to ~2.3× per doubling of
nodes. None of this was visible at the 40–60 nodes an evaluation episode produces, which
is why it survived 662 tests — but every long-lived workspace lands on the wrong side of
it, and "the belief state survives across sessions" is the product claim.- The dominant cost was an N+1 query.
get_all_nodesderives each node'sparent_ids
from the edges table and was issuing one query per node; the navigator calls it
several times per dispatch, so handing out two targets cost ~6 000 SQL round-trips.
Parents are now resolved for every row in a single query. HypoTreeGraph.add_edgeran a fullis_directed_acyclic_graph()check per edge, so
reloading E edges cost O(E·(V+E)). Newadd_edges_bulkchecks the finished set once
and rolls back if it is cyclic. The per-edge path stays for interactive creation, where
naming the offending edge is worth the cost.HypoTreeGraph.parents()scanned every edge in the graph and the frontier calls it once
per node. Parents are now indexed bydst.get_next_targetssyncs the in-memory graph once per batch instead of once per pick.
The batch loop takes leases, which change node rows but never the topology.- A long-lived incremental graph — invalidating on write, never rebuilding — is
deliberately not done. A cached graph that disagrees with SQLite would produce
silently wrong selections, and at 110 ms there is no case for taking that risk.
- The dominant cost was an N+1 query.
Added
- The closed-world assumption is now declared rather than assumed. Deduction by elimination
— confirming the last survivor of an exclusion group for free — is sound only if the listed
candidates really are all of them, and nothing checked that. Withhold one axis's winning
value and the engine refuted the four that remained and confirmed the survivor with no
observation of its own: a free confirmation of a value that is wrong, recorded as a
confirmation. The run then endedempty_frontierwith the goal unmet and never said the
question had been incomplete. This is the mechanism the README credits for the moat, and it
had shipped unguarded for months.exclusion_closedoncreate_hypotheses(schema10,nodes.exclusion_closed,
NOT NULL DEFAULT 1— exactly what every group written before it meant). Defaults to
true. Passfalsewhere the next candidate always exists — "which learning rate", "which
prompt wording", "which threshold" — and the last-one-standing deduction is withheld, along
with the pruning that rests on it. Confirming a member still retires the others: that is an
observation about the confirmed value, not a claim about the list.- Openness is per-group and one member declaring it is enough. It withdraws an inference,
so the cautious declaration governs; the reverse would let a later call that forgot the flag
quietly re-enable deduction over a list its author knew was partial. - A deduction can now be withdrawn. When every question is answered and the assembly still
falls short, the belief with no measurement behind it is the one to doubt. The engine hands
it back asUNTESTEDand the navigator dispatches it, so one probe settles whether the value
was wrong or the list was incomplete. It never asserts the value false — nothing was ever
observed about it, and asserting on no evidence is the one thing the belief state may not do.
The same applies at conviction: a deduction named as a conflict's sole cause is tested rather
than convicted. _is_deducedreads the whole history, not its last entry. A deduction that passes
through conflict review comes back marked "released from review", which describes the last
thing that happened to it and erases where the confirmation came from — a node never once
observed looked like one that had stood up to scrutiny.dead_questionnow says which kind. Over a closed group it means the list was wrong or
one of the eliminations was, and what depended on those values is pruned. Over an open one it
means the answer is very likely one that was never listed — and nothing is pruned,
because an untried candidate could still satisfy it.- Measured. Across all 30 seeds × 5 axes with the winner withheld, 144/150 (96%) now
enddead_questionnaming the incomplete axis, against 0/150 before — every one of which
wasempty_frontier. Still 0 solved, which is correct: the winning value was never offered,
so the goal is genuinely unreachable, and what changed is that the run says why. A wrong
answer reported as wrong is a result; the same answer reported as an empty frontier is not.
Fully-declared self-play is untouched at 30/30, mean 15.9.
- The dashboard runs by default. It was behind
--dashboardwhile it was unproven, and the
result was that the people it was built for never saw it — nobody opts into a feature they
have not been shown. It binds loopback only, mints a fresh token per start, and cannot take
the MCP server down: a bind failure is reported and the server carries on, because the cost
of a viewer failing must never be the cost of the tooling failing.--dashboard-port PORTpicks the first port to try (it still probes upward, so several
workspaces can be open at once).--no-dashboardopens no socket at all.--no-mcpreplaces--dashboard-onlyand reads better beside its sibling. v0.4.0 is
unreleased, so the old spelling never shipped and is simply gone.
- The agent can hand over the dashboard link. The URL is minted at startup and printed to
stderr, which the model never sees — so an agent asked "where can I watch this?", which is
constant, had no way to answer.get_workspace_infonow returnsdashboard_url, and the new
hypotree://dashboardresource returns it on its own. Both carry the session token, which is
the whole credential;nullmeans none is running, and the text says how to start one. - Backward pruning over a complete question. The exact dual of deduction by elimination,
and the half of that symmetry that had never shipped: all-but-one candidate eliminated
confirms the survivor for free, and all of them eliminated says the question has no answer
among the candidates offered — so nothing that assumes one of them can ever be satisfied.
Those branches are pruned under their own reason marker and the navigator reports a new DONE
reason,dead_question, naming the group that ran out and the goal it was holding up.- Soundness rests entirely on the exclusion group. "All four approaches I tried failed"
does not entail "the objective is unreachable" — it entails "I need a fifth idea", and a
rule that could not tell those apart would let an unimaginative agent declare its own goal
impossible. What makes this version sound is that the caller declared these to be the
competing answers to one question. - Three guards, each closing a way it could fire on a live search: at least two candidates
(one value is not a question); every member eliminated on its own evidence, which is
strictly stronger than the elimination test deduction uses —PRUNEDdoes not count,
because it is a statement about an ancestor's subtree, and neither does a member with no
observation of its own; and no open conflict over any member, because conflict diagnosis
may hand one straight back. - It reaches what the refutation cascade deliberately spares. A premise settled as
EXHAUSTEDwas never refuted, so its dependents stay on the frontier being re-attempted
long after the premise ran out; that is the case this exists for. dead_questionis an instruction, not an ending: the fix is to add the candidate that was
never listed, and the eval harness treats it as a reason to continue.
- Soundness rests entirely on the exclusion group. "All four approaches I tried failed"
record_evidenceis batch-native. Passresults=[{node_id, success, depth, claim_id}, …]
to report every experiment from one turn in one call.create_hypotheses,update_status
andget_next_targetshave all been batch-native for a while;record_evidencewas the
last singular high-frequency tool, and it is what pinned a synchronous agent at two turns
per experiment — one to probe, one to report. The single-result form still works and is the
k=1shorthand.- Results are applied in order, so a refutation's cascade lands before the next result
in the same batch is read and the belief state ends where it would have ended had the
results arrived separately. - A refused report does not abort the batch. Every result was paid for by an experiment
that already ran, so discarding k-1 of them to punish one badnode_iddestroys evidence.
The refusals come back underfailed, named and explained. - The fused dispatch runs once, after the whole batch, and is still a top-up.
- Results are applied in order, so a refutation's cascade lands before the next result
get_conflictsnow reportsskipped_no_substituteper member.- A read-only web dashboard:
hypotree --dashboard [PORT]andhypotree --dashboard-only [PORT].
A belief state that revises itself is hard to appreciate from a status column. The
dashboard runs on the MCP server's own event loop and serves the graph, the narrative and
a timeline over localhost HTTP.--dashboard-onlystarts no MCP server at all — the
try-before-you-wire path for someone who has not configured a client yet.- The interface. Vue 3 over server-computed coordinates, with
d3-zoomfor
hardware-accelerated pan and zoom. Untested nodes glow at their real chance of being
dispatched next; in-progress nodes pulse; new nodes fade in as the agent creates
them and pruned branches desaturate rather than disappear, because the claim being
demonstrated is that they were considered and cut. Edges into a dead branch turn red. - A timeline scrubber over the bi-temporal history: drag back to any instant, or press
play and watch the investigation replay. - Proof mode overlays what each belief cost — score, depth, commit,
source_ref. - The learning path is typeset markdown, ready to paste into a report.
- Everything is vendored (Vue 3, nine d3 micromodules,
marked— 276 KB). No CDN, no
npm, no build step: it works on a plane and in an air-gapped network. d3 micromodules
rather than the full build (63 KB against 273 KB) because the layout is computed
server-side and nothing else in d3 is wanted. - The observer physically cannot write. Its store is opened
mode=ro, so a read path
that is wrong fails loudly instead of quietly mutating the belief state being watched.
WAL already permits a second reader, so it never contends for the engine's write lock. - Layout is computed server-side with
networkxand shipped as coordinates. No
JavaScript graph library, so nothing large is downloaded and the layout is deterministic
and testable in Python rather than eyeballed in a browser. p_selectis the real selection probability, estimated by repeating the Thompson
draw and counting argmax wins — not a monotone stand-in like the posterior mean.- Time travel is a
WHEREclause.?at=<iso8601>reconstructs any past instant from
the SCD2 status and posterior intervals, so a replay shows what was believed then. - SSE carries a revision number, never a payload.
events.seqadvances inside every
mutation's transaction, so it is a correct change signal by construction; clients refetch.
A subscriber that cannot keep up is dropped rather than allowed to apply backpressure. - Everything is cached on the revision, which is also the SSE payload — without it a
scrubber would rebuild the whole graph per frame. - Security. Binds
127.0.0.1explicitly, mints a session token at startup and requires
it on every/api/*call, validates theHostheader (the DNS-rebinding defence behind
CVEs in Jupyter, Ray and TensorBoard), refuses cross-origin requests outright, and serves
the page from a string so path traversal is unrepresentable rather than filtered. The URL
is printed to stderr — stdout is the JSON-RPC channel.
- The interface. Vue 3 over server-computed coordinates, with
- Pin and suspend: human scheduling directives that are not beliefs. A new
node_directivestable records "look at this first" or "leave that alone" with an actor and
a reason. Directives change what the navigator offers, never what is believed — writing a
click intoalpha/betawould make it forever indistinguishable from an experiment and
inject unlogged nondeterminism into a seeded sampler. With none set, selection is unchanged. edges.created_atrecords when an edge appeared, so a timeline replay no longer draws the
final topology at every tick. Additive; edges written before it are treated as original.- A migration adding a column to a table the DDL had just created failed. The DDL
described the current schema and ran before the migration chain, so a table the file was
missing got created already-modern and the step adding its column then hitduplicate column name— a collision every future migration would have met, not just this one. The
schema is now defined the way Doctrine and Rails define one:BASE_DDLis the original
shape, frozen,CREATE TABLE IF NOT EXISTSthroughout, and every change since is a
numbered step applied in order. A fresh database is built exactly as an old one is
upgraded, so the upgrade path is exercised on every install rather than only on other
people's machines.SCHEMA_VERSIONis derived from the last step and cannot drift from it. - New
schema_statetable: exactly one row, enforced byCHECK (id = 1). Carries the
schema version, the hypotree release that last opened the file, and when. It is the first
thing to look at when a database behaves unexpectedly or a bug report arrives with one
attached. The pre-0.4.0schema_metakey is still read as a fallback and kept in step for
one minor version. A freshly created database logs no migration events — it was created,
not upgraded. - The dashboard did not run in a browser.
default-src 'self'blocked Vue's global build
compiling its in-DOM template withnew Function, blocked the inline script carrying the
session token, and blockeddata:fonts injected by browser extensions. Every directive is
now named explicitly rather than leaning on the fallback, and the token travels in a
<meta>tag soscript-srccan keep refusing'unsafe-inline'— the directive that stops
an injected event handler from ever running. - The learning path rendered untrusted text through
v-html. Node statements are written
by an agent, andmarkedhas had no sanitiser since v5, so markup in a statement reached
the document verbatim. An allowlist sanitiser now runs over the parsed output; the strict
script-srccovers what a sanitiser bug would miss, and vice versa. - Work on one objective at a time:
goal_idonget_next_targets,
generate_learning_pathandget_goal_status. A workspace pursuing several goals
interleaves them — the frontier offers whatever is most uncertain anywhere, and the
learning path narrates every goal's dead ends into one story. Agoal_idrestricts all
three to the case for one objective: that goal, everything it depends on, and the
competing answers to those questions. Omitting it is the previous behaviour exactly,
down to selecting the identical node on an identically-seeded engine.- Scope is DEPENDENCY-ancestry plus the exclusion-group siblings of that ancestry,
not graph reachability. A competing answer is not an ancestor of the goal, but testing
it is how the engine learns whether the answer that is an ancestor holds — a scope
without it hands the navigator questions it is forbidden from answering. - New DONE reason
goal_scope_empty. Agents create premises before wiring them
(awaiting_compositionexists because they do), and unwired nodes are outside every
goal's scope by construction. Rather than reporting a bare empty frontier — which reads
as "the search is over" — it names how many untested hypotheses are outside the filter
and says the fix is usually a missing DEPENDENCY edge.
- Scope is DEPENDENCY-ancestry plus the exclusion-group siblings of that ancestry,
Fixed
-
The live stream died every fifteen seconds on Python 3.10.
asyncio.TimeoutErroronly
became an alias of the builtinTimeoutErrorin 3.11, so the keepalive'sexcept TimeoutError
caught nothing on the project's own minimum version: the first quiet interval killed the SSE
task, the browser reconnected, and the cycle repeated. It presented as a dashboard that
flickered between "live" and "reconnecting" for no reason, which reads as a flaky network
rather than as a bug. Caught asasyncio.TimeoutError, which is correct on both. -
A goal can no longer be given as a DEPENDENCY parent. This is the one modelling mistake
strong models make reliably: "the goal decomposes into phases" is how everyone thinks about
objectives, soparent_ids=[goal]on the phase reads as "this phase belongs to that goal". In
hypotree it means the opposite — the phase cannot be tested until the goal is VERIFIED — and
both halves then break silently. A goal is never dispatched andverify_upstreamonly promotes
IN_PROGRESS nodes along REFINEMENT edges, so the goal can never reach VERIFIED and the child is
blocked forever, while the goal still depends on nothing and so can never be reached either.
The engine had been handling this downstream, reporting the wreckage asblocked_frontier;
it is now refused upstream with aGoalDependencyErrorcarrying the corrected call. Only
DEPENDENCY is refused, because only DEPENDENCY gates a child on its parent.- A real behaviour change: five existing tests were constructing exactly this state to exercise
the downstream reporting. They now build the same situations from ordinary nodes —
blocked_frontierfrom premises wired under an EXHAUSTED combination, which settles without
cascading — which is the general case and the one still representable. - Databases written before this release keep the shape and nothing is rewritten. The engine may
change what it asserts only from an observation or a sound inference over one, and silently
flipping an edge someone declared is neither. The dashboard names such a goal's wiring as
invertedrather than merelyunwired, because "turn your edges around" and "attach some
work" are different instructions. parent_idsis described as what this rests on at every point of use, and the guide states
the direction before it states the edge types.
- A real behaviour change: five existing tests were constructing exactly this state to exercise
-
Selecting a goal hid the tree hanging off it. Agents routinely wire a goal the other way
round —goal -> phase0 -> work, reading the goal as a container rather than as something the
work supports. The navigator's goal scope is its DEPENDENCY ancestry, which is the right set
to dispatch from and the wrong set to draw: filtering to such a goal showed a single dot beside
a lecture about wiring while a nine-node tree hung off the node in question. A viewer that
hides your work to make a point about how you built it is not making the point. The dashboard
now walks both directions — the selection scope plus everything downstream plus the
exclusion siblings of the result — and reports the unreachability in a dismissible banner over
the canvas instead of in place of the graph. The navigator's scope is deliberately unchanged:
it must not start selecting downstream of a goal just because the viewer draws it. The timeline
follows the drawing for the same reason the graph does — the scrubber moves the picture, so a
tick it cannot show is a step that does nothing. -
The left panel flashed on every node click. Selecting a node cleared
detailbefore
fetching it, which mounted the entire narrative for one frame before replacing it again. The
panel now branches on what is selected rather than on what has arrived, so the card
renders immediately and fills in. A slow response for a node the reader has already moved off
no longer overwrites the one they are looking at. -
A directive reloaded the whole view. Pinning a node rebuilt the narrative and the layout
to change one badge; it now refreshes the node, the graph and the frontier, which are the only
things a directive can move. -
Binary assets were dropped from the manifest. The table only carried
.jsand.css, so
the logo would have 404'd — a broken header rather than a missing feature. PNG and SVG are
served with the right content type and ship in the wheel. -
A dry run wrote to the belief state.
get_next_targets(dry_run=True)is documented as a
peek and issues no claim, but when a conflict's targeted narrowing had run out of advice the
peek performed the fallback: it marked the conflict recovered, released its members from
review and reopened every alternative they had retired. A caller asking what it would be
given settled the question it was asking about, and the next real call saw a belief state a
peek had moved. The sibling fallback on the same path was already guarded, so this was an
inconsistency rather than a design decision. The peek now reports the instruction it would
still give and changes nothing. -
The conflict fallback timestamped itself off the wall clock rather than the instant the
selection pass was running at, so the status changes it wrote landed outside the interval
the rest of that pass recorded — a small tear in a history whose whole value is that it can
be read back. -
The dashboard's own buttons were refused as cross-origin. The write path rejected any
request carrying anOriginheader. Browsers attachOriginto same-origin JSON POSTs
too, so pin, suspend, back and clear each came back403 cross-origin requests are refused
— the CSRF defence was locking the page out of itself. The header is now validated against
the bound address instead of being treated as evidence of an attack: a foreign origin, or
the right host on a different port, is still refused, and an absent header still falls
through to the token andHostchecks. -
The narrative ignored the time-travel scrubber. Rewinding the graph left the learning
path describing conclusions the picture had not reached, which is worse than showing
nothing.generate_learning_pathtakesas_ofand/api/learning-pathtakesat, so the
briefing stops where the graph stops. Observation tallies are counted at that instant too,
via the newHypoTreeStore.count_evidence_by_node(before=...)— one aggregate that also
replaces the per-node cached counts on the live path. The rewound render says which line
still reflects the present, because goal achievement is read off the live graph. -
The timeline handle sat at the far left while showing the present. The scrubber ran from
-1, and-1meant live — so watching the current state pinned the handle to the start of
history and playback had nowhere to run. The present is now the right-hand end: the handle
rides it as new history arrives,livereturns to it, andplaypressed at the present
rewinds to the beginning first. -
A member the diagnosis could not test was reported as cleared. Substitution progress was
an integer cursor, which can only say "the first k were dealt with". A member the plan had
to pass over — because its question had no live alternative left to swap in — sat behind
the cursor the moment a later member was cleared, and was reported as exonerated by an
experiment that never ran. It could also never be revisited if an alternative freed up
later. Progress is now a persisted set of cleared ids: cleared and skipped are recorded
as the opposite claims they are, and the plan reconsiders a skipped member on every call.
Schema"9"→"10"addsnogoods.cleared_ids; the migration is additive and the set is
back-filled from the old cursor on read, so a diagnosis in flight across the upgrade keeps
the members it had genuinely cleared.probe_indexis kept in step and deprecated;
HypoTreeStore.advance_nogood_probewarns and will be removed in 0.5.0. -
Schema versions were ordered as strings.
"9" > "10"and even"3" > "10"are true
lexicographically, so the first two-digit schema version made a database with a corrupt or
older stamp report itself as "written by a newer hypotree" — sending the user to upgrade
the package instead of keeping the file. Ordered numerically now; a stamp that is not a
number is not treated as newer, which routes it to the "back it up and open an issue"
branch where it belongs. -
A success could resurrect a pruned branch. Recording a passing result on a
PRUNEDnode
flipped it toVERIFIED, leaving the belief state asserting both that the branch is dead
and that it holds — a confirmed node sitting on an invalidated parent. A pruned node's
status is a statement about its ancestry, so the measurement and the posterior are kept and
the status is left alone. Rare while probe and record alternated in fixed pairs; routine
once results arrive in batches, because one refutation in a batch can prune the node a
later result in the same batch belongs to. -
source_refwas advertised and dropped. 0.3.2 added the field, documented it on the
record_evidencetool schema, and never read it in the dispatch — so every value sent over
MCP was silently discarded. The single and batch paths now build evidence through one
helper, so they cannot drift apart again. -
The shortfall recovery fired on a search that had already won. When every question is
answered and the assembled answer falls short, the engine hands back the alternatives those
answers had retired — the reading being that one of the confirmations must be wrong. It did
not check whether a different assembly had since cleared the bar, and a composition that
succeeded withdraws exactly that reading: the answers were right, the earlier assembly was
simply the wrong combination of them. One real episode reopened six settled questions on the
turn it found its answer, because its goal node had never been wired to the winning
composition and an empty frontier looks the same from the inside either way. The recovery
now stands down once any composition isVERIFIED. -
Two different recoveries wrote the same reason marker.
_recover_from_underperformance
reusedINTERACTION_REOPEN_PREFIX, so a history could not distinguish "a conflict turned
out to be an interaction effect" from "every answer was in and the assembly still missed" —
and a run report claimed every conflict had been narrowed to a culprit and that six
alternatives had been reopened because a conflict was an interaction effect, which are
mutually exclusive endings. NewUNDERPERFORMANCE_REOPEN_PREFIX. Additive: histories written
before 0.4.0 keep the old prefix and still read as interaction reopens, which is the only
label that ever existed for them.
Changed
- The dashboard has been redesigned as a light analytical interface. The previous dark theme
was hard to read for long and, more to the point, spent colour on chrome — so status, the one
thing colour should carry, had to compete with it. Now warm paper (#FAFAFA), a gridded
canvas that pans with the graph, white cards on soft shadows instead of heavy borders, and a
single navy/cyan accent reused for goals, focus and selection so the eye learns one colour
rather than three. Every status has a pastel badge;PRUNEDadds a diagonal hatch, because
grey alone reads as "not started" rather than "cut". No webfont is vendored:font-src 'self'
would mean shipping a ~90 KB binary in the wheel to displace a system stack that is
Inter-class on every current OS.- Node cards put the id in its own monospaced bar — set in the prose face it read as part
of the sentence beneath it — and carry created and settled timestamps plus the
engine's own reason string. "Confirmed" and "confirmed last week and untouched since" are
different situations that a status column cannot tell apart. - The scheduling controls are labelled and explained.
pin/suspend/clearwere
three bare verbs in a row and nobody could tell what any of them would do. They are now
Test first / Hold back / Clear, under a heading, with the current directive
stated in words and the standing rule beside them: this changes what the navigator offers,
never what is believed. - The timeline is an activity histogram rather than a form slider. The shape of a run —
where the bursts were, where it stalled — is information, and a featureless track threw all
of it away. The handle travels along the bars. - Transitions run at a single
0.3s easeeverywhere, so a status arriving over SSE eases from
one colour to the next instead of jumping. - The goal
<select>, the panel scrollbar and the zoom controls are all styled; they were
browser defaults sitting inside a designed page.
- Node cards put the id in its own monospaced bar — set in the prose face it read as part
- Wide layers wrap into sub-rows. Five values on each of five axes puts twenty-five nodes on
one line, and a drawing 25 wide by 3 deep fits the viewport at a scale where nothing is
legible. The wrap width is derived from the graph's own size, so the layout is still a pure
function of the topology — same graph, same picture, which is most of what a viewer is for. /hypotree-nextand the server-level instructions now teach the batch shape.- Proof mode is no longer a toggle. What an experiment cost is the reason the panel is
worth reading; hiding it behind a button meant the default view was the less useful one.
Provenance is always shown, and the button is gone. - The left panel can be dragged wider, from its default out to 400 px more, and never
past half the window — a reading panel that eats the graph defeats putting them side by
side. - Zoom lives on the canvas.
+,−andfitsit at the bottom-right of the graph
rather than in the header, next to the thing they act on.
Eval harness
- The exclusion-yield verdict could not report two of its three outcomes. It compared
yield > baseline + 0.02againstyield < baseline + 0.02, which left "at the baseline"
reachable only on exact float equality and described a run performing below chance in the
same words as one performing at it. Now a symmetric band around the baseline, with a verdict
for each side of it. Runs J and K are unaffected — both sit above the band — but the
paragraph is the one the report calls "the only lever on premise cost", so a verdict it
could not actually produce is worse than no verdict. - Group size counted creation events, not members. The arm-B prompt tells the agent to
re-create nodes withif_exists="overwrite", and every re-creation logscreatedagain —
so an agent that rewrote its own nodes inflatedk, which inflates the baseline it is being
judged against, which flatters its own ordering. Counted by distinct node id now. Latent in
J and K (both measured exactly 5.0, the true axis size) and corrected before it was not. - A second conflict stole the first one's diagnosis. Attribution lived in a single slot,
so a conflict opening before the previous one resolved overwrote its mark: the first
resolution was measured from the wrong starting probe and the second was attributed to
nothing at all. Keyed by conflict id now, which the runner had to start logging on
conflict_recorded. Logs written before that id existed fall back to oldest-open-first —
exactly the single-slot behaviour they were read with — so J and K still report the 2.15 and
2.20 they always did. dead_questionprunes are counted as a mechanism of their own in the belief-state table,
apart from the refutation cascade they are deliberately distinct from.RunLogger.log_lease_eventremoved — no caller, and nolease_eventin any log.- Self-play can under-declare a question.
solve_seed(..., omit_winner_on=<axis>)leaves
the winning value off one axis's candidates. Every axis carries its winner by construction,
so a fully-declared question always confirms and the closed-world assumption is never
actually put under load; this is the only way this landscape can do it.
Eval harness — batch recording
- The closed-world check is a gated pre-flight.
eval.shstep 2c withholds one axis's
winning value on every seed and requires at least 80% of those runs to enddead_question
rather thanempty_frontier. Gated on the diagnosis, not on a solve: the winning value was
never offered, so 0 solved is the correct answer and the only thing worth checking is whether
the run says why. Currently 30/30. solve_seed(..., omit_winner_on=<axis>)is how that is constructed. Every axis carries its
winner by construction, so a fully-declared question always confirms and the closed-world
assumption is never actually put under load; withholding one is the only way this landscape can
do it.- Withdrawn deductions are counted in the belief-state table, apart from every other
mechanism. Each one is the engine catching its own assumption being wrong, and each costs
exactly one probe to settle. - One bad result no longer discards the rest of a batch. The harness looped the
single-result call, so a batch containing an unknown node id aborted: the results already
applied stayed in the belief state while the whole call was reported as an error, and every
result after the bad one was never attempted. Each of those is an experiment that has
already been paid for — which is exactly whatengine.record_resultsper-item isolation
exists to prevent, and the harness was silently overriding it. Never fired in run H (zero
record failures), so no result is affected. - Rejected batch items are logged (
record_rejected) and reported. Per-item isolation makes
the containing call succeed, so without this a refusal appeared nowhere: not in the tool
census, which sees an ok call, and not in the evidence count, which never saw the result. seed_readerreports probes to name the culprit. The report said how many conflicts
were narrowed and never what they cost, so the member ordering — the thing that decides
that cost, one probe per position — was invisible. Run G: 1.73. Run H: 2.13. A run
can resolve every conflict and still have got slower, and nothing said so.- A win no longer cuts the episode off mid-protocol. The environment decides the win the
instant a probe clears the target, but the agent is holding an unreported result at that
moment. On the seeds where the swap that identifies a culprit is also the winning
combination, the engine resolved the conflict and the scoreboard recorded it as unresolved,
because the record that proves it was never allowed to land — run G's "11 of 13 conflicts
resolved" was really 13 of 13. The agent now gets a bounded wind-down (2 turns) to file what
it holds. Further experiments are refused and the step counter is frozen, so this cannot
move the headline metric; arms with nothing outstanding wind down in zero turns. - The harness
record_evidenceaccepts and instruments the batch shape, attributing every
belief-revision side effect to the result that caused it rather than to the last in the
batch.seed_readerreports results per record call — 1.00 means the batch shape is not
being used and turns/step cannot fall. - One HTTP error no longer kills a 90-episode sweep.
_call_openai_apiwrappedurlopen
in nothing at all, so a single 5xx from the inference server propagated out ofrun()and,
undereval.sh'sset -e, aborted everything that had not run yet. RunIdied at seed
1210 of 30 and took 61 episodes with it. Transient faults (5xx, 429, connection failures,
read timeouts) are now retried with capped exponential backoff plus jitter — five attempts,
2 s to 60 s — and every retry is logged so a run with odd numbers can be checked against how
much trouble the transport was having. A 4xx other than 429 still fails immediately: a
malformed request does not become well-formed by being sent again. - An episode that dies is excluded from scoring, not censored at budget. Exhausted retries
raiseLLMUnavailableError, which becomes a cleanrun_endcarryinginfra_failed. The
gate drops those episodes from the paired set and the reader names them as transport faults.
Censoring assumes the agent had its full budget and did not solve the task; an episode killed
by a dropped connection had neither, and scoring it at budget charges one arm for the
server's bad minute. eval.shsurvives a failed episode. The runner call no longer aborts the sweep, failed
episodes are collected and reported at the end with the--resumecommand that recovers
them, and the resume check now requires a terminal record and the absence of an infra
marker — otherwise--resumewould skip precisely the episodes that need re-running.seed_readerreports alternatives reopened after a shortfall separately from those
reopened by an interaction effect, and lists infrastructure failures apart from episodes
with no terminal record: one needs the server looked at, the other needs a re-run.- A batch result missing
node_idwas refused with the word'node_id'. That is
str(KeyError('node_id'))— the agent was told a key name, not a contract, and had no way
to work out what to send instead; one run lost a paid-for probe to it. The refusal now
names the missing field and states the shape of a result. exclusions_appliedundercounted re-exclusions. The counter required a status change,
which dropped any sibling retired, released when its justification was withdrawn, and
retired again by a later confirmation._apply_exclusionskips any sibling that is not
open, so arriving back at EXHAUSTED requires having passed through UNTESTED — the guard's
stated rationale (avoiding double-counted re-attribution) had stopped being reachable.seed_readerreports exclusion yield against its blind baseline. Every question is
settled exactly once, by a probe or by the exclusion inference retiring it for free, so the
two are halves of one fixed total and the split is the only lever on premise cost. Probing
a group of k answers in ignorance costs (k+1)/2 and retires the rest, a yield of (k-1)/2k —
40% for k=5. Two runs differing by a probe an episode differed here and nowhere else, and
without the baseline there was no way to tell an ordering win from a lucky draw.eval.sh --skip-arm-aruns B and F only. Arm A is the ergonomic baseline and the most
expensive arm to run — it is the one that duplicates probes — so dropping it roughly halves
the wall clock while iterating. The flag says loudly that criterion 1a is unscorable and the
run cannot decide a gate.
Full Changelog: https://github.com/tygryso/hypotree/blob/master/CHANGELOG.md#040---2026-08-04
Full Commit: https://github.com/tygryso/hypotree/commits/v0.4.0