Skip to content

v0.4.0

Choose a tag to compare

@tygryso tygryso released this 04 Aug 16:13
· 16 commits to master since this release
Immutable release. Only release title and notes can be modified.

v0.4.0

Performance

  • Selecting a target is 41× faster on a large belief state: 1200 nodes went from
    4 494 ms to 110 ms per dispatch, and the curve from ~3.8× to ~2.3× per doubling of
    nodes. None of this was visible at the 40–60 nodes an evaluation episode produces, which
    is why it survived 662 tests — but every long-lived workspace lands on the wrong side of
    it, and "the belief state survives across sessions" is the product claim.
    • The dominant cost was an N+1 query. get_all_nodes derives each node's parent_ids
      from the edges table and was issuing one query per node; the navigator calls it
      several times per dispatch, so handing out two targets cost ~6 000 SQL round-trips.
      Parents are now resolved for every row in a single query.
    • HypoTreeGraph.add_edge ran a full is_directed_acyclic_graph() check per edge, so
      reloading E edges cost O(E·(V+E)). New add_edges_bulk checks the finished set once
      and rolls back if it is cyclic. The per-edge path stays for interactive creation, where
      naming the offending edge is worth the cost.
    • HypoTreeGraph.parents() scanned every edge in the graph and the frontier calls it once
      per node. Parents are now indexed by dst.
    • get_next_targets syncs the in-memory graph once per batch instead of once per pick.
      The batch loop takes leases, which change node rows but never the topology.
    • A long-lived incremental graph — invalidating on write, never rebuilding — is
      deliberately not done. A cached graph that disagrees with SQLite would produce
      silently wrong selections, and at 110 ms there is no case for taking that risk.

Added

  • The closed-world assumption is now declared rather than assumed. Deduction by elimination
    — confirming the last survivor of an exclusion group for free — is sound only if the listed
    candidates really are all of them, and nothing checked that. Withhold one axis's winning
    value and the engine refuted the four that remained and confirmed the survivor with no
    observation of its own
    : a free confirmation of a value that is wrong, recorded as a
    confirmation. The run then ended empty_frontier with the goal unmet and never said the
    question had been incomplete. This is the mechanism the README credits for the moat, and it
    had shipped unguarded for months.
    • exclusion_closed on create_hypotheses (schema 10, nodes.exclusion_closed,
      NOT NULL DEFAULT 1 — exactly what every group written before it meant). Defaults to
      true. Pass false where the next candidate always exists — "which learning rate", "which
      prompt wording", "which threshold" — and the last-one-standing deduction is withheld, along
      with the pruning that rests on it. Confirming a member still retires the others: that is an
      observation about the confirmed value, not a claim about the list.
    • Openness is per-group and one member declaring it is enough. It withdraws an inference,
      so the cautious declaration governs; the reverse would let a later call that forgot the flag
      quietly re-enable deduction over a list its author knew was partial.
    • A deduction can now be withdrawn. When every question is answered and the assembly still
      falls short, the belief with no measurement behind it is the one to doubt. The engine hands
      it back as UNTESTED and the navigator dispatches it, so one probe settles whether the value
      was wrong or the list was incomplete. It never asserts the value false — nothing was ever
      observed about it, and asserting on no evidence is the one thing the belief state may not do.
      The same applies at conviction: a deduction named as a conflict's sole cause is tested rather
      than convicted.
    • _is_deduced reads the whole history, not its last entry. A deduction that passes
      through conflict review comes back marked "released from review", which describes the last
      thing that happened to it and erases where the confirmation came from — a node never once
      observed looked like one that had stood up to scrutiny.
    • dead_question now says which kind. Over a closed group it means the list was wrong or
      one of the eliminations was, and what depended on those values is pruned. Over an open one it
      means the answer is very likely one that was never listed — and nothing is pruned,
      because an untried candidate could still satisfy it.
    • Measured. Across all 30 seeds × 5 axes with the winner withheld, 144/150 (96%) now
      end dead_question naming the incomplete axis, against 0/150 before — every one of which
      was empty_frontier. Still 0 solved, which is correct: the winning value was never offered,
      so the goal is genuinely unreachable, and what changed is that the run says why. A wrong
      answer reported as wrong is a result; the same answer reported as an empty frontier is not.
      Fully-declared self-play is untouched at 30/30, mean 15.9.
  • The dashboard runs by default. It was behind --dashboard while it was unproven, and the
    result was that the people it was built for never saw it — nobody opts into a feature they
    have not been shown. It binds loopback only, mints a fresh token per start, and cannot take
    the MCP server down
    : a bind failure is reported and the server carries on, because the cost
    of a viewer failing must never be the cost of the tooling failing.
    • --dashboard-port PORT picks the first port to try (it still probes upward, so several
      workspaces can be open at once).
    • --no-dashboard opens no socket at all.
    • --no-mcp replaces --dashboard-only and reads better beside its sibling. v0.4.0 is
      unreleased, so the old spelling never shipped and is simply gone.
  • The agent can hand over the dashboard link. The URL is minted at startup and printed to
    stderr, which the model never sees — so an agent asked "where can I watch this?", which is
    constant, had no way to answer. get_workspace_info now returns dashboard_url, and the new
    hypotree://dashboard resource returns it on its own. Both carry the session token, which is
    the whole credential; null means none is running, and the text says how to start one.
  • Backward pruning over a complete question. The exact dual of deduction by elimination,
    and the half of that symmetry that had never shipped: all-but-one candidate eliminated
    confirms the survivor for free, and all of them eliminated says the question has no answer
    among the candidates offered — so nothing that assumes one of them can ever be satisfied.
    Those branches are pruned under their own reason marker and the navigator reports a new DONE
    reason, dead_question, naming the group that ran out and the goal it was holding up.
    • Soundness rests entirely on the exclusion group. "All four approaches I tried failed"
      does not entail "the objective is unreachable" — it entails "I need a fifth idea", and a
      rule that could not tell those apart would let an unimaginative agent declare its own goal
      impossible. What makes this version sound is that the caller declared these to be the
      competing answers to one question.
    • Three guards, each closing a way it could fire on a live search: at least two candidates
      (one value is not a question); every member eliminated on its own evidence, which is
      strictly stronger than the elimination test deduction uses — PRUNED does not count,
      because it is a statement about an ancestor's subtree, and neither does a member with no
      observation of its own; and no open conflict over any member, because conflict diagnosis
      may hand one straight back.
    • It reaches what the refutation cascade deliberately spares. A premise settled as
      EXHAUSTED was never refuted, so its dependents stay on the frontier being re-attempted
      long after the premise ran out; that is the case this exists for.
    • dead_question is an instruction, not an ending: the fix is to add the candidate that was
      never listed, and the eval harness treats it as a reason to continue.
  • record_evidence is batch-native. Pass results=[{node_id, success, depth, claim_id}, …]
    to report every experiment from one turn in one call. create_hypotheses, update_status
    and get_next_targets have all been batch-native for a while; record_evidence was the
    last singular high-frequency tool, and it is what pinned a synchronous agent at two turns
    per experiment — one to probe, one to report. The single-result form still works and is the
    k=1 shorthand.
    • Results are applied in order, so a refutation's cascade lands before the next result
      in the same batch is read and the belief state ends where it would have ended had the
      results arrived separately.
    • A refused report does not abort the batch. Every result was paid for by an experiment
      that already ran, so discarding k-1 of them to punish one bad node_id destroys evidence.
      The refusals come back under failed, named and explained.
    • The fused dispatch runs once, after the whole batch, and is still a top-up.
  • get_conflicts now reports skipped_no_substitute per member.
  • A read-only web dashboard: hypotree --dashboard [PORT] and hypotree --dashboard-only [PORT].
    A belief state that revises itself is hard to appreciate from a status column. The
    dashboard runs on the MCP server's own event loop and serves the graph, the narrative and
    a timeline over localhost HTTP. --dashboard-only starts no MCP server at all — the
    try-before-you-wire path for someone who has not configured a client yet.
    • The interface. Vue 3 over server-computed coordinates, with d3-zoom for
      hardware-accelerated pan and zoom. Untested nodes glow at their real chance of being
      dispatched next; in-progress nodes pulse; new nodes fade in as the agent creates
      them and pruned branches desaturate rather than disappear, because the claim being
      demonstrated is that they were considered and cut. Edges into a dead branch turn red.
    • A timeline scrubber over the bi-temporal history: drag back to any instant, or press
      play and watch the investigation replay.
    • Proof mode overlays what each belief cost — score, depth, commit, source_ref.
    • The learning path is typeset markdown, ready to paste into a report.
    • Everything is vendored (Vue 3, nine d3 micromodules, marked — 276 KB). No CDN, no
      npm, no build step: it works on a plane and in an air-gapped network. d3 micromodules
      rather than the full build (63 KB against 273 KB) because the layout is computed
      server-side and nothing else in d3 is wanted.
    • The observer physically cannot write. Its store is opened mode=ro, so a read path
      that is wrong fails loudly instead of quietly mutating the belief state being watched.
      WAL already permits a second reader, so it never contends for the engine's write lock.
    • Layout is computed server-side with networkx and shipped as coordinates. No
      JavaScript graph library, so nothing large is downloaded and the layout is deterministic
      and testable in Python rather than eyeballed in a browser.
    • p_select is the real selection probability, estimated by repeating the Thompson
      draw and counting argmax wins — not a monotone stand-in like the posterior mean.
    • Time travel is a WHERE clause. ?at=<iso8601> reconstructs any past instant from
      the SCD2 status and posterior intervals, so a replay shows what was believed then.
    • SSE carries a revision number, never a payload. events.seq advances inside every
      mutation's transaction, so it is a correct change signal by construction; clients refetch.
      A subscriber that cannot keep up is dropped rather than allowed to apply backpressure.
    • Everything is cached on the revision, which is also the SSE payload — without it a
      scrubber would rebuild the whole graph per frame.
    • Security. Binds 127.0.0.1 explicitly, mints a session token at startup and requires
      it on every /api/* call, validates the Host header (the DNS-rebinding defence behind
      CVEs in Jupyter, Ray and TensorBoard), refuses cross-origin requests outright, and serves
      the page from a string so path traversal is unrepresentable rather than filtered. The URL
      is printed to stderr — stdout is the JSON-RPC channel.
  • Pin and suspend: human scheduling directives that are not beliefs. A new
    node_directives table records "look at this first" or "leave that alone" with an actor and
    a reason. Directives change what the navigator offers, never what is believed — writing a
    click into alpha/beta would make it forever indistinguishable from an experiment and
    inject unlogged nondeterminism into a seeded sampler. With none set, selection is unchanged.
  • edges.created_at records when an edge appeared, so a timeline replay no longer draws the
    final topology at every tick. Additive; edges written before it are treated as original.
  • A migration adding a column to a table the DDL had just created failed. The DDL
    described the current schema and ran before the migration chain, so a table the file was
    missing got created already-modern and the step adding its column then hit duplicate column name — a collision every future migration would have met, not just this one. The
    schema is now defined the way Doctrine and Rails define one: BASE_DDL is the original
    shape, frozen, CREATE TABLE IF NOT EXISTS throughout, and every change since is a
    numbered step applied in order. A fresh database is built exactly as an old one is
    upgraded, so the upgrade path is exercised on every install rather than only on other
    people's machines. SCHEMA_VERSION is derived from the last step and cannot drift from it.
  • New schema_state table: exactly one row, enforced by CHECK (id = 1). Carries the
    schema version, the hypotree release that last opened the file, and when. It is the first
    thing to look at when a database behaves unexpectedly or a bug report arrives with one
    attached. The pre-0.4.0 schema_meta key is still read as a fallback and kept in step for
    one minor version. A freshly created database logs no migration events — it was created,
    not upgraded.
  • The dashboard did not run in a browser. default-src 'self' blocked Vue's global build
    compiling its in-DOM template with new Function, blocked the inline script carrying the
    session token, and blocked data: fonts injected by browser extensions. Every directive is
    now named explicitly rather than leaning on the fallback, and the token travels in a
    <meta> tag so script-src can keep refusing 'unsafe-inline' — the directive that stops
    an injected event handler from ever running.
  • The learning path rendered untrusted text through v-html. Node statements are written
    by an agent, and marked has had no sanitiser since v5, so markup in a statement reached
    the document verbatim. An allowlist sanitiser now runs over the parsed output; the strict
    script-src covers what a sanitiser bug would miss, and vice versa.
  • Work on one objective at a time: goal_id on get_next_targets,
    generate_learning_path and get_goal_status.
    A workspace pursuing several goals
    interleaves them — the frontier offers whatever is most uncertain anywhere, and the
    learning path narrates every goal's dead ends into one story. A goal_id restricts all
    three to the case for one objective: that goal, everything it depends on, and the
    competing answers to those questions. Omitting it is the previous behaviour exactly,
    down to selecting the identical node on an identically-seeded engine.
    • Scope is DEPENDENCY-ancestry plus the exclusion-group siblings of that ancestry,
      not graph reachability. A competing answer is not an ancestor of the goal, but testing
      it is how the engine learns whether the answer that is an ancestor holds — a scope
      without it hands the navigator questions it is forbidden from answering.
    • New DONE reason goal_scope_empty. Agents create premises before wiring them
      (awaiting_composition exists because they do), and unwired nodes are outside every
      goal's scope by construction. Rather than reporting a bare empty frontier — which reads
      as "the search is over" — it names how many untested hypotheses are outside the filter
      and says the fix is usually a missing DEPENDENCY edge.

Fixed

  • The live stream died every fifteen seconds on Python 3.10. asyncio.TimeoutError only
    became an alias of the builtin TimeoutError in 3.11, so the keepalive's except TimeoutError
    caught nothing on the project's own minimum version: the first quiet interval killed the SSE
    task, the browser reconnected, and the cycle repeated. It presented as a dashboard that
    flickered between "live" and "reconnecting" for no reason, which reads as a flaky network
    rather than as a bug. Caught as asyncio.TimeoutError, which is correct on both.

  • A goal can no longer be given as a DEPENDENCY parent. This is the one modelling mistake
    strong models make reliably: "the goal decomposes into phases" is how everyone thinks about
    objectives, so parent_ids=[goal] on the phase reads as "this phase belongs to that goal". In
    hypotree it means the opposite — the phase cannot be tested until the goal is VERIFIED — and
    both halves then break silently. A goal is never dispatched and verify_upstream only promotes
    IN_PROGRESS nodes along REFINEMENT edges, so the goal can never reach VERIFIED and the child is
    blocked forever, while the goal still depends on nothing and so can never be reached either.
    The engine had been handling this downstream, reporting the wreckage as blocked_frontier;
    it is now refused upstream with a GoalDependencyError carrying the corrected call. Only
    DEPENDENCY is refused, because only DEPENDENCY gates a child on its parent.

    • A real behaviour change: five existing tests were constructing exactly this state to exercise
      the downstream reporting. They now build the same situations from ordinary nodes —
      blocked_frontier from premises wired under an EXHAUSTED combination, which settles without
      cascading — which is the general case and the one still representable.
    • Databases written before this release keep the shape and nothing is rewritten. The engine may
      change what it asserts only from an observation or a sound inference over one, and silently
      flipping an edge someone declared is neither. The dashboard names such a goal's wiring as
      inverted rather than merely unwired, because "turn your edges around" and "attach some
      work" are different instructions.
    • parent_ids is described as what this rests on at every point of use, and the guide states
      the direction before it states the edge types.
  • Selecting a goal hid the tree hanging off it. Agents routinely wire a goal the other way
    round — goal -> phase0 -> work, reading the goal as a container rather than as something the
    work supports. The navigator's goal scope is its DEPENDENCY ancestry, which is the right set
    to dispatch from and the wrong set to draw: filtering to such a goal showed a single dot beside
    a lecture about wiring while a nine-node tree hung off the node in question. A viewer that
    hides your work to make a point about how you built it is not making the point. The dashboard
    now walks both directions — the selection scope plus everything downstream plus the
    exclusion siblings of the result — and reports the unreachability in a dismissible banner over
    the canvas instead of in place of the graph. The navigator's scope is deliberately unchanged:
    it must not start selecting downstream of a goal just because the viewer draws it. The timeline
    follows the drawing for the same reason the graph does — the scrubber moves the picture, so a
    tick it cannot show is a step that does nothing.

  • The left panel flashed on every node click. Selecting a node cleared detail before
    fetching it, which mounted the entire narrative for one frame before replacing it again. The
    panel now branches on what is selected rather than on what has arrived, so the card
    renders immediately and fills in. A slow response for a node the reader has already moved off
    no longer overwrites the one they are looking at.

  • A directive reloaded the whole view. Pinning a node rebuilt the narrative and the layout
    to change one badge; it now refreshes the node, the graph and the frontier, which are the only
    things a directive can move.

  • Binary assets were dropped from the manifest. The table only carried .js and .css, so
    the logo would have 404'd — a broken header rather than a missing feature. PNG and SVG are
    served with the right content type and ship in the wheel.

  • A dry run wrote to the belief state. get_next_targets(dry_run=True) is documented as a
    peek and issues no claim, but when a conflict's targeted narrowing had run out of advice the
    peek performed the fallback: it marked the conflict recovered, released its members from
    review and reopened every alternative they had retired. A caller asking what it would be
    given settled the question it was asking about, and the next real call saw a belief state a
    peek had moved. The sibling fallback on the same path was already guarded, so this was an
    inconsistency rather than a design decision. The peek now reports the instruction it would
    still give and changes nothing.

  • The conflict fallback timestamped itself off the wall clock rather than the instant the
    selection pass was running at, so the status changes it wrote landed outside the interval
    the rest of that pass recorded — a small tear in a history whose whole value is that it can
    be read back.

  • The dashboard's own buttons were refused as cross-origin. The write path rejected any
    request carrying an Origin header. Browsers attach Origin to same-origin JSON POSTs
    too, so pin, suspend, back and clear each came back 403 cross-origin requests are refused
    — the CSRF defence was locking the page out of itself. The header is now validated against
    the bound address instead of being treated as evidence of an attack: a foreign origin, or
    the right host on a different port, is still refused, and an absent header still falls
    through to the token and Host checks.

  • The narrative ignored the time-travel scrubber. Rewinding the graph left the learning
    path describing conclusions the picture had not reached, which is worse than showing
    nothing. generate_learning_path takes as_of and /api/learning-path takes at, so the
    briefing stops where the graph stops. Observation tallies are counted at that instant too,
    via the new HypoTreeStore.count_evidence_by_node(before=...) — one aggregate that also
    replaces the per-node cached counts on the live path. The rewound render says which line
    still reflects the present, because goal achievement is read off the live graph.

  • The timeline handle sat at the far left while showing the present. The scrubber ran from
    -1, and -1 meant live — so watching the current state pinned the handle to the start of
    history and playback had nowhere to run. The present is now the right-hand end: the handle
    rides it as new history arrives, live returns to it, and play pressed at the present
    rewinds to the beginning first.

  • A member the diagnosis could not test was reported as cleared. Substitution progress was
    an integer cursor, which can only say "the first k were dealt with". A member the plan had
    to pass over — because its question had no live alternative left to swap in — sat behind
    the cursor the moment a later member was cleared, and was reported as exonerated by an
    experiment that never ran. It could also never be revisited if an alternative freed up
    later. Progress is now a persisted set of cleared ids: cleared and skipped are recorded
    as the opposite claims they are, and the plan reconsiders a skipped member on every call.
    Schema "9" → "10" adds nogoods.cleared_ids; the migration is additive and the set is
    back-filled from the old cursor on read, so a diagnosis in flight across the upgrade keeps
    the members it had genuinely cleared. probe_index is kept in step and deprecated;
    HypoTreeStore.advance_nogood_probe warns and will be removed in 0.5.0.

  • Schema versions were ordered as strings. "9" > "10" and even "3" > "10" are true
    lexicographically, so the first two-digit schema version made a database with a corrupt or
    older stamp report itself as "written by a newer hypotree" — sending the user to upgrade
    the package instead of keeping the file. Ordered numerically now; a stamp that is not a
    number is not treated as newer, which routes it to the "back it up and open an issue"
    branch where it belongs.

  • A success could resurrect a pruned branch. Recording a passing result on a PRUNED node
    flipped it to VERIFIED, leaving the belief state asserting both that the branch is dead
    and that it holds — a confirmed node sitting on an invalidated parent. A pruned node's
    status is a statement about its ancestry, so the measurement and the posterior are kept and
    the status is left alone. Rare while probe and record alternated in fixed pairs; routine
    once results arrive in batches, because one refutation in a batch can prune the node a
    later result in the same batch belongs to.

  • source_ref was advertised and dropped. 0.3.2 added the field, documented it on the
    record_evidence tool schema, and never read it in the dispatch — so every value sent over
    MCP was silently discarded. The single and batch paths now build evidence through one
    helper, so they cannot drift apart again.

  • The shortfall recovery fired on a search that had already won. When every question is
    answered and the assembled answer falls short, the engine hands back the alternatives those
    answers had retired — the reading being that one of the confirmations must be wrong. It did
    not check whether a different assembly had since cleared the bar, and a composition that
    succeeded withdraws exactly that reading: the answers were right, the earlier assembly was
    simply the wrong combination of them. One real episode reopened six settled questions on the
    turn it found its answer, because its goal node had never been wired to the winning
    composition and an empty frontier looks the same from the inside either way. The recovery
    now stands down once any composition is VERIFIED.

  • Two different recoveries wrote the same reason marker. _recover_from_underperformance
    reused INTERACTION_REOPEN_PREFIX, so a history could not distinguish "a conflict turned
    out to be an interaction effect" from "every answer was in and the assembly still missed" —
    and a run report claimed every conflict had been narrowed to a culprit and that six
    alternatives had been reopened because a conflict was an interaction effect, which are
    mutually exclusive endings. New UNDERPERFORMANCE_REOPEN_PREFIX. Additive: histories written
    before 0.4.0 keep the old prefix and still read as interaction reopens, which is the only
    label that ever existed for them.

Changed

  • The dashboard has been redesigned as a light analytical interface. The previous dark theme
    was hard to read for long and, more to the point, spent colour on chrome — so status, the one
    thing colour should carry, had to compete with it. Now warm paper (#FAFAFA), a gridded
    canvas that pans with the graph, white cards on soft shadows instead of heavy borders, and a
    single navy/cyan accent reused for goals, focus and selection so the eye learns one colour
    rather than three. Every status has a pastel badge; PRUNED adds a diagonal hatch, because
    grey alone reads as "not started" rather than "cut". No webfont is vendored: font-src 'self'
    would mean shipping a ~90 KB binary in the wheel to displace a system stack that is
    Inter-class on every current OS.
    • Node cards put the id in its own monospaced bar — set in the prose face it read as part
      of the sentence beneath it — and carry created and settled timestamps plus the
      engine's own reason string. "Confirmed" and "confirmed last week and untouched since" are
      different situations that a status column cannot tell apart.
    • The scheduling controls are labelled and explained. pin / suspend / clear were
      three bare verbs in a row and nobody could tell what any of them would do. They are now
      Test first / Hold back / Clear, under a heading, with the current directive
      stated in words and the standing rule beside them: this changes what the navigator offers,
      never what is believed.
    • The timeline is an activity histogram rather than a form slider. The shape of a run —
      where the bursts were, where it stalled — is information, and a featureless track threw all
      of it away. The handle travels along the bars.
    • Transitions run at a single 0.3s ease everywhere, so a status arriving over SSE eases from
      one colour to the next instead of jumping.
    • The goal <select>, the panel scrollbar and the zoom controls are all styled; they were
      browser defaults sitting inside a designed page.
  • Wide layers wrap into sub-rows. Five values on each of five axes puts twenty-five nodes on
    one line, and a drawing 25 wide by 3 deep fits the viewport at a scale where nothing is
    legible. The wrap width is derived from the graph's own size, so the layout is still a pure
    function of the topology — same graph, same picture, which is most of what a viewer is for.
  • /hypotree-next and the server-level instructions now teach the batch shape.
  • Proof mode is no longer a toggle. What an experiment cost is the reason the panel is
    worth reading; hiding it behind a button meant the default view was the less useful one.
    Provenance is always shown, and the button is gone.
  • The left panel can be dragged wider, from its default out to 400 px more, and never
    past half the window — a reading panel that eats the graph defeats putting them side by
    side.
  • Zoom lives on the canvas. +, − and fit sit at the bottom-right of the graph
    rather than in the header, next to the thing they act on.

Eval harness

  • The exclusion-yield verdict could not report two of its three outcomes. It compared
    yield > baseline + 0.02 against yield < baseline + 0.02, which left "at the baseline"
    reachable only on exact float equality and described a run performing below chance in the
    same words as one performing at it. Now a symmetric band around the baseline, with a verdict
    for each side of it. Runs J and K are unaffected — both sit above the band — but the
    paragraph is the one the report calls "the only lever on premise cost", so a verdict it
    could not actually produce is worse than no verdict.
  • Group size counted creation events, not members. The arm-B prompt tells the agent to
    re-create nodes with if_exists="overwrite", and every re-creation logs created again —
    so an agent that rewrote its own nodes inflated k, which inflates the baseline it is being
    judged against, which flatters its own ordering. Counted by distinct node id now. Latent in
    J and K (both measured exactly 5.0, the true axis size) and corrected before it was not.
  • A second conflict stole the first one's diagnosis. Attribution lived in a single slot,
    so a conflict opening before the previous one resolved overwrote its mark: the first
    resolution was measured from the wrong starting probe and the second was attributed to
    nothing at all. Keyed by conflict id now, which the runner had to start logging on
    conflict_recorded. Logs written before that id existed fall back to oldest-open-first —
    exactly the single-slot behaviour they were read with — so J and K still report the 2.15 and
    2.20 they always did.
  • dead_question prunes are counted as a mechanism of their own in the belief-state table,
    apart from the refutation cascade they are deliberately distinct from.
  • RunLogger.log_lease_event removed — no caller, and no lease_event in any log.
  • Self-play can under-declare a question. solve_seed(..., omit_winner_on=<axis>) leaves
    the winning value off one axis's candidates. Every axis carries its winner by construction,
    so a fully-declared question always confirms and the closed-world assumption is never
    actually put under load; this is the only way this landscape can do it.

Eval harness — batch recording

  • The closed-world check is a gated pre-flight. eval.sh step 2c withholds one axis's
    winning value on every seed and requires at least 80% of those runs to end dead_question
    rather than empty_frontier. Gated on the diagnosis, not on a solve: the winning value was
    never offered, so 0 solved is the correct answer and the only thing worth checking is whether
    the run says why. Currently 30/30.
  • solve_seed(..., omit_winner_on=<axis>) is how that is constructed. Every axis carries its
    winner by construction, so a fully-declared question always confirms and the closed-world
    assumption is never actually put under load; withholding one is the only way this landscape can
    do it.
  • Withdrawn deductions are counted in the belief-state table, apart from every other
    mechanism. Each one is the engine catching its own assumption being wrong, and each costs
    exactly one probe to settle.
  • One bad result no longer discards the rest of a batch. The harness looped the
    single-result call, so a batch containing an unknown node id aborted: the results already
    applied stayed in the belief state while the whole call was reported as an error, and every
    result after the bad one was never attempted. Each of those is an experiment that has
    already been paid for — which is exactly what engine.record_results per-item isolation
    exists to prevent, and the harness was silently overriding it. Never fired in run H (zero
    record failures), so no result is affected.
  • Rejected batch items are logged (record_rejected) and reported. Per-item isolation makes
    the containing call succeed, so without this a refusal appeared nowhere: not in the tool
    census, which sees an ok call, and not in the evidence count, which never saw the result.
  • seed_reader reports probes to name the culprit. The report said how many conflicts
    were narrowed and never what they cost, so the member ordering — the thing that decides
    that cost, one probe per position — was invisible. Run G: 1.73. Run H: 2.13. A run
    can resolve every conflict and still have got slower, and nothing said so.
  • A win no longer cuts the episode off mid-protocol. The environment decides the win the
    instant a probe clears the target, but the agent is holding an unreported result at that
    moment. On the seeds where the swap that identifies a culprit is also the winning
    combination, the engine resolved the conflict and the scoreboard recorded it as unresolved,
    because the record that proves it was never allowed to land — run G's "11 of 13 conflicts
    resolved" was really 13 of 13. The agent now gets a bounded wind-down (2 turns) to file what
    it holds. Further experiments are refused and the step counter is frozen, so this cannot
    move the headline metric; arms with nothing outstanding wind down in zero turns.
  • The harness record_evidence accepts and instruments the batch shape, attributing every
    belief-revision side effect to the result that caused it rather than to the last in the
    batch. seed_reader reports results per record call — 1.00 means the batch shape is not
    being used and turns/step cannot fall.
  • One HTTP error no longer kills a 90-episode sweep. _call_openai_api wrapped urlopen
    in nothing at all, so a single 5xx from the inference server propagated out of run() and,
    under eval.sh's set -e, aborted everything that had not run yet. Run I died at seed
    1210 of 30 and took 61 episodes with it. Transient faults (5xx, 429, connection failures,
    read timeouts) are now retried with capped exponential backoff plus jitter — five attempts,
    2 s to 60 s — and every retry is logged so a run with odd numbers can be checked against how
    much trouble the transport was having. A 4xx other than 429 still fails immediately: a
    malformed request does not become well-formed by being sent again.
  • An episode that dies is excluded from scoring, not censored at budget. Exhausted retries
    raise LLMUnavailableError, which becomes a clean run_end carrying infra_failed. The
    gate drops those episodes from the paired set and the reader names them as transport faults.
    Censoring assumes the agent had its full budget and did not solve the task; an episode killed
    by a dropped connection had neither, and scoring it at budget charges one arm for the
    server's bad minute.
  • eval.sh survives a failed episode. The runner call no longer aborts the sweep, failed
    episodes are collected and reported at the end with the --resume command that recovers
    them, and the resume check now requires a terminal record and the absence of an infra
    marker — otherwise --resume would skip precisely the episodes that need re-running.
  • seed_reader reports alternatives reopened after a shortfall separately from those
    reopened by an interaction effect, and lists infrastructure failures apart from episodes
    with no terminal record: one needs the server looked at, the other needs a re-run.
  • A batch result missing node_id was refused with the word 'node_id'. That is
    str(KeyError('node_id')) — the agent was told a key name, not a contract, and had no way
    to work out what to send instead; one run lost a paid-for probe to it. The refusal now
    names the missing field and states the shape of a result.
  • exclusions_applied undercounted re-exclusions. The counter required a status change,
    which dropped any sibling retired, released when its justification was withdrawn, and
    retired again by a later confirmation. _apply_exclusion skips any sibling that is not
    open, so arriving back at EXHAUSTED requires having passed through UNTESTED — the guard's
    stated rationale (avoiding double-counted re-attribution) had stopped being reachable.
  • seed_reader reports exclusion yield against its blind baseline. Every question is
    settled exactly once, by a probe or by the exclusion inference retiring it for free, so the
    two are halves of one fixed total and the split is the only lever on premise cost. Probing
    a group of k answers in ignorance costs (k+1)/2 and retires the rest, a yield of (k-1)/2k —
    40% for k=5. Two runs differing by a probe an episode differed here and nowhere else, and
    without the baseline there was no way to tell an ordering win from a lucky draw.
  • eval.sh --skip-arm-a runs B and F only. Arm A is the ergonomic baseline and the most
    expensive arm to run — it is the one that duplicates probes — so dropping it roughly halves
    the wall clock while iterating. The flag says loudly that criterion 1a is unscorable and the
    run cannot decide a gate.

Full Changelog: https://github.com/tygryso/hypotree/blob/master/CHANGELOG.md#040---2026-08-04
Full Commit: https://github.com/tygryso/hypotree/commits/v0.4.0