Phase 13: model-driven work-routing + earned backend selection#58
Merged
Conversation
NilCore's North Star puts coding-fluency judgment in the model, but a >=40-word /
8-keyword string heuristic (chatShouldSupervise) was OVERRULING the frontier-model
classifier that already proposes the route. This stops the overrule.
internal/session/router.go:
- reconcile() now returns a PARSEABLE classifier work-route as-is (removed the
heuristic upgrade native->supervise / downgrade supervise|project->native arms).
The string heuristic survives ONLY as the no-classifier / unparseable-output
fallback (fallback() unchanged). An optional, default-OFF one-directional
ClampDownToNative backstop is the sanctioned operator lever (inert by default).
- classifierSys gains a capability + COST manifest: native = cheapest single loop;
supervise = bounded fan-out (up to the operator caps), verifier-judged; project =
supervised loop under an outer budget/deadline. The model sizes by work, not
wording, and picks the cheapest honest route. Output contract {route,reason} and
the 256-token cap unchanged; routing still logged metadata-only (route+reason_len,
I5/I7); RouteContinue stays local (no model call).
- router_test.go: the three overrule cases now assert the classifier proposal wins
(short no-keyword "rewrite the auth subsystem" -> RouteSupervise; 40 words of
trivial chatter -> RouteNative); fallback + clamp-backstop covered.
cmd/nilcore/main.go: new -auto-supervise (default false) gives `nilcore run` the
optional supervised seam (wireAutoSupervise reuses the build path's supervisor +
project loop + caps verbatim; trigger is the same model classifier when a native
provider exists, else chatShouldSupervise). OFF => Project/ShouldSupervise stay nil,
`nilcore run` byte-identical (proven by TestWireAutoSuperviseDefaultOff).
No blast-radius bound touched: caps, budget wall, verifier (I2) all unchanged; the
verifier still judges every route. stdlib only; contract files untouched.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…assume native)
Stops treating native as the privileged default. Neither codex nor claude-code is
better a priori — so let the operator state a preference and let the SYSTEM pick the
best AVAILABLE backend, seeded by that preference and learned from verifier-judged
evidence (the Trust Ledger). Default-off byte-identical.
internal/onboard: Config gains `preferred_backend` (the durable prior) and Backend now
also accepts "auto" (the durable "always decide for me"). Validate(): Backend ∈
{native,codex,claude-code,auto}; PreferredBackend ∈ {native,codex,claude-code} (auto
rejected — a preference must name a concrete backend). wizard onPath → exported OnPath.
cmd/nilcore:
- availableBackends(cfg,cred): the subset usable on THIS host — native iff an executor
model + its key resolve; codex iff `codex` on PATH + CODEX_API_KEY; claude-code iff
`claude` on PATH + ANTHROPIC_API_KEY (creds via the SecretStore seam — I3).
- resolveAutoBackend: orders available PREFERENCE-FIRST (-prefer-backend > cfg
> native) then by trust.Replay(log).Order() — the existing ledger ranks earned
backends ahead and PRESERVES the preferred-first order among unproven ones, so the
cold-start needs ZERO trust-leaf change; evidence overtakes preference as races
accrue. Broken chain ⇒ logs trust_replay_error + keeps preferred order (fail-soft,
never aborts). Emits a metadata-only `backend_auto` event (names only). Empty avail
⇒ a clear `nilcore init` error, not a panic.
- `-backend auto` resolves to one best backend before resolveProvider (runMain +
serve + buildRunOrchestrator, so config backend:auto flows everywhere). `-backends
auto` expands to ALL available (ledger-ordered competition), collapsing to the
single path on a native-only host. New `-prefer-backend` flag.
I2: preference/ledger only ORDER which backend RUNS; the verifier still judges every
race + the final gate (internal/trust + the verifier untouched). No auto + no
preference ⇒ exact current behavior (TestRunDefaultNoAutoIsByteIdentical). stdlib only;
backend.go (I1) + go.mod untouched. make verify green; golangci-lint 0.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adversarial review came back CLEAN on I2 / default-off / I3 (all refuted); these
close the confirmed quality findings (1 medium + 3 low, all on the opt-in paths).
- MEDIUM: availableBackends' native probe was STRICTER than resolveProvider("native")
— it keyed only off cfg.Executor and ignored NILCORE_MODEL + the built-in default
model, so `-backend auto` could wrongly exclude a native backend that `nilcore run`
resolves fine (spurious "no usable backend" or a delegated mis-pick on a
NILCORE_MODEL-only / default-model host). Now resolves the SAME spec
(modelSpec(NILCORE_MODEL, cfg.Executor)) and gates on its provider key — the two
paths agree. + regression tests (default-model + NILCORE_MODEL cases) and a TestMain
that unsets NILCORE_MODEL so the probe is host-deterministic.
- LOW: -prefer-backend typo was validated only in run/serve, not the
buildRunOrchestrator commands (watch/schedule/propose-edit) — a typo was silently
dropped. Validation moved into preferredBackendName, the single chokepoint every
auto path flows through, so it fails loudly + uniformly.
- LOW: -auto-supervise + a delegated backend (codex/claude-code) with no native key
fataled mid-run when a goal sized complex. The supervised seam is an enhancement,
never required, so it now logs auto_supervise_skipped + continues as a single-task
run instead of aborting.
- LOW: corrected chatShouldSupervise's stale doc comment (the router no longer
upgrades/downgrades the classifier — the heuristic is fallback-only now).
I2/I3 intact (verifier still sole authority; events name-only). make verify green;
golangci-lint 0; stdlib only.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
README (auto/preferred backend + classifier-decides-routing + -auto-supervise), CHANGELOG [Unreleased] entry, a minimal ARCHITECTURE note on the work-route authority + auto-backend seam, and the CONVERSATIONAL §3.4 router (classifier authoritative, heuristic = fallback). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two fixes, one philosophy — don't let the harness hard-code a judgment the model/evidence should own, but keep the deterministic bounds. Adversarially reviewed (CLEAN on I2 / default-off / I3); the confirmed quality findings are fixed.
Fix A — the model classifier is authoritative for work-routing
NilCore's North Star puts coding-fluency judgment in the model, but a
≥40-word / 8-keywordstring function (chatShouldSupervise) was overruling the frontier-model classifier that already proposes the route.internal/session/router.goreconcile()now returns the classifier's parseable route as-is; the heuristic survives only as the no-model / unparseable-output fallback (plus an OFF-by-defaultClampDownToNativebackstop). The classifier prompt gained a capability + cost manifest so it sizes by work, not wording. New-auto-supervise(default off) gives plainnilcore runthe optional supervised seam (same caps asnilcore build; trigger is the same classifier when a native provider exists, else the heuristic). Default-off byte-identical; the verifier still judges every route (I2); blast-radius caps untouched.Fix B — earn backend strength, don't assume native
New
preferred_backendconfig +-prefer-backendflag, and-backend auto/-backends auto/ configbackend: auto: the system picks the best available backend (native / codex / claude-code — those whose CLI + key are present), ordered preference-first then by the verifier-judged Trust Ledger (trust.Replay → Order). A fresh install honors the stated preference; evidence overtakes it as races accrue — the ledger needs zero change becauseOrderalready preserves input order among unproven backends. Per-backend creds via the SecretStore seam (I3); the ledger/preference only order which backend runs — the verifier still judges (I2). Default-off byte-identical.Review fixes (1 medium + 3 low, all opt-in paths)
availableBackendsnative probe now mirrorsresolveProvider("native")exactly (honorsNILCORE_MODEL+ the default model) so-backend autocan't wrongly exclude a usable native backend (+ regression tests + a hermeticTestMain).-prefer-backendvalidation moved to the single chokepoint (preferredBackendName) so every auto path rejects a typo uniformly.-auto-supervise+ a delegated backend with no native key now skips the seam gracefully instead of fataling mid-run.chatShouldSupervisedoc comment.Gate
make verifygreen ·go test -racegreen ·golangci-lint run ./...0 issues ·internal/backendcontract +go.mod/go.sum+internal/trust/internal/agentuntouched (I1/I6) · default binary behavior byte-identical without the new flags. Built in an isolated worktree offorigin/main— contains zero of the concurrent browser session's WIP.🤖 Generated with Claude Code