Skip to content

v1.108.241 - offloadable-work annotation, off by default

Choose a tag to compare

View all tags
@jgravelle jgravelle released this 04 Aug 21:51
v1.108.241
d461674

get_symbol_source can now tell you whether the work its payload enables is
grunt-work a cheaper model can do. Set JMUNCH_OFFLOADABLE=1 (suite-wide) or
JCODEMUNCH_OFFLOADABLE=1 (this server alone — the narrower scope wins) and
every reply carries an advisory _meta.offloadable block. Off by default:
no gate, no field, no cost.

We label. We never route, execute, or hold model credentials.

No new process, no network call, no new catalog action, and no model of ours
ever runs. What to do with the label is entirely the client's decision.

Every modelmaxxing router available today classifies the prompt, because
the prompt is all a router standing in front of the model can see. This sits
downstream of retrieval and classifies the evidence we just assembled —
whether the answer is literally present in the payload, how many files it
spans, whether anything was truncated, and whether any freshness or coverage
signal came back unknown.

"offloadable": {
  "offloadable": "offloadable",
  "qualifiers": ["EXTRACTIVE", "SINGLE_CONTAINER", "IDENTITY_LOOKUP", "NO_ABSENCE_FLAGS", "SELF_CONTAINED"],
  "disqualifiers": [],
  "shape": {"units": 1, "containers": 1},
  "verify_with": {"tool": "check_references", "args": {"identifier": "compute_confidence"}},
  "criterion_version": 2
}

Tri-state and reason-coded, never a bare score. not_evaluated is not
not_offloadable — "we did not assess it" and "this is not grunt-work" are
different facts. The criterion fails closed: every unknown bearing on the
answer disqualifies, because a false offloadable sends real work to a model
that will confabulate over the gap, while a false not_offloadable costs
nothing but a missed saving.

verify_with names the call that would adjudicate a cheaper model's answer
over this payload. An annotation you cannot check is a vendor assertion; this
one ships next to the tool that checks it.

Measured before it shipped, with its limits stated

Three pinned corpora — this repository, fastapi/fastapi at a64dfbbd, and
django/django — using the harness in benchmarks/offload/.

  • The falsifier checks a necessary condition, never a sufficient one. It
    verifies the ground truth was present in the payload; it cannot tell you a
    cheap model will answer correctly. A pass is an upper bound on achievable
    accuracy and must not be read as an accuracy figure.
  • On an index whose freshness cannot be established, every payload is
    refused.     Django's index carries no source root, and all 300 sampled
    identity lookups gated on TRI_STATE_UNKNOWN. That is the fail-closed rule
    working, not a defect.
  • The batch arm's rate is a floor, not a representative rate: it
    deliberately scatters its sample across many files.

Suite contract

Identical field contract in jdocmunch-mcp 1.123.0 (sections/documents) and
jdatamunch-mcp 1.31.0 (columns/datasets). EvidenceShape speaks units
and containers, never symbols and files, and a pinned CONTRACT_DIGEST plus
a generated contract test fails the build in any of the three that drifts.

Additive: one new _meta key, emitted only when gated on. No tool, schema or
INDEX_VERSION change. Tests: 84 + 23. Suite 6944 passed / 7 skipped.

Assets 10
Loading