Skip to content

Releases: Mechanism-world/upshift

v0.5.0 — the reliability release

Choose a tag to compare

@atilavahedian atilavahedian released this 08 Sep 00:31
7699de1

v0.5.0 — 2026-09-07

The reliability release. Everything below came out of running upshift against real
migration incidents (the private rescue campaign: 109 OpenAI-track and 54 Anthropic-track
cases) and fixing what broke, with a regression test per finding. Headline changes:

  • Verification scope is explicit everywhere (request_contract / adapted_agent /
    native_application); no output claims application-level verification unless the
    application ran.
  • Native runner: an agent.json runner block lets the application run itself (Python and
    Node reference runners, protocol v1, authorization gate, minimal environment).
  • upshift verify-patch: the exact exported patch is applied to a clean copy and every
    case's first request is rebuilt and compared with the run that verified it.
  • Honest verdicts: INCONCLUSIVE with reason codes; collateral protection measured and
    reported (never assumed); fresh final verification separate from candidate selection;
    evidence identity that refuses stale reuse; SAFE wording says what was measured.
  • Endpoint translation is a tested table: reasoning, output caps, tool choice, sampling,
    seed, state-linking, response_format; every drop is recorded; local SDK failures are
    sdk_validation, never a manufactured HTTP status; repairs that change a capability or
    cost carry a disclosure and rank last.
  • Repair loop: every sibling candidate is screened before one is accepted; adjudication
    cannot be skipped; transient provider errors are retried and never counted as behaviour.
  • Capture: continuation policy, machine-readable unsupported-fields findings.
  • Docs: the real data flow (what reaches the provider, what adapt sends to the
    extraction model), a capabilities table with three evidence levels, a provider matrix
    with unverified rows marked.

Note: --budget (max repair candidates) now defaults to 24 because sibling screening
counts every candidate screened; use --max-cost-usd to bound spend.

Reliability follow-ups

Five findings from the OpenAI and Anthropic rescue tracks, each of which produced a wrong or
unsupported ANSWER rather than a crash. Sources are the private ops repo's case files.

  • The repair loop screens every sibling candidate before it accepts one (ghisdk-127,
    ghc-062). Greedy acceptance let the playbook's emission order decide the verdict: waku
    accepted a candidate that restored 8 of 9 cases and published STAY PINNED while a sibling in
    the same generation restored all nine, and crispen shipped a repair with a disclosed change
    of guarantee ahead of the maintainer's own equivalent fix. Restorers are now ranked by full
    restoration, then cases restored, then the absence of a disclosure, then the playbook rank,
    and verified in that order. Screening costs one screen run per sibling and --budget still
    bounds the candidates TRIED, so its default is 24 rather than 6.
  • A candidate whose adjudication cannot run is rejected, not accepted (ghisdk-052). When
    the cost ceiling stops the N reps that settle a contested case, the candidate is rejected,
    the suspects are recorded as adjudication_skipped, and the verdict is
    INCONCLUSIVE(cost_ceiling) — never SAFE WITH PATCH on the strength of the sample that raised
    the suspicion.
  • Transient provider failures are retried, and never recorded as behaviour (ghi56-019,
    ghi56-006, ghi56-021). A flex capacity 429 was written down three times as a permanently
    failing rep, moving the pass rate the verdict is computed from. 429/5xx/timeouts/connection
    errors are now retried per rep (3 attempts, jittered exponential backoff, ≤90s) outside the
    SDK's own retries; an unrecovered one is recorded as transient_provider_error, which the
    differ files under harness_error and the verdict treats as INCONCLUSIVE. Billing still
    aborts, auth is not retried, and a 400 is never retried or reclassified. New
    --retry-errored on run/upgrade re-runs, on a resume, exactly the reps whose
    recorded error was non-behavioural.
  • response_format is carried, not deleted (ghisdk-051). adapt used to drop it, which
    removed the entire subject of a structured-output incident and produced an agent that asked
    for no structured output at all. It is now a param, translated per endpoint: verbatim on
    chat/completions, text.format (with the json_schema object flattened) on /v1/responses,
    dropped with a recorded reason on Anthropic's messages. A nullable tool parameter also keeps
    its null branch (ghisdk-052): every spelling is canonicalised to anyOf with null.
  • A disclosed strict-schema repair (ghisdk-051, p2-001). New signature
    api_error_schema_invalid and new candidate schema-strict-compat:<schema>, applying
    OpenAI's three documented strict rules to one named schema. Disclosed: optional fields become
    required-and-nullable, so downstream code must accept null.

Capture fixes (previously v0.4.1-dev)

Three fixes found by running capture mode on its first real case outside pydantic-ai
(rescue-ops cases/A-075, litellm 1.83.9). The first is a false pass, in the one direction
this product must never fail.

  • A capture-derived agent now replays the framework's per-turn shape. adapt --from-capture wrote ONE episode-level tool_choice, picked by frequency over the recorded
    requests. A framework that forces a tool on turn 1 and then goes auto — pydantic-ai and
    litellm both do, and so does every structured-output and routing agent — became an adapter
    that forced a tool on every turn. Under a forced choice the model cannot answer in text,
    so the replayed episode called tools until max_turns and failed its own turns_at_most
    on the baseline model; with nothing passing on the baseline there was no regression to
    find, and the verdict read SAFE, "the candidate model is a drop-in replacement", over a
    suite that never worked once.
    • agent.json turn_params (ADAPTER.md): param overrides applied over params by
      assistant-turn index, the last entry repeating, null meaning the field was not sent on
      that turn. agent_loop applies turn N's overrides to turn N's request; the repair loop
      reads the sequence too, so removing a forced tool_choice removes it from every entry
      instead of finding nothing to repair.
    • adapt --from-capture derives it for exactly the params one recorded conversation
      sent differently across its turns, and leaves those out of params. Captures that
      disagree with each other in a way no sequence can express are not resolved quietly:
      every variant and its count land in ADAPT_EDITS.md under CONFLICTS, and
      adapt --from-capture --strict exits non-zero. A tie is broken by canonical text and
      reported — never by which request the recorder happened to see first, which is what
      Counter.most_common was doing.
  • BASELINE_BROKEN — the guard that catches this independently of capture. A run whose
    baseline model passed no case measured nothing about the candidate and can never be
    SAFE. Checked before every other verdict, and unable to mask one (a regression needs a
    case the baseline passed). upshift upgrade also says it between the two legs, before the
    candidate run spends money on a comparison that cannot mean anything.
  • Captures, run records and adapt records are stamped with the version that actually ran.
    upshift.__version__ was a hand-maintained literal three releases behind pyproject.toml,
    so every one of them claimed 0.1.0 while upshift --version reported the truth. It now
    reads the installed package metadata, the way cli._version() always did. Provenance only:
    no measurement changes, and the 169 already-committed lab records carry the stale string.
  • Tool fields the adapter cannot carry are reported, not dropped in silence. The
    chat-style shape holds name/description/input_schema; everything else a recorded tool
    carried is now a per-tool note in ADAPT_EDITS.md, louder for a server tool — a
    computer_20241022 with no input_schema at all was becoming a plain custom tool with an
    empty schema, with nothing anywhere saying so. Structural deviation 2 stops claiming
    byte-identity and names the two fields dropped on purpose.

Also unreleased, from the OpenAI migration-rescue track (product gaps the lab hit while
running 54 cases):

  • Endpoint routing translates the output-token cap. /v1/responses spells it
    max_output_tokens; an agent written against /v1/chat/completions carries max_tokens
    (classic families) or max_completion_tokens (gpt-5*/o-series), and passing either to the
    Responses SDK raises TypeError: Responses.create() got an unexpected keyword argument 'max_completion_tokens' before a request is sent — crashing the whole run rather than
    recording a failed rep. Since endpoint routing is the documented repair for the gpt-5.5+ /
    gpt-5.6 "function tools ... in /v1/chat/completions" 400, the untranslated cap made that
    repair unusable for any agent that sets one. map_params now maps both spellings to
    max_output_tokens on responses; an explicitly-spelled max_output_tokens wins.

  • Pricing for the rest of the gpt-5.6 family. upshift cost reported "unknown rate" for
    gpt-5.6-terra and gpt-5.6-luna. Standard-tier rates per 1M tokens, from
    https://developers.openai.com/api/docs/pricing (fetched 2026-09-03): sol $4.00 in /
    $0.40 cached / $20.00 out (unchanged), terra $2.00 / $0.20 / $12.00, luna $0.20 / $0.02 /
    $1.20. The published flex and batch rows are exactly half of standard and the cached rows
    exactly 10% of input, which is what the existing tier and cache multipliers already do.

  • upshift run and upshift upgrade take --max-cost-usd. Only adapt, which makes one
    paid call, had a spend ceiling; run and upgrade make thousands — baseline reps,
    candidate reps, t...

Read more

v0.3.1 — pre-launch security and CLI fixes

Choose a tag to compare

@atilavahedian atilavahedian released this 03 Sep 08:20

v0.3.1 — 2026-09-03

Pre-launch security pass over adapt, the runs root and the shell sandbox. Hostile input
here means a repository you point upshift adapt at, and the model output it steers.

  • adapt refuses a source that git would read as an option (--upload-pack=…/x.git and
    friends) and clones with -- terminating option parsing.
  • The repo walk skips any path that resolves outside the repository root, so a checked-in
    symlink can no longer pull ~/.ssh/id_rsa or /etc/passwd into the evidence sent to the
    model. (The pointer-following round already resolved paths against the root.)
  • The generated backend.py escapes everything interpolated into its header docstring —
    origin, commit, and the model-written tool names and citations — and the rendered file is
    parsed before it is written. A repo whose text contains """ can no longer place
    statements in a file upshift upgrade imports and runs.
  • --tag and case ids are validated as single directory names, so everything upshift writes
    (and everything the repair loop replaces) stays under the runs root.
  • shell_gpt sandbox: added --memory 512m and --security-opt no-new-privileges.
  • A hostile integer literal in a target repo no longer aborts adapt with an uncaught
    ValueError from CPython's int-to-string digit limit.
  • New tests/test_security.py pins all of the above; pip-audit clean.

v0.3.0 — two providers, two migrations documented

Choose a tag to compare

@atilavahedian atilavahedian released this 02 Sep 09:28

v0.3.0 — 2026-09-02

  • Second provider: Anthropic Messages API (--provider anthropic, endpoint messages),
    same statistics and repair loop, no provider forks in the core. Prompt caching on the
    cached prefix; cache-write pricing; identity-linked keys via ANTHROPIC_WORKSPACE_ID.
  • Anthropic's documented Fable 5 → 5.1 changes as detectors and repairs: forced
    tool_choice 400 (remove + instruction), thinking-block invalidation (detect, refuse
    with the documented pointer), serialized tool calls (turns_at_most + the documented
    batching sentence), reduced retrieval at low effort (effort ladder + documented nudge),
    unsupported sampling params (drop). Effort calibration is a first-class repair.
  • sim-fable-5 / sim-fable-5-1 for keyless rehearsal; upshift adapt reads Anthropic
    call sites and Jupyter notebooks.
  • Release-day report on four open-source Claude agents (reports/fable-5-1-upgrade.md).
  • Runner aborts on billing/quota errors instead of recording junk reps.

Full evidence: reports/fable-5-1-upgrade.md · reports/shellgpt-upgrade.md