Skip to content

Releases

2.25.9

Choose a tag to compare

@feronovak feronovak released this 19 Nov 07:09
0f2fdaf

Added

  • gemini-4-neon now has an official snapshot price, an active retirement row and tier 1 in the model importance list. Rates are flat across the 1M window, in USD per million tokens (input / output / cached input): 3.00 / 15.00 / 0.15, read from Google's pricing page on 2026-11-19. It takes the Gemini 4 request checks with XHIGH thinking, like gemini-4-hydrogen; the eight-case measured run is a follow-up. Breaking: configs that name gemini-4-neon change results. --pricing-check exits 0 instead of 1; --dry-run with require_current_pricing or max_estimated_cost_usd exits 0 instead of 2; approval receipts recorded without a cost cap now report mismatch; the retirement verdict is active instead of unknown, so --ci exits 3 once the row is stale (from 2026-12-20); --verify-approval-receipt exits 1 instead of 0 for approval receipts recorded with no cost cap. A cachedContent setting, a thinking level outside LOW/MEDIUM/HIGH/XHIGH (MINIMAL is rejected) or a synchronous output cap over 65,536 now fails before any request (--dry-run does not check these). Sampling parameters are omitted, and catalog discovery no longer claims temperature support.

Changed

  • Re-measured glm-5.3, glm-5.3-flashx, glm-5.3-flash and kimi-k3.1-pro on 2026-11-18 (eight cases each, same method as 2026-10-29) and updated the README table, comparison guide and results image. glm-5.3-flashx improved from 7/8 to 8/8; the other two GLM sizes are unchanged at 7/8; Kimi K3.1 Pro is still incomplete (3 responses and 5 HTTP 429 rate limits). The README now shows 225 of 232 checks passed and 26 of 29 models passing all eight.

2.25.8

Choose a tag to compare

@feronovak feronovak released this 18 Nov 08:44
2fd9445

Changed

  • Added gpt-6.2-astra to the measured-models comparison: an eight-case run on the native openai route, rendered into the results image and the README table.
  • All bundled price and retirement rows were re-checked against their official pages on 2026-11-18; prices, retirement dates and replacements are unchanged. The review dates moved, so the rows now become stale on 2026-12-19 (strict pricing, and --ci exit 3 for retirement rows). Two groups keep their earlier date: gpt-5.6-sol stays 2026-11-04 and becomes stale on 2026-12-05, because its promotional price runs to 2026-11-30 and the post-promotion price is re-read separately, and the rows reviewed on 2026-11-17 (claude-opus-4-5-20251101, claude-opus-4-6, claude-opus-4-7, claude-sonnet-4-6, the four DeepSeek rows and the four Qwen rows) stay stale on 2026-12-18. 55 OpenAI retirement rows that the deprecations page lists under Past deprecations (gpt-5.4-cyber and 54 older ones) are now retired instead of deprecated; their shutdown dates had already passed, so no verdict changes. Breaking: every approval receipt, mock plans included, now verifies as mismatch (--dry-run --verify-approval-receipt exits 1 where it exited 0, and migrate --run gates on the same receipt), because the retirement snapshot's oldest and newest review dates are part of every plan, and the pricing review date is part of any plan that names a re-dated price row; record them again with --dry-run --approval-receipt --replace-approval-receipt.

2.25.7

Choose a tag to compare

@feronovak feronovak released this 18 Nov 07:03
4e88c86

Added

  • gpt-6.2-astra and gpt-6.2-sol now have official snapshot prices, so a smoke or dry run reports a cost instead of "missing price" (USD per million tokens, input / output / cached input / 30-minute cache write, up to 272,000 input tokens): 8.00 / 40.00 / 0.40 / 10.00 for Astra and 2.00 / 10.00 / 0.10 / 2.50 for Sol; above that, 16.00 / 60.00 / 0.80 / 20.00 and 4.00 / 15.00 / 0.20 / 5.00. Both are rated tier 1 in the model importance list. Like the other GPT-6 models they omit temperature, accept only the standard service tier and 30-minute cache retention, and require a completed response status. Saved plans for these models change once because they are now priced. Breaking: for a config naming either model, --pricing-check now exits 0 where it exited 1, and --dry-run exits 0 where it exited 2 when the config sets require_current_pricing or max_estimated_cost_usd. A config that sets service_tier other than default, standard or auto, or prompt_cache_retention: "12h", for either model now makes --dry-run exit 2 where it exited 0 (a config that also sets a cost cap or require_current_pricing already exited 2) and --doctor exit 1 where it exited 0, and a run stops before any request; remove that setting, then record approval receipts again with --dry-run --approval-receipt --replace-approval-receipt. An approval receipt recorded before this change for a config naming either model with no cost cap and no require_current_pricing now fails --dry-run --verify-approval-receipt with mismatch (exit 1 where it exited 0) because the plan is now priced; record it again with --replace-approval-receipt. gate (CLI and MCP) now returns routes_changed (exit 1 where it exited 0) for a passing CI record attested before this change for a config naming either model with owner prices, because the route fingerprint now records that temperature is not sent; attest a new CI record. A Responses reply for these models without a status now fails the request.

Changed

  • Added gpt-6.2-sol to the measured-models comparison: an eight-case run on the native openai route, rendered into the results image and the README table.
  • The four DeepSeek rows (deepseek-flash, deepseek-v4-flash, deepseek-v4-flash-vision-exp, deepseek-v4-pro) were re-reviewed against the official pricing page on 2026-11-17; their prices are unchanged and they now become stale on 2026-12-18. The legacy deepseek-v4-flash names bill at the Flash price. Breaking: approval receipts recorded for a DeepSeek plan before this change verify as mismatch (--dry-run --verify-approval-receipt exits 1), because the pricing review date is part of the plan; record them again with --dry-run --approval-receipt --replace-approval-receipt.
  • The pricing review date for qwen3.8-max, qwen3.8-flash and qwen3.7-plus is now 2026-11-17, re-checked against the official Model Studio pricing page. Prices are unchanged, so their snapshot stays fresh through 2026-12-17.

2.25.6

Choose a tag to compare

@feronovak feronovak released this 17 Nov 15:30
675aa52

Added

  • claude-opus-4-5-20251101, claude-opus-4-6, claude-opus-4-7 and claude-sonnet-4-6 now have official snapshot prices, so a smoke or dry run reports a cost instead of "missing price" (USD per million tokens, input / output / cached input): 5.00 / 25.00 / 0.50 for the three Opus models and 3.00 / 15.00 / 0.30 for Sonnet 4.6. No cache-write rate is bundled, as for the other Anthropic rows. Saved plans for these models change once because they are now priced. claude-opus-4-5-20251101 does not support inference_geo, so a plan that sets it is refused before any request, as for claude-haiku-4-5-20251001. Breaking: for a config that sets inference_geo for claude-opus-4-5-20251101, --dry-run now exits 2 where it exited 0, --doctor exits 1 where it exited 0, a run stops with exit 2 before any request (it used to send one the API rejects with HTTP 400), and the MCP dry_run_plan tool returns an error. To keep such a config runnable, remove inference_geo for this model, then record receipts again with --dry-run --approval-receipt --replace-approval-receipt. The four priced models also change exit codes: --pricing-check now exits 0 where it exited 1, and --dry-run exits 0 where it exited 2 when the config sets require_current_pricing or max_estimated_cost_usd.

Fixed

  • claude-opus-4-7 no longer fails with HTTP 400: temperature is deprecated for this model: default requests now omit temperature for it, and so does a temperature set in the run request, like the other Anthropic models that reject it.

Changed

  • Added qwen4-max to the measured-models comparison: an eight-case run on the native qwen route, rendered into the results image and the README table.

2.25.5

Choose a tag to compare

@feronovak feronovak released this 17 Nov 10:02
66178ec

Added

  • qwen4-max now has an official snapshot price on the native qwen route: 1.50 / 4.50 USD per million tokens (input / output), cached input 0.075 and cache write 1.875 (5-minute rate). Saved plans for this model change once because it is now priced.
  • qwen4-max on the native qwen route now sends max_completion_tokens by default, so the output limit covers reasoning plus the answer, as for Qwen 3.8. Breaking: a qwen4-max config with max_tokens_parameter: max_tokens and thinking not explicitly disabled is now refused before any request, with a message naming the constraint: --dry-run exits 2 where it exited 0, --doctor now reports the model as failed and exits 1, where it exited 0, and a run exits 2 before any request where it sent the request and could exit 0 or 1. To keep such a config runnable, set max_tokens_parameter to max_completion_tokens (the default) or explicitly disable thinking.

2.25.4

Choose a tag to compare

@feronovak feronovak released this 14 Nov 12:29
8120681

Fixed

  • mistral-large-4 and mistral-large-4-0 with temperature: 0 and no top_p: 1 are now refused before any request, with a message naming the constraint, instead of ending as an HTTP 400 at run time. Set provider_options.mistral.top_p to 1 or omit temperature in the run request. The init --provider starter and the catalog watch request now set that top_p, so they pass for Mistral Large 4. Doctor and --dry-run apply the default request when a config has no request block, as a run does. Breaking: for such a config, --dry-run and a run now exit 2 before any request, where --dry-run exited 0 and the run exited 1 with the HTTP 400 case failed; --doctor now reports the model as failed and exits 1, where it exited 0. This also applies to a config with no request block and to the top-level low-latency and structured presets, which set temperature 0 and are refused too unless top_p is 1. The built-in tests that use the structured preset (structured-output-check, strict-json-extraction, code-patch-summary, source-grounded-quiz, and the agent-smoke suite that contains them) are refused even with top_p 1, because they replace provider_options: --dry-run exits 2 where it exited 0, a run exits 2 where it exited 1, and --doctor with profiles exits 1 where it exited 0. To keep a configuration runnable, set provider_options.mistral.top_p to 1 or omit temperature in the run request; to test this model, use a custom prompt with its own request instead of those built-in tests. No request fingerprint changes.

2.25.3

Choose a tag to compare

@feronovak feronovak released this 13 Nov 08:52
edb7205

Added

  • Six models now have official snapshot prices, so their cost is no longer unknown (USD per million tokens, input / output / cached input): gpt-6-cyber 20.00 / 100.00 / 2.00 (cache write 25.00; above 272,000 input tokens 40.00 / 150.00 / 4.00, with no published long-context cache-write rate, so a budget for such a prompt that may write the cache stays unknown), gpt-5.6-cyber 12.50 / 75.00 / 1.25 (cache write 15.625), gemini-4-hydrogen 1.50 / 7.50 / 0.075, and grok-4.20-0309-reasoning, grok-4.20-0309-non-reasoning and grok-4.20-multi-agent-0309 1.25 / 2.50 / 0.20 (2.50 / 5.00 / 0.40 from 200,000 input tokens). Saved plans for these models change once because they are now priced. gpt-5.6-cyber supports only the Responses API, so a run needs adapter: openai_responses.
  • gemini-4-hydrogen is now in the measured-models comparison: all eight support-routing cases passed on the native Gemini route for $0.007221, and it appears in the README image and table, the comparison guide and the snapshot pages.

Changed

  • gemini-4-hydrogen gets the Gemini 4 request checks, with its own thinking levels: LOW, MEDIUM, HIGH and XHIGH are accepted and MINIMAL is rejected. As for gemini-4-argon and gemini-4-carbon, an explicit cachedContent and a max_output_tokens above 65,536 fail before any request, ignored sampling fields are omitted, and discovery no longer reports temperature support. A thinking level other than LOW, MEDIUM or HIGH on gemini-4-argon or gemini-4-carbon still fails.

Fixed

  • The bundled replacements for four OpenAI models retiring on 2026-12-11 now match the official deprecations page: gpt-5-2025-08-07 and o3-2025-04-16 name gpt-6.1-sol (was gpt-5.6-sol), gpt-5-mini-2025-08-07 names gpt-6.1-terra (was gpt-5.6-terra), and gpt-5-nano-2025-08-07 names gpt-6-luna (was gpt-5.6-luna). The verdict, migrate and next_command show the new replacement. The four rows were reviewed on 2026-11-13 and become stale on 2026-12-14. Breaking: approval receipts recorded before this change verify as mismatch (exit 1), because newest_as_of in every dry-run plan is now 2026-11-13. Record them again with --dry-run --approval-receipt --replace-approval-receipt.

2.25.2

Choose a tag to compare

@feronovak feronovak released this 12 Nov 22:17
a8996f4

Fixed

  • --dry-run, --doctor, the interactive preview and the MCP dry_run_plan tool now warn (owner_price_ignored) when an owner price field is replaced by the bundled official snapshot, for example a cached-only rate on gpt-4.1. Resolved prices and cost figures are unchanged. The interactive and --changed-since dry-run paths carry the warning too, and a non-numeric owner value is named without being echoed. Breaking: --dry-run --verify-approval-receipt exits 1 (mismatch) for a receipt recorded for an affected configuration, where it exited 0 before, because pricing_warnings gains an entry; migrate --run gates on the same receipt. Record such receipts again with --dry-run --approval-receipt --replace-approval-receipt.

Changed

  • The agent guidance (the MCP initialize instructions and safe-workflow resource, the Codex plugin skill, and the MCP and coding-agent guides) now tells an agent to report pricing_warnings, including owner_price_ignored, from a dry run. The configuration and pricing pages state that an owner cache_write_input_cost_per_million applies where the bundled row has no cache-write rate, and docs/reference/results.md describes pricing_warnings as covering ignored owner prices too.

2.25.1

Choose a tag to compare

@feronovak feronovak released this 12 Nov 16:56
0185bcb

Changes below are marked Breaking: where they can change an exit code.

Added

  • The bundled retirement rows now include OpenAI gpt-5.5-cyber-preview: deprecated, removal 2027-02-01, no replacement named on the official page. --audit-source --ci and --dry-run --ci now exit 3 for it (it was unknown, exit 0), and 1 from 2027-02-01. The row was reviewed on 2026-11-12 and becomes stale on 2026-12-13. Breaking: approval receipts recorded before this change verify as mismatch (exit 1), mock plans included, because the new row changes retirements.snapshot.rows (119 to 120) and newest_as_of in every dry-run plan. Record them again with --dry-run --approval-receipt --replace-approval-receipt (without --replace-approval-receipt the command exits 2 when the receipt file exists).

Fixed

  • Breaking: a run on adapter: openai_responses now fails with the tool-call error when the response contains a tool call (such as a function_call item), instead of passing on any text beside it. Such a run exited 0 before and now exits 1.
  • Breaking: --replay now checks the saved result's source_config shape like every other config path: a wrong-typed field such as presets: 7 exits 2 and names the field instead of raising a traceback and exiting 1.
  • Breaking: estimates for cache hits on gpt-4.1, gpt-4.1-mini, gpt-5.4-mini, gpt-5.4-nano, claude-fable-5, claude-opus-5 and claude-sonnet-5 now use the published cache-read rates (USD per million tokens: 0.50, 0.10, 0.075, 0.02, 1.00, 0.50 and 0.20). They used the full input rate before, so estimates were too high. Cache writes stay unpriced for these models. Saved dry-run plans change once for configs using these models because the plan's model row gains cached_input_cost_per_million, so approval receipts recorded for them verify as mismatch (--dry-run --verify-approval-receipt exits 1, where it exited 0); record them again. An owner-set cached_input_cost_per_million is kept only together with the owner's own input and output prices; otherwise the bundled rate now applies to these seven.

2.25.0

Choose a tag to compare

@feronovak feronovak released this 12 Nov 07:42
e14cc9e

Changes below are marked Breaking: where they can change an exit code.

Added

  • The retirement report now carries snapshot.valid_until, the last day the rows of the selected models are not stale, in the doctor and dry-run JSON (the scan JSON carries the bundled-wide date). Text output of the doctor, dry-run and scan adds a notice during the last 7 days; it never changes the decision or exit code. Saved dry-run plans change once because the plan now includes the field, so approval receipts for the old plan verify as mismatch.
  • The Action guide explains how to keep the pin current with Dependabot.
  • The MCP safe-workflow resource, the initialize instructions, the plugin skill and the agent guides now tell an agent to report retirement_snapshot.valid_until from audit_source and retirements.snapshot.valid_until from dry_run_plan, and the CLI agent guides name alias_unmapped and the dry-run limit breach. The text is additive; no schema, tool, key or exit code changes.

Changed

  • The README's measured-models image now shows all 25 summarized models in three result groups, ordered by provider inside each group and including Gemini 3.1 Flash-Lite, with a light variant, shorter columns and larger text, and the README gives the same rows as a collapsed table. scripts/render_measured_models.py writes both variants.
  • Breaking: --audit-source now reads whole model IDs in backtick strings in .js, .mjs, .cjs, .ts and .tsx files. A JavaScript or TypeScript project with a retired model ID in backticks now exits 1 under --audit-source --ci where it used to exit 0. --change-plan uses the same scanner, so a changed JavaScript or TypeScript file with a backtick model ID now reports a model_id trigger instead of contract_surface. Backticks in comments count too, and template strings with ${…} are still not matched.

Fixed

  • Breaking: --audit-source, --dry-run --ci, --doctor, migrate and the MCP audit_source and
    migrate_plan tools now look model IDs up in lower case and resolve an Anthropic alias such as
    claude-opus-4-1, claude-sonnet-4-0 or claude-3-7-sonnet-latest to the verdict of its dated snapshot row (the
    verdict carries matched_id). Such an alias of a retired model used to be unknown and exit 0 under --ci;
    it now exits 1.
  • Breaking: an OpenAI alias with dated snapshot rows but no row of its own (gpt-5, gpt-5-mini, gpt-5-nano,
    gpt-5-pro, o3, o3-pro, and the two search-preview IDs) is now alias_unmapped (decision inconclusive, with
    alias_candidates) and exits 3 under --ci, where it exited 0. retirement_report summaries gain an
    alias_unmapped count and migrate reports the alias_unmapped outcome.
  • The --audit-source --ci exit 0 line no longer says "no referenced model is retired, retiring or stale" when IDs
    are unknown. It says no referenced model has a retirement finding in the snapshot, and counts the model IDs that
    have no snapshot row and are not verified.
  • Breaking: --dry-run (also catalog watch-new --test --dry-run) now prints the plan and then fails with exit 2,
    and the same message as the live run, when the config breaks a limit the live run refuses: max_requests below
    the possible requests, max_estimated_cost_usd exceeded or with unknown pricing, or incomplete pricing under
    require_current_pricing. It used to exit 0 and the live run then refused. The plan, text and JSON, carries
    budget_limits and a Limit breach: line, and the MCP dry_run_plan tool returns ok: false with the plan.
    Approval receipts written by 2.24.2 verify as mismatch because the hashed plan gains budget_limits;
    record them again with --dry-run --approval-receipt. A max_requests or max_estimated_cost_usd that is not a
    finite number is invalid input and is now refused with a must be a number error and exit 2, before any request,
    in the dry run and the live run. A non-numeric value used to be a traceback, and "inf", "Infinity" or "NaN" as
    max_estimated_cost_usd ran on 2.24.2 with no effective cost cap.
  • Breaking: verify and gate resolve an Anthropic alias route (for example anthropic:claude-opus-4-1) to its
    dated snapshot row. A CI record whose checked route is a retired alias used to verify with exit 0 and pass the gate
    as fresh_pass; it is now superseded: verify exits 3 and gate exits 1.
  • The examples/vision and examples/project-integration configs bound retries with request.retry
    max_attempts: 1, so their one-request cap holds on a live run.
  • Provider error text, saved results, reports and MCP output now mask the exact value of every environment variable
    named by api_key_env (eight characters or more) once a request or catalog discovery reads it, and a bare
    Bearer token of 16 or more characters. Keys without a known prefix (Mistral, MiniMax, Cohere, Z.ai,
    openai_compatible) used to be echoed back unchanged when a provider error body repeated them.
  • The retirement guide now states the right review and stale dates, and lists what the source scan does not see. The Action guide says non-mock configs need their key variables set for the doctor step. The CLI reference documents init --api-key-env, --write-env-example and --force, --replace-approval-receipt, catalog probe --ledger, pricing-refresh --provider, and --all-unapproved, --select-candidates and --write-config for catalog refresh and watch-new. The decision reference notes that undated hand-typed prices only block under require_current_pricing.