Repository navigation
since-cutoff 0.4.1: removed parameters point to extra_body; imported-scope notes lead with what breaks #23
MohammadHijjawi97
announced in
Announcements
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
since-cutoff 0.4.1 is on PyPI (
uvx since-cutoff@latest scan). Two changes to the notes, both found while preparing the head-to-head benchmark.A removed parameter no longer reads as "drop it"
For anthropic 1.8.0 the 0.4.0 note said: "
Messages.create()no longer acceptstemperature,top_kortop_p; do not pass them." In the benchmark pilot, a task asked fortemperature=0. Every agent run with that note obeyed it and left the temperature out, so the hidden test failed. The parameters left the signature, but the pinnedcreate()still sends extra fields throughextra_body. A static diff can see that; it cannot see whether the API still accepts the field. The note now says only what it knows:This applies to any SDK method that takes
extra_body/extra_query: anthropic, openai and the other Stainless-generated clients. Other removed parameters keep "do not pass them".sync --scope importedleads with what breaks new codeWith
--scope imported(notes for the packages your code imports, not only the APIs it already uses), the notes now come in this order:mcp.server.fastmcp→mcp.server.mcpserverused to be one note per module.FastMCP→MCPServeris found because 29 of its 30 public members are kept; on anthropic, openai, huggingface-hub and pandas this finds no renames at all.--per-package Nsets how many notes per package.On mcp 1.28.1 → 2.2.0 the five notes went from parameter removals on
ClientSessionand friends to: the package move, the two renames (McpError→MCPError,FastMCP→MCPServer),streamablehttp_client→streamable_http_client, andlist_prompts(cursor=)→params.1013 tests, 94% branch coverage. Full changelog: https://github.com/MohammadHijjawi97/since-cutoff/blob/main/CHANGELOG.md
Benchmark status. The pilot (2 tasks × 4 arms × 2 runs, 16 sessions) is done and excluded from the analysis. It confirmed the harness works: Context7 connected in 8 of 8 runs, and a session took a median of 54 s at a median cost of $0.26. In the pilot, the docs arm called Context7 in only 2 of 4 runs. The pre-registered rule for that case is a fifth arm that is instructed to use Context7, so the main run is now 24 tasks × 5 arms × 3 runs. Because the note wording changed after I saw a pilot task, the results will be published twice, with and without the two pilot tasks, and a claim counts only if it holds in both.
All reactions