Record how a verdict was sampled, and stop promising replay - #694
Merged
Conversation
The durable inference record has carried request_temperature and request_top_p since it was built, named for the OpenTelemetry GenAI convention, and no producer ever filled them. On the pilot record 583 of 583 calls wrote their max-token bound and 0 of 583 wrote how they were sampled, so every recorded verdict was one whose sampling nobody could state. The port now carries both dials, both optional, and both adapters send only what a caller actually set. Silence stays silent: inventing a default would put a sampling claim on the record that no caller made, and the SDK's real default is not necessarily the documented one. All three tasks that build a chat request pin temperature to zero. Each picks from a fixed set under a schema the provider already enforces, so sampling variance buys nothing and costs consistency. Zero is not determinism and the comment at each site says so: Anthropic exposes no seed at all, and batching on a shared GPU perturbs the in-house path. What pinning buys is lower variance plus a number the record can state because we chose it. That makes the catalog's old claim untenable, so it is withdrawn. ArchivabilityTier said Pinned meant "results stay re-executable", which promised replay. Replay needs sampling settings and a seed, and no tier setting supplies the seed. The tier grades whether the identity is still invocable, which is availability, not reproducibility. That docstring is published verbatim into openapi.json, so the wording was an API claim rather than a note, and the correction ships with the schema. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Re-running the comparison after a code change is not the same request twice, and each way of getting it wrong yields a full, fast, plausible result instead of an error, which is what makes them worth writing down. Re-used idempotency keys replay the previous walk's stored responses without reaching the model. A process not restarted after a pull serves the old build; the check for that is a record one, since nothing wrote request_temperature before sampling provenance shipped. And the new decisions have to be separated from the old by event position rather than by the field being examined, or the measurement agrees with itself. Also names the join. Inference entries are the record's second tier and their event_id is a derived dedup key, not a row in events, so the obvious event_id-to-event_id join matches nothing on a healthy deployment. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Coverage report
Click to see where and how coverage changed
This report was generated by python-coverage-comment-action |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
xmap
added a commit
that referenced
this pull request
Aug 20, 2026
…698) #694 pinned temperature to 0.0 in all three chat-request builders, on the reasoning that these tasks pick from a fixed verdict set so sampling variance buys nothing. That reasoning holds. The parameter does not. Anthropic has deprecated temperature, and newer models reject a request carrying it. Measured against the Argo gateway on 2026-08-20: Sonnet 4.5, Haiku 4.5, Sonnet 4.6, Opus 4.6 accept it Opus 4.7, Sonnet 5, Opus 5 400, "`temperature` is deprecated for this model" So the pin does not steady a modern model's answer, it prevents one. Every agent shipped today declares a model on the accepting side of that line, so nothing in place broke; what the pin did was silently foreclose every upgrade, and fail in the least legible way when one was attempted. The refusal arrives as an invalid-request error, the debrief defers, and nothing in the deferral names sampling. It was found by pointing a new RunDebriefer at Sonnet 5 and watching the preflight refuse on call one. The port keeps both dials and the record still writes whatever a caller sets, so the provenance work stands. What is withdrawn is the judgement that these tasks should set one. A caller that sets nothing records nothing, which is the honest state, and honesty is the point of that column. There is a wider consequence worth stating for anyone reasoning about replay: what can be recorded about how a model was asked is bounded by what the vendor still permits to be specified, and that surface is contracting as sampling knobs give way to adaptive reasoning. Recording sampling faithfully does not arrest that. Co-authored-by: xmap <16776958+xmap@users.noreply.github.com> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The durable inference record has carried
request_temperatureandrequest_top_psince it was built, named for the OpenTelemetry GenAI convention, and no producer ever filled them. On the pilot record 583 of 583 calls wrote their max-token bound and 0 of 583 wrote how they were sampled.What changes
LLMChatRequestcarriestemperatureandtop_p, both optional, defaulting toNone.anthropic.omit; the OpenAI-compatible path builds the field dict conditionally.run_debrief,caution_drafter,_llm_decide_prompt) pintemperature = 0.0.entries_decision_inferencesthroughAgentInferenceTrace.ArchivabilityTier's "results stay re-executable" claim is withdrawn in favour of "the identity stays invocable".Two things worth a reviewer's attention
Identity checks, not truthiness. A deliberate mutation to
if request.temperature:makestest_sends_the_sampling_the_caller_setfail. Zero is the only value anything sets, so a truthiness check would have silently dropped every configured value while the code read as wired.The retraction is an API change.
ArchivabilityTier's docstring is emitted verbatim intoopenapi.json, so the old wording was a promise to API consumers, not a maintainer note. Replay needs sampling settings and a seed; Anthropic's Messages API accepts no seed at any price, so no tier setting can deliver reproducibility. The archaeology sits in a#comment above the class so it does not ship into the public schema.Scope note
Callers that set nothing still record nothing. The 583 existing rows ran at provider defaults that cannot be recovered, so they stay blank permanently; only future calls carry sampling provenance.
Gates
pytest tests/unit tests/architecture: 45068 passed, 630 skippedpytest tests/contract tests/integration: 4663 passed, 1 skippedpyright src/cora tests/unit: 0 errors; ruff clean🤖 Generated with Claude Code