Releases
0.3.290
Compare
Sorry, something went wrong.
No results found
Features
active_model_event_id() returns the in-flight model call attempt's event id (#6529 ) (c060c20 )
each message keeps one id through the agent bridge and the log, and tool events record the message that proposed them (#6143 ) (187d19f )
excluded epochs are left out of epoch reduction, and score reasons gain excluded and zeroed (#6528 ) (b4aa05b )
google: support Gemini 4 Carbon (#6466 ) (8c6f22a )
google: support Gemini 4 Hydrogen (#6547 ) (d0c2ae1 )
logs of errored or cancelled evals keep their total sample count (#6476 ) (5a1654f )
meta: support Muse Ember 1.0 and Muse Wonder 1.0 (#6467 ) (e95b39b )
mistral: support Mistral Large 4 (#6531 ) (1ac1482 )
sandbox agent bridge inside a sandbox identity scope serves only the scope's user (#6270 ) (9a0b77e )
sentinel: count a sentinel sidecar's judging time as waiting and its judge models' usage in the eval (#6513 ) (0152514 )
sentinel: sign calls to a sentinel sidecar instead of sending the eval's token (#6444 ) (9961043 )
Bug Fixes
a bridged agent's host tools make no model call after a limit is reached (#6500 ) (55e9344 )
a bridged agent's queued requests and native compactions make no model call after a limit is reached (#6434 ) (10dc86e )
a large denied sandbox bridge host tool call no longer uses extra memory to check for retries (#6471 ) (6a1efac )
a retry chosen in the cancel dialog no longer re-runs a task cancelled with inspect ctl (#6457 ) (40d60b8 )
a second eval() in the same process ignores a changed display setting (#6267 ) (674ac5e )
ACP approval and ask_user requests stuck on a client that switched sample or rejected the request (#6487 ) (acd87bb )
agents wrapped with as_tool or as_solver fail when reused with limits (#6387 ) (842a233 )
an operator cancel during scoring no longer hides a scorer error (#6455 ) (4364a56 )
anthropic: long requests no longer retry forever on the 10 minute read timeout (#6511 ) (fbb88c6 )
async fsspec reads block the event loop (#6012 ) (099d83a )
computer: type action doesn't use Return for newlines (#6490 ) (fba8be6 )
count a sample once toward fail_on_error when its score_on_error scoring also fails (#6507 ) (36294bb )
eval() raises an error instead of silently running a different task registered under the same name (#6231 ) (918246a )
keep a host's Rich progress context usable after an eval (#6409 ) (95024b3 )
keep the other batch results when one result row fails to parse (#6217 ) (36eaa3e )
list_files() no longer runs a path starting with - as a find action (#6489 ) (88eef6a )
math() timeouts no longer look like wrong answers in eval logs (#6384 ) (16a39eb )
messages_df() keeps messages that several samples share (#6389 ) (8abad63 )
moonshot: honor low and high reasoning effort for Kimi K3 (#5946 ) (bc9998c )
openai-responses: preserve content_filter stop reason on partial tool calls (#6227 ) (d08cf15 )
openai: computer tool results with no screenshot no longer fail the next request with a 400 (#6516 ) (35fa403 )
pick the JSON reader from the URL path, not the whole URL (#6209 ) (3391d54 )
read digits before a Chinese unit as one number (#6208 ) (c8c58d4 )
recovered samples keep their store (#6165 ) (13149f6 )
score numbers wrapped in full-width punctuation (#6381 ) (a82e702 )
sentinel: a sidecar call that ends a sample on a token, cost or turn limit shows only the model's time (#6542 ) (e4435ba )
stop Hugging Face provider sending two BOS tokens for Gemma and Llama 3 (#6385 ) (2ce28a3 )
vLLM and SGLang server API keys are no longer written to logs or startup errors (#6493 ) (793f79e )
web_search() Google provider no longer logs the API key on connection retry (#6491 ) (050c34f )
Performance Improvements
copy shard sample members into the merged log without recompressing them (#6488 ) (26f6d05 )
You can’t perform that action at this time.