You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This commit was created on GitHub.com and signed with GitHub’s verified signature.
Added
eval run --live executes every tool for real against the host's environment for a factory run: no replay, no
fixtures, only tasks of provenance: live, the result marked live and never a board row (eval report refuses a
results directory holding one, eval lint --results names it, make scoreboard lints first). It fails closed
unless TARACODE_EVALS_LIVE_SANDBOX names the sandbox the machine is and the kubeconfig's current context agrees
(the evals README, "Factory runs"). A task of provenance: live has no fixtures and runs only under --live.
eval run --temperature samples above the board's temperature 0 (up to 2) and records the value; a sampled result
is a factory result like a live one and never ranks. Neither may write into docs/evals/results.
eval run --transcripts-json writes each run's transcript as JSON next to the text log: the tools offered, every
message with the reply's reasoning (aligned to the appended messages), the tool calls and results, the calls
block, the score, the usage, the row's flags, and the lab's private names it carries, if any.
The agent offers a ReplyObserver hook and Conversation() and ToolDefinitions() copies for headless callers.
Changed
A shell call is a bounded action. The command runs in a session of its own: a process group to itself and no
controlling terminal, so a prompt (a sudo password, an ssh host key) fails at once with its own message instead
of waiting for input the model cannot give. Its timeout interrupts that whole group, as Ctrl-C would (terraform
stops at a safe point), and ends what remains after a 30 s grace, TERM then KILL, so a hung child no longer lives
on. A background child that keeps the output open can no longer hold the call past sh's exit for more than a
few seconds. Whatever the command leaves running when it returns is ended the same way, with a line in the
result saying so: cmd & and nohup cmd & no longer survive the call. On Linux a process meant to outlive it
needs setsid CMD >FILE 2>&1 </dev/null &, which the result tells the model where the system has setsid. Ctrl-C,
a hangup or a TERM during a command interrupts the command first and taracode quits once it has stopped; a second
signal moves to TERM, a third to KILL (Ctrl-\ goes straight to KILL); a signal taracode started with ignored (a
nohup, a background job) stays ignored. The policy classifier knows setsid as a wrapper, like nohup, and
names a background job for what it is.