Skip to content

llx 0.5.0

Choose a tag to compare

@rsaryev rsaryev released this 17 Nov 11:01
· 51 commits to main since this release
09b4b92
  • LLX_RECORD=<dir> keeps each session's run whole as JSON lines: every call and its whole output as the model read it, before any elision, the model's words, each turn's usage, each elision and the answer, an agent's events named by it. Off by default; for reading a session again (what an elision took, what the agent read again after one).

  • The prompt cache breaks less often in a long task: a pass of the history now elides down to 40% of llx's ceiling instead of 60%, so one cache miss buys about twice as many turns. Each pass rewrites what follows it in the cache (about $0.41 on claude-sonnet-5-5 at 170k tokens), and paid itself back only after some 55 requests; modelled on a task that grows 1k to 3k tokens a request, the deeper pass makes 40% fewer breaks and costs 20% to 26% less, its requests being smaller too (22 breaks instead of 26 in the 300-turn simulation of the tests, whose turns are large for its ceiling). What goes is old outputs, each with a pointer to get it back.

  • llx spend [ID]: what a session spent, without reading its file. By task, by agent and by model: requests, input, the share read from the prompt cache, what was written to it, output and money; then the prompt cache's misses, each with its likely cause (the context elided, a change of model, a pause past the cache's five minutes, a new conversation) and what they cost together, and the dearest requests. /usage in a session ends with the same report of its earlier tasks. A session's file now keeps, for each model turn, the agent whose turn it is and the model that answered, the tokens written to the cache, and each elision as a step: before, an agent's turns were the main session's steps with no name (its own file had none), so a session's total held its agents' but no one could tell them apart.

  • A miss of the prompt cache is noted as it happens: ✻ The prompt cache missed (policy): 182k tokens sent uncached, $0.492, after the context was elided, in the lane of the agent it is. An agent's elisions are noted in its lane too; before, they were not shown at all.

  • The history no longer elides an output at every request ("The context neared the model's limit; elided to save it: 1 outputs" at each turn), which rewrote the conversation from there and missed the prompt cache each time. Two causes, seen in a session of 380 turns:

    • What no stage elides grows in a long task: the model's reasoning, carried back with each turn and counted by the provider (on gpt-6.1-sol about 0.7 token for each it wrote). Once it passed 60% of the ceiling, a pass could not get there, the gate stayed open, and each request elided the output just past the protected tail. Now a pass that could not reach its target opens again only once the conversation has grown 30% of the ceiling past where it left it, and then elides all that grew at once.
    • Past llx's ceiling (272k on a window of a million), the tail itself gave way: the model kept its last two outputs and lost the instruction files, at every request ($1.07 and $1.35 requests with nothing read from the cache). The tail now gives way only past the model's own limit.
    • After a switch of model (gpt-6.1-sol to claude-sonnet-5-5), the estimate counted the sealed reasoning of the other model's turns, which no provider sends to another: a conversation of 200k tokens was taken for more than the ceiling, and every request elided (up to $0.48 a request instead of about $0.04). A conversation now lets go of another model's reasoning as the model changes (it costs no cache: a provider's is its model's).
  • Old turns' reasoning goes first when a long task nears the limit, before any output it read: OpenAI's sealed reasoning items and Anthropic's thinking blocks, which grow with every turn of a task and no stage could elide (on gpt-6.1-sol a task of 218 turns carried about 0.7 token of it for each one written). The newest turns keep theirs, as the APIs need; the words and calls of a turn stay in their order. Hosts over chat completions keep theirs (DeepSeek and Kimi ask for their reasoning back within a task, Gemini for its signatures), as AI SDK's pruneMessages leaves a call's own data.

  • Examples: deps.yml, a dependency update read: when a pull request changes a manifest or a lockfile (Dependabot's, Renovate's or yours), llx works out which packages move, reads what their new versions changed, searches the code for what that touches and runs the tests, then one comment it updates says safe, review or breaking with the lines that need a change. For Dependabot's pull requests the key must also be a Dependabot secret.

  • --schema: a result made just as an agent answers (or a background command or a monitor ends) waits for that answer, which reaches the model first; before, the data could be taken without it, the answer left unread.

  • --schema's format: "regex" holds a pattern to the whole of ECMA-262's syntax even where it has what is not checked here: a back-reference to a group the pattern does not have ((?=a)\1, \k<n> with no group n), or what is invalid after a lookaround, a back-reference or a property ((?=a)(), is invalid; before, the first unchecked part hid it. A long pattern is judged in one pass however many back-references and lookarounds it has, and a lone surrogate at a class range's end ([z-\uD800]) or a group name spelled with an escape or a combining mark stays valid.

  • A reply from OpenRouter or another OpenAI-compatible host that carries "error": null beside its completion is the completion; before, it was taken for a failure and asked again.

  • A duration from 9,950 to 9,999 ms reads 10s, not 10.0s.

  • A token count from 999,950 to 999,999 reads 1.0M, in llx spend, /usage and the footer; before, it read 1000k or 1000.0k.

  • --schema's format: "regex" checks a valid pattern with no allocation when it holds a lookaround, a back-reference or a \p{...} too, as the schema package promises for valid data (\p{L} 155 ns to 65 ns).

  • With an OpenAI API key, a reply that fails, is withheld by the content filter or runs past the window names OpenAI; so does a 5xx or a 429 that outlasts its retries (OpenAI answered 503: …); before, they named ChatGPT, as with a sign-in, or no host at all.

  • An agent that calls agents, ask or jobs is told it works with bash and apply_patch; before, the refusal named grep too, a tool llx no longer has.

  • The interface has no indent where text is read or copied: an answer starts at the left edge with no bullet and no hanging indent (a long line wraps flush), a quiet line from llx (a note, a table, the footer under the prompt, a list's ↑ 3 more) too, and the list of sessions, agents and commands under the prompt has no pointer column: only the row the keys are on has its ❯. What you typed keeps its ❯, and a call keeps its bullet and its └ steps.