0.8.4: a full review and security audit

A full code review and a security audit of the whole toolkit, with about 80 fixes.

lms get cervezagua/agent-toolkit

lms get cervezagua/agent-toolkit-loop

Security

• Programs in a cloned project. Git, the shell and other programs now start from their place on your PATH, never from a file with the same name in the project folder. Windows looks in that folder first, so a repository shipping its own git.exe could have run it when a chat opened.
• The git snapshot that opens each chat. It runs without asking, so it now stops a repository's own git config from starting a program (core.fsmonitor).
• Instruction files. AGENTS.md and CLAUDE.md aren't read through links, so a cloned repository can't pull in a file from elsewhere on your machine.
• Memories. memory_save gets the same injection scan as saved skills, because a memory reaches every later chat.
• Secret redaction. Secrets starting with "my" or "your" (MyS3cret!2024) are now hidden.
• Other writes. pdf_to_images follows the file tools' write rules. /resume refuses a linked or oversized note. A task left from an earlier session isn't killed by an old process ID.

Fixes

Shell

• A failing command run in a subfolder reports its real exit code; it used to report 0.
• A bash command ending in a comment no longer hangs.

Files

• Edits refuse files that aren't UTF-8 instead of damaging them.
• Editing a symlink updates the real file.
• Undo can't overwrite changes you made yourself.

Reply loop

• A tool call with broken JSON no longer ends the reply.
• The tool-output budget can't get stuck at zero.
• Remote models get the in-reply context checks.
• Results that arrive while the model is still writing are counted, so a reply can't overflow the context.

Web

• fetch_url names a nearby working page after a 404.
• Pages are read in their own charset.
• DuckDuckGo links are decoded correctly.
• One slow page no longer fails a whole research run.

Chat start

• A chat that opens with /doctor still loads AGENTS.md.
• An unreadable memory folder no longer stops your message from sending.

Tested

• 1071 tests pass on Windows, macOS and Linux.
• Long tasks on a 16k context, with agent-toolkit-loop: one local model went from 13/15 to 14/15, and the other stayed at 15/15.

0.8.3: /doctor

Type /doctor in a chat to check its setup. The model shows a report built by agent-toolkit and says what to fix first.

lms get cervezagua/agent-toolkit

New

/doctor checks, each line marked ✓ fine, ~ worth knowing, or ✗ needs fixing:

• The context: how much of the model's context the tools' descriptions take on every message. It's counted with the loaded model's own tokenizer, across every plugin enabled in the chat. Over half leaves little room for the work.
• Duplicate tools: tools offered by two plugins, and the five older plugins agent-toolkit replaced, if they're still enabled.
• The reply loop: which plugin runs the reply, and whether agent-toolkit-loop would help on a small context.
• The Project Folder and AGENTS.md.
• Programs the enabled groups need: ripgrep, the shell, Python, git, the GitHub CLI and yt-dlp, each with what you lose without it.

It changes nothing. To measure the tools, every enabled plugin builds them, as at the start of a reply. A check that can't get an answer says so rather than guessing, and never holds up your message.

Fixed

The README's tool-size figure: counted exactly, every group's tools take about 7,700 tokens with one current model, not the 9,600 estimated before.

Tested

985 tests pass on Windows, macOS and Linux.

0.8.2: summaries that pay for themselves

On a small context, agent-toolkit-loop summarizes earlier messages only when that makes the prompt smaller.

lms get cervezagua/agent-toolkit-loop

lms get cervezagua/agent-toolkit

Fixed

• A summary no longer makes the prompt bigger. The record that replaces earlier messages always holds your words, a list of the steps taken and the summary. When the messages it replaced were short, as on a 16k context where tool descriptions take most of the room, that record was bigger than they were. Before asking the model for a summary, the loop now measures the record and keeps to hiding old tool output when a summary wouldn't help. A first summary is still tried whenever it could save even a few hundred tokens, since those decide whether a reply fits.
• No repeated summaries for nothing. A summary that still comes out bigger is dropped, and its size stops the loop asking again on the next message.

Tested

969 tests pass on Windows, macOS and Linux. Long tasks on a 16k context, with agent-toolkit-loop:

• one local model went from 9/15 to 13/15;
• the other stayed at 15/15.

0.8.1: long tasks in a small context

Long tasks fit a small context better. On a 16k context, a task that writes a whole program used to fail for every model tested. Now one of them passes it every time.

lms get cervezagua/agent-toolkit

lms get cervezagua/agent-toolkit-loop

Fixed

• Earlier thinking no longer fills the context. Within one reply, LM Studio sends each round's thinking back to the model, for models whose template keeps it. On a small context that alone could fill it. agent-toolkit-loop now leaves earlier thinking out, and when a reply is filling the context, it rebuilds the prompt without it once.
• A big tool result can't overflow the context. One large web page or PDF could push a nearly full reply past the context, and LM Studio refused the request. agent-toolkit-loop now cuts each result to the room actually left.
• A reply cut off with nothing left to hide is asked once to go on in smaller steps, and told how much room there is.

New

• write_file can append (append: true), so a model can write a long file in parts. It refuses files that aren't UTF-8 text, so they aren't damaged.

Good to know

With every group on, the tools' descriptions take about 9,600 tokens on every message: more than half of a 16k context. On a small context, switch off the groups you don't need.

Tested

967 tests pass on Windows, macOS and Linux. Evals on two local models, with agent-toolkit-loop:

• long tasks at 16k went from 11/15 to 15/15 on one model;
• the web tasks that overflowed now pass 6/6 on both.

v0.8.0 - /handoff and /resume

Two commands carry long work into a new chat. A long chat gets slower with every message, and a fresh one starts from nothing.

lms get cervezagua/agent-toolkit

New

• /handoff has the model write down where the work stands, with exact file paths, commands and error messages:

  • the goal;
  • what's done, file by file;
  • the current state;
  • the decisions, and why;
  • the open problems;
  • the next steps.

  It saves the note to .agent-toolkit/handoff.md in your project and shows it in the reply. Add a focus if you like: /handoff mention the flaky test.

• /resume, in a new chat with the same Project Folder, loads that note. The model says where things stand and carries on. Add an instruction if you like: /resume and then add the tests.

Both chats need the same Project Folder: without one, each chat has a folder of its own.

Safety

The note is written by a model, possibly after reading web pages meant to steer it, and a cloned repository could ship one. So /resume loads it as information from an earlier chat, never as instructions:

• It is fenced, so the note can't pose as the plugin's own text.
• It is scanned like AGENTS.md. If it looks written to steer the model, it isn't loaded, and the status line says why.
• Secrets in it are hidden.

A project's own command file can't take the name /handoff or /resume.

Tested

960 tests pass on Windows, macOS and Linux.

v0.7.3 - Always allow, context cut-offs, and security fixes

agent-toolkit-loop now follows LM Studio's Always allow, keeps a reply going when a full context cuts it off, and follows up a reply that ended inside the model's thinking. A security review of everything since 0.7.0 is fixed in this release.

lms get cervezagua/agent-toolkit
lms get cervezagua/agent-toolkit-loop

Fixed

• Always allow works with agent-toolkit-loop. LM Studio's own loop skips the confirmation for tools you chose to always allow, or for every tool when confirmation is switched off. agent-toolkit-loop now does the same. It used to ask about every call.
• A full context no longer ends a reply mid-sentence. When the context fills while the model is still writing, older tool output is hidden and the model carries on from where it stopped. If there is nothing left to hide, a notice says the context is full. Between rounds the loop also keeps more room free (15% of the context), which a model that thinks at length needs.
• A reply that ends inside the model's thinking (some reasoning models plan a command in their thinking and stop there) gets one follow-up asking for the answer, so the chat doesn't seem to pause.

Security

• The file and document tools can no longer change LM Studio's own files: its settings, credentials, installed plugins, chats, and the pointer to its folder. A model could otherwise have switched approval off by writing LM Studio's settings. The folders chats work in stay writable.
• Always allow is read once per reply, from LM Studio's folder as found when the plugin loads, so a change made during a reply cannot approve the rest of it.
• Errors from the toolkit's own tools are checked for secrets before the model sees them. Since 0.7.2 those errors reach the model, and a failed git URL can carry a token.
• Always allow on run_command is, in effect, approval for everything, because a shell command can change any file, LM Studio's settings included. SECURITY.md now says so.

For developers

• npm run chat-doctor checks a saved LM Studio chat for replies that look stuck or broken:
  • an approval with no buttons;
  • a tool card left on "calling tool";
  • a call with no result;
  • a reply cut off, or made only of thinking.
• agent-toolkit-loop's tests run its chat output against a fake LM Studio that records what LM Studio would save, and apply the same rules.

Tested

957 tests pass on Windows, macOS and Linux. Always allow, the context cut-off and the thinking follow-up were checked by hand in LM Studio.

v0.7.2 - tool errors reach the model, failed replies end cleanly

Two fixes for agent-toolkit-loop: a failing tool no longer freezes the chat, and a reply that fails always ends cleanly.

lms get cervezagua/agent-toolkit-loop

Fixed

• A tool that fails gives the model its error. A script that failed in js-code-sandbox's run_javascript (Node's require in Deno, say) made the tool throw. LM Studio's own loop turns that into a result; through agent-toolkit-loop it ended the whole reply. Now any tool that throws, from any plugin, answers Error: ..., and the model can try another way.
• A failed reply ends cleanly. When a reply failed, its tool card stayed on "calling tool". The chat then showed a reply that looked busy, with Stop greyed out and no way out but to regenerate. Now every open card is closed with the reason, a status line says why the reply stopped, and the reply finishes like any other. Cards are also closed when you stop a reply.

Tested

934 tests pass. Checked by hand in LM Studio: a failed run_javascript reached the model, which switched to run_command, and the reply went on through 9 approved calls.

agent-toolkit has no code change; it's republished so the version it reports matches this release.

v0.7.1 - tool approvals work with agent-toolkit-loop

A fix for agent-toolkit-loop: tool calls that need your approval show their Approve and Deny buttons again.

lms get cervezagua/agent-toolkit-loop

Fixed

• Approving a tool call works with agent-toolkit-loop on. In 0.7.0 the loop asked LM Studio to confirm each call, but LM Studio's chat shows the Approve and Deny buttons only on a call's status card. LM Studio's own loop creates that card for every call; agent-toolkit-loop didn't. A reply then waited for an approval that couldn't be given. Each call now gets a status card as LM Studio's own loop makes it: generating, waiting for you, running, then done, failed or declined.

Tested

933 tests pass. Checked by hand in LM Studio: the buttons show, and approving a call lets the reply carry on.

agent-toolkit has no code change; it's republished so the version it reports matches this release.

v0.7.0 - long chats with agent-toolkit-loop, commands and /rewind

A companion plugin keeps long chats going at small contexts, and the toolkit gains commands, checkpoints with /rewind, and checks after every edit.

lms get cervezagua/agent-toolkit
lms get cervezagua/agent-toolkit-loop

New: agent-toolkit-loop

A second plugin, enabled next to agent-toolkit, that takes over LM Studio's reply loop for long chats. Switch it off and LM Studio's own loop is back as before. See its README.

• Long chats keep going. When the prompt nears the context limit, old tool output is hidden first, with no model call. Only if that isn't enough does the model write a running summary. Your own words and a list of what happened, read from the tool calls, are kept verbatim beside it. The whole chat is kept in a transcript the model can read back.
• A long reply makes room as it goes, instead of dying in the middle of a tool call.
• /btw side questions are answered from the chat without tools, and kept out of the task.
• A model stuck in a loop is stopped.
  • A call that already failed twice exactly as written is refused.
  • So is a call that got the same result twice with nothing changed in between.
  • A garbled tool call becomes a message the model can fix, instead of ending the reply.
• The task list is restated at the end of the prompt every few messages, where small models pay most attention.
• Tool calls still ask for your approval, through LM Studio's own confirmation.
• Remote models served through remote-lmstudio or openai-compat-endpoint use the same loop. Set Usable Context to the model's real size.

New in agent-toolkit

• Commands:

  • /review reviews your uncommitted changes.
  • /test runs and fixes the tests.
  • /commit commits your changes.
  • /init writes an AGENTS.md.
  • /help lists them.

  A project can add its own as Markdown files in .agent-toolkit/commands/.

• /rewind lists the messages that changed files, and /rewind N puts those files back as they were. Changes made by shell commands aren't tracked.

• Edits are checked. After a JavaScript, Python or JSON file is written, its syntax is checked and any problem is in the same tool result.

• task_wait waits for a background download or build to finish, or for a line in its output, without spending tokens while it waits.

• Skill triggers. A skill with triggers: in its header is loaded when your message names one of them.

• think, an optional scratchpad tool for models without a reasoning mode. It's off by default.

Fixed

• Multi-line commands no longer hang in the persistent PowerShell session, such as a multi-line python -c "...". They used to wait until the timeout.
• A failed program is no longer hidden in PowerShell when a later statement succeeds (node app.js; echo done). The result now says so.
• write_file says "No change" when the file already has that content, which breaks a common loop.
• SearXNG's "JSON format is off" error (HTTP 403) now says how to switch it on.

Tested

932 tests pass on Windows, macOS and Linux. The eval suite ran on a local model (xortron-xprt3-fast, 3 runs per task):

Suite	Without the loop	With agent-toolkit-loop
Full suite	73/75	73/75
Long tasks at a 16k context	9/15	11/15

The loop and the new commands were reviewed by separate code and security review passes, and their findings were fixed.

The standalone plugins (feed-reader, page-reader, video-transcripts) haven't changed since 0.6.0.

v0.6.0 - runs on SDK 2.1, Brave key under Search Backend

The plugin now runs on LM Studio SDK 2.1, which fixes a hang with newer versions of LM Studio. The Brave Search API Key moved next to the Search Backend setting.

lms get cervezagua/agent-toolkit

Fixed

• Model lists in the settings are filled again. With newer LM Studio, SDK 1.5 couldn't read LM Studio's list of downloaded models (it now includes a model type 1.5 doesn't know), and the request never returned. The plugin waited out a 5-second timeout at startup and left the model choices empty, such as Vision Model and Research Model.
• The vision tools no longer hang when no vision model is loaded. They answer straight away and name the vision models you could load.

Changed

• Brave Search API Key is now under Search Backend, set per chat. It used to be a shared setting at the bottom of the panel, far from the setting that needs it. If you use Brave, enter your key again in the new field: the old one isn't carried over. The "no key" error and web_doctor point to the new place.

Tested

861 tests pass on Windows, macOS and Linux. The plugin was checked in LM Studio with a local model: files and shell, several edits in one call, todos, git, a refused path, the vision fallback, web fetching and the settings panel.

The standalone plugins (feed-reader, page-reader, video-transcripts) are republished with the same code as this release.