Skip to content

mem toolbox

Jump to bottom
Valerio edited this page Oct 16, 2026 · 2 revisions

mem-toolbox gives your agent a shared memory of tooling: which command-line tool, library or runtime to use for a given task, and whether it is actually installed on the machine the agent is running on. It ships an opinionated catalogue for media, documents and data, lets your project override it with its own entries, and keeps what exists on each computer in one machine file outside every repository, shared by every project on that computer, so one developer's setup never misleads another developer's agent. It is read-only by default; the only things it writes outside your knowledge base are that machine file, once you have confirmed it, and a package into a dedicated agent environment, after you approve it.
Version 1.0.3
Commands MEM TOOLS, MEM TOOLS CHECK, MEM TOOLS SETUP
Executable content No. Every file it ships is Markdown that the agent reads
External actions Yes, two kinds: writing this computer's machine file, ~/.mem/mem-toolbox/host.md, once you have confirmed it, and an approved package install into a dedicated agent environment
Requirements None for the extension itself. Library entries need their runtime: Python 3.11+ or Node 22+

Why it exists

Without it, an agent re-derives the same two answers every session. Which tool should resize this image, or turn this data into a PDF? Is that tool even installed here, and where? It probes, it asks you again, it proposes a tool you already said no to last week, and when it finally finds a path it writes it down wherever it can: a catalogue page, references/, a daily log.

mem-toolbox keeps those two answers apart, and that separation is the point of the extension:

  • Which tool a project uses is portable knowledge. It belongs in the repository and is the same on every machine.
  • Whether a tool exists at a given path is true of one machine only, so it lives on the machine: one file per computer, outside every repository, stamped with the host and the date.

The knowledge base is committed and shared. If "ffmpeg is available" were recorded there as a plain fact, an agent on a colleague's machine would read it, skip asking, and fail. So availability is always tied to a host, and an observation made on another machine is treated as a hint about what to try, never as proof.

Keeping machine facts on the machine also means they are asked once per computer, not once per project. Where your portable tools live and which Python or Node environment the agent uses are answered one time, and a tool found while working on one project is known in all the others.

Absence is recorded too. A tool that is missing, or one you declined, is a persistent answer: the agent does not re-probe or re-ask about it every session.

How it works

Three catalogues, two places

Catalogue Answers Lives in
catalog/ which tool to use for a task the knowledge base, base layer: distributed with MEM, updated with the extension
custom/tools/ which tool this project uses the knowledge base, custom layer: yours
the machine file what exists on this computer, and where ~/.mem/mem-toolbox/host.md, outside every repository

The first two travel with the project. The third describes the machine and stays on it: one file per computer, per user, shared by every project on that computer. Because it is in no repository, it never produces a merge conflict and one machine's reality never overwrites another's. ~/.mem/ is the per-user folder MEM extensions use for machine state; mem-stats keeps its ledger there too.

Precedence is custom wins: a project entry overrides a catalogue entry, and the agent says so the first time it relies on the override in a session. This is deliberately the opposite of mem-commands. A catalogue entry is reference data; a local entry is closer to the truth about this project and this machine, so overriding it is a correction, not a hijack.

The machine file

The machine file has a short header (the host name, the OS, the tools root and the agent environments) followed by one section per tool, each with its status, path, observed version and the date and method of verification.

It is created before any tool is looked up, even when it has nothing to say yet. A path has to be recorded the moment a tool is first used, which is exactly when there would otherwise be no file to hold it; an agent under pressure would then put it wherever it could, and the separation would break on first contact. A header with no tool sections is a meaningful state ("this computer is known, nothing probed yet"), different from a missing file, which means "this computer has never been set up".

Looking up a tool

Catalogues are keyed by capability, not by tool name, because the question always starts from a task ("resize an image", "create a PDF"), not from a filename like imagemagick.md. A capability can have both CLI and library candidates; they compete in one list, in preference order.

Before an operation that needs a tool or library, the agent:

  1. reads custom/index.md, then catalog/index.md and the sub-index for the task's domain, and picks a candidate;
  2. reads the machine file for that candidate. If this computer has no machine file yet, it creates one first (see "Installing and first setup"), then acts on the tool's status:
Status in the machine file What the agent does
confirmed Uses the recorded path and invocation. Re-probes first when the operation is destructive, or the verification is old
outdated Uses it if it works, and says once per session that it is below the entry's minimum version
unavailable or declined Does not ask again. Offers the next candidate
no section yet Probes the tool and records the result
  1. for a library, does the same for its runtime first, then for the library inside the agent environment;
  2. if no catalogue has a candidate, asks you. If you agree on a tool, it writes the project entry (with your confirmation) and then the machine-file section.

Every machine-file section must point to a catalogue or project entry by id. A tool found in neither gets a project entry first, so the host data never degenerates into an undocumented list of paths.

Libraries, runtimes and agent environments

"Create a PDF" can be answered by the typst CLI or by the Python library ReportLab, so libraries are first-class entries. A library is looked up in two steps: its runtime (python or node, each with a minimum version), then the library inside that runtime's agent environment. A library whose runtime is missing is not probed; the agent reports the runtime instead. Library probes use the environment's own interpreter, not whatever is first on PATH: Python checks that the import works and reads the installed version from package metadata; Node uses npm ls against the environment folder. Entries record the package name separately from the import name, because they often differ (Pillow imports as PIL, python-docx as docx), and guessing one from the other is how a probe reports a present library as missing.

The agent keeps one environment per runtime, per computer, never one per project, and records it in the machine file. Whether it is dedicated or the system interpreter is your decision: the agent never presets the answer, and states the trade-offs every time it asks:

Dedicated System
Every project on this computer shares the same tools and versions yes yes
Isolated from the OS and other software yes no
Needs admin rights no often
Undone by deleting one folder uninstalling package by package
The agent may install into it, with approval yes no, it only proposes
After an interpreter upgrade the folder remains; rebuild it for the new version packages may be dropped with the old interpreter

The first row is the same in both columns on purpose: sharing across projects comes from having one environment per computer, not from using the system interpreter. What neither gives is the same versions on another computer; that is what the min version in each entry is for. A dedicated environment lives under your tools root, in envs/python-<X.Y>/ (a venv, tied to one interpreter version) or envs/node/ (an npm prefix folder). With tools root: none, the agent asks where the environment should live and may suggest a location under %LOCALAPPDATA% or $HOME. Note that Debian 12 and later, Ubuntu and Homebrew mark the system Python as externally managed (PEP 668), which refuses pip install outside a venv; the agent says so when it asks.

Where tools live on disk

Portable tools are usually archives someone unpacked, and their folder names bury the tool name behind a prefix and a version. The extension proposes a two-segment layout under your tools root:

<tools root>/
  <tool-id>/
    <version>[-<variant>][-<target>]/
Unpacked as Becomes
oxipng-10.1.1-x86_64-pc-windows-msvc oxipng/10.1.1-x86_64-pc-windows-msvc
vips-dev-8.18 vips/8.18-dev
realesrgan-ncnn-vulkan-v0.2.0-windows realesrgan/v0.2.0-ncnn-vulkan-windows
ffmpeg-9.0.1-essentials_build ffmpeg/9.0.1-essentials

The first segment is the catalogue id, so a directory listing maps straight onto catalogue entries. The version leads the second segment, so builds sort in version order, and everything that varies per build (target, flavour, implementation) stays in it. Several versions coexist, and an upgrade adds a folder instead of overwriting one, so rolling back is a path change. The payoff is a concrete proposal: instead of "you could install ffmpeg", the agent tells you which folder to unpack the archive into. That is a sentence in a proposal, not an action. You unpack it; the agent then probes and records the result in the machine file. A tool installed by a platform installer or a package manager keeps the path the installer gave it.

The shipped catalogue

The catalogue is a default toolchain, not an encyclopedia. Its value is preference and provenance: which tool to reach for, and where it comes from. Every licence was read from upstream and dated in the entry.

Domain Entries
Runtimes python, node
Media imagemagick, vips, oxipng, resvg, realesrgan, ffmpeg, exiftool (CLI); pillow, svgo (libraries)
Documents qpdf, pandoc, typst (CLI); reportlab, pypdf, pdfplumber, python-docx, openpyxl, python-pptx (libraries)
Data duckdb, jq (CLI); pandas, matplotlib (libraries)

Entries are short and practical: the canonical invocation, the flags that matter, and gotchas already paid for (for example, ffmpeg needs -y or -n or it can hang waiting for an overwrite prompt; openpyxl does not evaluate formulas). PyMuPDF and Ghostscript are deliberately absent because they are AGPL-3.0; a project can still add either as a custom entry, knowing the licence.

Installing and first setup

mem-toolbox is listed in the MEM extension manifest, so the agent can propose it when you use MEM TOOLS or when a task needs a tool it has no record of. As with any extension, it installs only after you confirm a request naming the files and the source URL, and it is loaded only once it is registered as active in your project's extensions/EXT.md.

The machine-specific part is set up once per computer, with MEM TOOLS SETUP. If you have not run it, the agent offers it the first time a task needs a tool on a computer with no machine file; if you would rather skip the wizard, it asks only the tools root question and writes the file with that. The wizard asks three questions, with the recommended option first and each choice explained:

  1. Where do portable tools live on this computer? Recorded as tools root. The agent must ask rather than guess; it may suggest a directory it has evidence for (one already holding a catalogue tool, or one on PATH) and show you that evidence. Valid answers are a directory, none (everything comes from installers or a package manager), or a directory that does not exist yet, which is recorded as intended but never created by the agent.
  2. Which agent environment for each runtime? For python and node: dedicated, system, or decide the first time that runtime is needed. The trade-offs above are stated, and nothing is preselected.
  3. Keep a copy in this knowledge base? No (recommended) or yes. This one is asked per project, because it is a property of the repository, not of the computer; see "Configuration".

Nothing is written before the end. The agent then shows the plan: the path and full content of the machine file and, if you chose the copy, the MEM.config.md line and the copy's path. One confirmation writes all of it, and that confirmation is also your approval for the extension to keep its machine file: from then on, its automatic writes go to that one file (and to the copy, if enabled). Once the machine file exists, its questions are never asked again on that computer, from any project.

MEM TOOLS SETUP can be run again at any time to change an answer. The current values are offered as defaults, tool sections are kept, and nothing is re-probed.

If a knowledge base already holds a host file for this computer (custom/installed/<host>.md, matched by hostname or by a label you recognise; the agent asks which one when several could match), the agent proposes importing it into the machine file with a single confirmation instead of asking the questions again. The tools root, the agent environments and every tool section are carried over. The file in the knowledge base stays where it is: with the copy enabled it becomes the maintained copy, otherwise it is a snapshot that is no longer updated, which you may keep or remove.

If the agent's client cannot write outside the repository (a sandbox, or a refused permission), the agent says so and shows you the file's path and content to save yourself. It never falls back to writing the machine file into the repository.

Commands

Command What it does
MEM TOOLS Read-only. Lists the known tools with their capability, availability on this computer, the agent environments, and stale or outdated verifications, from the machine file. It does not re-probe anything. Without a machine file it says so and proposes MEM TOOLS SETUP
MEM TOOLS CHECK Probes every runtime entry and every agent environment recorded for this computer, writes the results to the machine file (and to the knowledge base copy, if enabled), and reports what is missing or below its minimum version, with a proposal for each. It never installs, never reaches the network, and runs only when you ask, never at session start
MEM TOOLS SETUP The wizard for the machine file: tools root, agent environments, and whether to keep a copy in this knowledge base. Shows the plan and writes everything after one confirmation. Rerunnable to change an answer

Most of the extension's work happens without a command: the lookup and probing described above run whenever a task needs a tool.

Configuration

The extension has one option in MEM.config.md:

Option Default Effect
extensions_mem_toolbox_kb_copy false When true, every write to the machine file is mirrored into this knowledge base as extensions/mem-toolbox/custom/installed/<host>.md, so "on Windows hosts we use vips at this path" becomes team knowledge that survives the machine

The copy is a mirror, never the source: on its own computer the machine file is authoritative, and a copy from another computer is a hint. The <host> key is the hostname or a label of your choosing, and it is the only place the knowledge base records machine identity. Turning the option on changes MEM.config.md, so it needs your confirmation (MEM TOOLS SETUP asks). Turning it off leaves the existing copy in place; the agent tells you it is no longer updated and removes it only if you ask. The default is false because the copy is the only part of the extension that puts machine facts into a repository, and most projects do not need them there.

The other settings are answers recorded in files:

Setting Where Default
tools root header of the machine file none: asked once per computer, never guessed
<runtime> env (dedicated <path> or system), one line per runtime header of the machine file none: asked in MEM TOOLS SETUP or the first time that runtime is needed; absent means "not asked yet", never system
min version each catalogue or custom entry optional; decided offline from upstream support policy, never "latest"
Project tool choices custom/tools/<tool>.md and custom/index.md none; they override the catalogue

Per-tool paths in the machine file are written relative to the tools root, so a whole toolchain can be relocated by editing one line.

The core option extensions_allow_external_side_effects: true does not waive the install approval described below: that option relaxes confirmation for external actions in general, and a package install executes third-party code.

Safety and permissions

The extension ships no code, so extensions_allow_executable_content does not affect it. Its entries are stored instruction text: reviewed like any other file in your repository, unable to grant themselves permissions your configuration denies, and never carrying a secret. A tool that needs a token documents the variable's name, never its value.

Writes are split by what they represent:

  • the machine file is created only after your confirmation, because it is a file outside the repository; from then on it is written automatically, because each write records an observation (a version command answered);
  • the knowledge base copy, when enabled, is rewritten automatically from the machine file after each write to it;
  • a project tool entry needs your confirmation, because "this is what we use for X" is an editorial judgement;
  • enabling the copy changes MEM.config.md and needs your confirmation.

Installing is split by whether the change can be undone by deleting one folder:

Target The agent
A package into the dedicated environment, or creating that environment may run the install, after your explicit approval
A runtime, a CLI tool, or a package into a system interpreter (including npm install -g) only proposes the exact command or destination; you run it

An install approval must name the package, the version or constraint, the environment and the registry, and must say that installing a package runs its code. The package name must come from a catalogue entry's package field or be given verbatim by you, because a misspelt package name is not a typo, it is running someone else's code. The agent asks once per package per session, and a refusal is recorded as declined. After an install it probes the package and records the result like any other observation.

What it never does:

  • install anything on its own initiative, or put an install command inside an entry;
  • download, unpack, move or delete anything under your tools root (the one exception is envs/, and only with approval), or reorganise your tools directory;
  • create a tools root directory that does not exist yet;
  • write the machine file into the repository when it cannot write it in ~/.mem/;
  • reach the network during MEM TOOLS CHECK;
  • install a library your product depends on into an agent environment. Those belong in your project's own dependency files (requirements.txt, pyproject.toml, package.json) and in references/;
  • propose again an update you refused, until you ask.

Where its files and data live

extensions/mem-toolbox/        in your knowledge base
  index.md                     the specification
  TOOL.template.md             schemas for catalogue entries and the machine file
  catalog/                     the shipped catalogue (base files)
  custom/                      yours: never in the manifest, never touched by updates
    index.md                   project capability -> tool
    tools/<tool>.md            project tool entries
    installed/<host>.md        optional copy of a machine file

~/.mem/mem-toolbox/            per user, per computer
  host.md                      the machine file: tools root, agent environments, one section per tool

On Windows the machine file is %USERPROFILE%\.mem\mem-toolbox\host.md. It is in no repository and is never written by an update, so its paths may be absolute. custom/tools/ does not exist until the first project entry is written, and custom/installed/ only when the copy is enabled; both absences are normal. Updates write only the base files listed in the manifest, so your entries and any copy survive every update. The tools themselves and the dedicated environments live under the tools root you chose.

Limits and known issues

  • MEM TOOLS shows what was last observed. It does not re-probe; data refreshes when a tool is actually used, or when you run MEM TOOLS CHECK.
  • The catalogue is opinionated and small. Anything outside it goes in a custom entry, which then wins over the catalogue.
  • Licences can go stale. Each was read from upstream on a recorded date; re-verify before relying on one. For ffmpeg the licence of the binary build you installed can differ from the project's, and the reportlab entry covers only the open-source toolkit.
  • Some tools reach the network themselves. For example, a typst document that imports a package downloads it at compile time; the agent should say so before compiling.
  • The machine file is lost with the machine. It lives in no repository; that is acceptable because it can be re-derived by probing. Enable the knowledge base copy if your team wants that knowledge kept.
  • Sandboxed clients need a hand. A client that cannot write outside the repository cannot create the machine file; the agent shows you the content and path to save yourself.
  • Paths in the copy leak. A path such as C:\Users\<name>\... exposes a username, and a tools directory exposes internal layout, into a committed repository. In the copy, prefer environment-relative forms such as %LOCALAPPDATA%, %PROGRAMFILES% or $HOME, use a label instead of a hostname if you like, and keep per-tool paths relative to the tools root.
  • Another machine's record is not proof. A copy from a colleague's computer tells the agent what to try on yours, nothing more.

See also

MEM

Extensions

Repository · Specification

Clone this wiki locally