Skip to content

Emulo v0.6.6: the mine-then-install path works end to end

Choose a tag to compare

@ohad6k ohad6k released this 25 Sep 09:07
· 26 commits to main since this release

Fixed

  • The documented mining path stopped at the install step. RUN_ME.md asks your agent to write emulo-out/you.md and then prints emulo --install emulo-out/you.md --target claude first. The claude and codex targets install the profile as a skill, which needs name and description frontmatter, and RUN_ME.md never asked for any, so a you.md written exactly as instructed was refused with "profile must start with exact name and description frontmatter fields". Found by following the documented flow end to end while writing a user guide. --install now adds name: you and a default description to the installed copy when the file has no frontmatter, and prints a note saying so. Your you.md is not modified. A file whose frontmatter already has name and description is installed as written, whatever the name says. A frontmatter block that is not closed, lacks name or description, has a line that is not key: value, leaves a value empty or repeats a key is still refused, now with a line that says how to fix it. The fix sits in the installer rather than in RUN_ME.md because every other target strips frontmatter anyway, so asking the agent for YAML it only needs for two targets would add a step it can get wrong, and a you.md written from an older RUN_ME.md installs too. A UTF-8 byte order mark, blank lines, or an indent before the opening --- no longer hide the file's own frontmatter, so such a file is installed with its own block instead of having the default stacked on top. The same blank line had also let a profile's frontmatter leak into AGENTS.md, GEMINI.md, the OpenCode rules file and the Cursor rule in earlier releases; it is now stripped there too.
  • Some secrets passed through redaction. Anthropic keys (sk-ant-...), Google API keys (AIza plus 35 characters) and the password in a connection URL (postgres:// with a user:secret pair before the host, and the same for mysql, mongodb, redis, amqp, https or any other scheme) were written to the corpus and chunks as typed. A password on a host with a dot was only removed by accident, because the email rule took password@host.tld and the host with it; on localhost it survived. All three are now redacted. The URL rule keeps the user and host and runs to the last @ before a space, /, ? or #, so a raw p@ss is covered while http://localhost:3000?next=a@b.com stays a port and a query. The same check found that OpenAI project, service account and admin keys (sk-proj-..., sk-svcacct-..., sk-admin-...) also passed, because the plain sk- rule stops at their first dash; those are redacted too. The Anthropic and OpenAI project rules only match at the start of a word, so a kebab-case branch or tool name that contains a prefix mid-word is left alone. A review before release found that shape ten times in one real history.
  • The browser scan at /scan had the same gaps. It runs its own port of the redaction table, and it now has the same rules in the same order.
  • /scan used a regular expression feature that Safari before 16.4 cannot parse. Its phone rule, live since at least 0.6.4, used a lookbehind, and one pattern the browser cannot parse stops the whole script, so the scan would not load there at all. Found by a review that required no lookbehind in the page. The phone rule and the new rules now keep that boundary another way, with output identical to Python's, and a test fails if the page contains a lookbehind. Not checked on a real Safari older than 16.4.
  • The redaction count did not say what it counted. secrets/PII redacted: 2 read as two items, but it is the number of messages that had anything redacted. The line is now messages with secrets/PII redacted: 2, and the stat on /scan reads "messages redacted".
  • An empty CODEX_HOME or XDG_DATA_HOME searched the current directory. Set but empty, either one made discovery look in relative sessions, archived_sessions and opencode folders under wherever you ran emulo, and mine whatever it found there. Empty now means unset.
  • --card without a card pointed at a flow that no longer exists. It said to paste MINING_PROMPT.md into your agent. The RUN_ME.md flow writes you.md and no card; only the agent mining flow (run emulo through the bootstrap, or emulo:mine) writes card.json, into the active profile. The message now says that, and names emulo plugin status to find the card and emulo --card <card_path> to render it.
  • Docs that disagreed with the code. SECURITY.md listed three sources where Emulo reads five (OpenCode and Antigravity were missing) and said redaction covers "common API keys"; it now names every pattern, each one tested, and a case it misses. The README said only .jsonl messages are mined (OpenCode is SQLite and JSON), that emulo --dry-run prints the read-only plan (it prints counts; the JSON plan comes from emulo plugin preflight on the agent path, and plain emulo writes chunks with no approval step because it makes no model calls), that --card works after mining (not after the RUN_ME.md path), and that the MCP server serves the profile you mined locally (it serves the profile activated by the agent mining flow, so a you.md installed from RUN_ME.md is invisible to it). The --path help said .jsonl only. The MCP tool description named work, design and write but not video, which the tool has accepted since 0.4.0. site/llms.txt and the working profile guide at /agent-profile named 0.6.2 as the published CLI. The guide also told plain CLI users to review a mining plan that path does not have and said the MCP server serves any local profile, and the home page said the RUN_ME.md step shows the plan and cost before it starts. All three now describe the two paths the way the README does.

Verified

  • The documented path, run with the installed emulo command in a clean home directory: emulo --source codex --chunks 1 on a history holding an Anthropic key, a Google API key and a Postgres URL with a password wrote one chunk with [ANTHROPIC_KEY], [GOOGLE_API_KEY] and postgres://app:[REDACTED]@localhost:5432/app_dev, and none of the three secrets appears in any output file. A you.md written exactly as the generated RUN_ME.md specifies, with no frontmatter, installed with all three commands RUN_ME.md prints and with the other three targets it lists (agents, gemini, opencode): six exit codes of 0. emulo verify traced both of its quotes. The same run had CODEX_HOME and XDG_DATA_HOME set to empty and a sessions/ folder in the working directory, and nothing from that folder was read.
  • 12 new Python tests. Ten failed before the fix: the full RUN_ME.md path (mine a fixture history with the real CLI, write you.md as instructed, run every install command it prints plus the other listed targets, all must succeed), the note printed when frontmatter is added, a BOM before valid frontmatter, frontmatter after blank lines, whitespace-only lines or an indented marker for both skill targets, the same file installed to AGENTS.md without its frontmatter, Anthropic, Google and OpenAI project keys with prose that must survive ("the AIza prefix marks a Google key"), connection URLs across schemes with URLs that must be left alone (no password, a port, a path with a colon), the count label on a message holding three redacted items, empty CODEX_HOME and XDG_DATA_HOME through both the module and the real CLI, and the MCP description naming every domain its enum accepts. Two are guards that passed before and are not evidence for a change: emulo plugin status answering with no profile, and a timing test that the URL rule scans a 200,000-character token once. Three existing tests changed: the --card hint test now also requires the new hint and failed before the fix, a card render test that required the old MINING_PROMPT.md hint now forbids it, and the dry-run test expects the new count label.
  • tests/test_redaction_table.py holds one realistic fake sample for every pattern SECURITY.md names, 48 rows, each of which must come out as its documented placeholder, plus the documented miss, which must still pass through. A third test reads SECURITY.md and fails if a pattern it names has no row or a row is not named there, and a fourth holds six kebab-case words containing a key prefix, which must come through untouched. The kebab test failed before the left boundary was added. The browser scan runs the same rows, read from the same Python table. To check that these tests hold the rules in place, each of the 17 rules was removed in turn from emulo.py and from /scan, five alternatives were narrowed in both (svcacct, admin, gh[pousr], xox[baprs], api[_-]?key), the left boundary was taken off each new key rule, and the ? and # guard on the URL rule was removed: 50 mutations, and a test failed for every one.
  • 5 new site tests: no lookbehind anywhere in the page, the URL rule staying fast on a 600,000-character input, the browser redaction for the new patterns, the full SECURITY.md table, and the page's own folder-picker handler, fed a stubbed directory, running a scan that renders "messages redacted". The byte-for-byte parity test against Python now also runs 23 credential samples, positive and negative. The lookbehind test, the new-pattern test and the parity test failed before the change. The timing test was checked the other way: with the URL rule swapped for a plain global pattern with no boundary, it took 131 s and failed, and with a capture-group boundary in place of the scanner, the parity test failed on a://u:p@b://v:q@h, where Python redacts both passwords and that form misses the second. The table test was written after it, so its evidence is the mutation run above rather than a failure before the fix; the same holds for the Python table test.
  • Full suite on Python 3.11 with the [pro] extra: 568 tests, 5 skipped, 0 failures. On Python 3.13 without cryptography: 546 tests, 13 skipped, 0 failures. On Windows the symlink tests skip, and the two release pin tests skip until the v0.6.6 tag exists.
  • Site tests after a clean npm ci: 46 tests, 0 failures.
  • The v0.6.6 bootstrap runtime pins emulo.py to SHA-256 9771458e43c74c9b (prefix); MINING_PROMPT.md is unchanged at ee22077c2cda3c1c (prefix). Full digests in .agents/skills/emulo/runtime.json.
  • Not fixed, found while checking the redaction docs: AWS_SECRET_ACCESS_KEY=... is not redacted, because the assignment rule only matches a name ending in api_key, secret, token, password or passwd. SECURITY.md now says so. A connection URL password holding a raw #, /, ? or space, or following a user name with a raw @, is not redacted; none of those is valid there unencoded, the encoded forms are redacted, and SECURITY.md lists them with a test for each. The email rule's time grows with the square of a token's length, so one 100,000-character run of letters with no spaces takes about 7 s to redact; this release does not change it. Python and the browser scan disagree on what counts as whitespace in a pattern: Python treats the control characters U+001C to U+001F as whitespace and the browser does not, and the browser treats U+FEFF as whitespace and Python does not, so a message holding one of those characters next to a secret can redact differently in the two.
  • Not verified: Python 3.8, because no 3.8 interpreter is installed on the release machine, and pip install emulo==0.6.6 from PyPI, because the version is not published at the time of writing.

End to end, from a fresh install

$ emulo --source codex --chunks 1
messages with secrets/PII redacted: 2                     exit 0
  the eval key is [ANTHROPIC_KEY] and maps uses [GOOGLE_API_KEY]
  DATABASE_URL=postgres://app:[REDACTED]@localhost:5432/app_dev fix the migration
you.md written as RUN_ME.md specifies, no frontmatter
$ emulo --install emulo-out/you.md --target claude    note: adds default frontmatter   exit 0
$ emulo --install emulo-out/you.md --target codex     note: adds default frontmatter   exit 0
$ emulo --install emulo-out/you.md --target cursor --repo .                            exit 0
$ emulo --install emulo-out/you.md --target agents --repo .                            exit 0
$ emulo --install emulo-out/you.md --target gemini --repo .                            exit 0
$ emulo --install emulo-out/you.md --target opencode                                   exit 0
$ emulo verify emulo-out/you.md   -> 2/2 quotes traced                                 exit 0

On 0.6.5 the same you.md is refused at the first install command.