Skip to content

mindex 1.2.0

Choose a tag to compare

@silencespeakstruth silencespeakstruth released this 04 Aug 16:02
· 3 commits to master since this release

⚠️ REQUIRED IF THIS SERVER IS REACHABLE THROUGH A GATEWAY

[auth].enabled = true is mandatory behind deploy/gate/. The gateway admits on the
presence of a Bearer-shaped header — nginx cannot verify a signature and does not try —
so with authorization off, Authorization: Bearer x is admitted and served everything.

# 1. turn [auth] on in the server's config and restart, 2. then switch the gateway map
mindex mint-token --sub alice --project '*' --can search,research,index --days 30

That order is the only safe one: while mindex checked nothing, the API key was the only
thing guarding the remote path, and nothing breaks at the second step that the first did
not already break.

The shared X-Api-Key is gone, not deprecated. Remove it from every client; nothing
reads it.

A metrics scraper needs its own token/metrics is admin-scoped, so enabling
[auth] blanks every dashboard until the scrape carries one. Prometheus's
bearer_token_file must point outside $HOME (a unit with ProtectHome=true reads a
path there as absent, and the symptom is a scrape that stops rather than a permissions
message), and this is the one place --days 0 with its own --key-id is right.

Nothing here applies to an auth-off deployment, which stays byte-for-byte what it was.
enabled = false remains the default and now means exactly one thing: a server on a
trusted network that authorizes nothing.

FOR HUMANS

The server has a credential now, and it says what it may do and who holds it. Before
this, mindex authenticated nothing and a shared API key checked by a gateway stood in for
authorization. That key had no scope, no expiry, and no way to withdraw one holder without
withdrawing all of them — and it could not be fixed where it lived, because GET /projects
enumerates every project id in a response body, which no proxy filters without parsing
it. So the key is gone and a token replaces it: you mint one per person or per tool, name
the projects and the actions it may use, give it a lifetime, and that is the whole
decision. Nothing is stored server-side — no user table, no sessions — so there is nothing
to keep in sync and nothing to leak. Withdrawing a single holder is deleting one entry.

A token is finally worth pasting into an agent's context, which is the point of all of
it. mindex mint-token --project <this one> --can search,research --days 7 is a
credential that can read one project and cannot write, delete, or mint anything further —
so handing it to a coding agent is a decision you can actually reason about. VS Code will
issue one for you: a button in the Ask view mints a project-scoped, seven-day agent
token, offers read-only and read-and-write as the two obvious choices, puts delete behind
a deliberate tick, and never offers admin at all. The extension keeps its own token in
the OS keychain rather than in settings, because Settings Sync copies settings to every
machine you own. And a narrow token is a supported way to run the extension, not a broken
one — hand it a search-only credential and it disables Research and says which action is
missing, rather than failing at you.

A refused request finally says so. The gateway used to answer anything it did not like
by closing the connection, which is indistinguishable from a dead host — so a correctly
refused agent reported the whole deployment as broken. It now answers 401 with a real
error body naming what is served without a credential, while /.env and friends still get
silence. This was not theoretical: over three hours, one properly configured client
produced 840 closed connections on GET /health, and an agent that followed /llms.txt to
/config hit the same wall and was then banned for doing exactly what the document it had
just read told it to do. Of 1168 log lines the ban system called attacks, 1065 were this.

Point an agent at the URL and it can now work out the rest. /.well-known/mindex.json
is the service as data — identity, every endpoint, and the live config inlined — so
bootstrapping costs one request. It exists because the prose document was refused: /llms.txt
addressed the model in the imperative, which is the signature every modern agent harness has
to treat as prompt injection, and GitHub Copilot on a corporate machine declined to read it,
leaving that agent with nothing at all. The prose now argues instead of ordering, and JSON,
which has no register to object to, is the floor under it.

Research stopped punishing callers for not watching. POST /research streamed
events, always — and because disconnecting cancels a run, a caller that fired the request
and did not sit and read every frame spent the entire GPU budget, got nothing back, and saw
no error anywhere. It now answers one ordinary JSON body unless you ask for frames with
?stream=yes. Both shipped clients already ask, so nothing you use changes; what changes is
that the expensive mistake is no longer the default.

mindex-watch runs on Windows. It never had, since it was written — an unconditional
import of a Unix-only signal module — and nothing said so because nothing had ever tried to
build it there. The release workflow tried, and failed the job.

Plus a round of VS Code fixes for one repeated defect: a screen that disagreed with itself
after an action succeeded. Deleted reports that stayed on screen still ticked; a run that
finished and reached the panel through nothing at all, leaving a refuted report still
wearing its old trust badge; a cancel that then refused the reindex it was cancelled for.
None of them surfaced as an error.

The changelog has
the rest. Install the extension with
code --install-extension mindex-vscode-1.2.0.vsix --force.

FOR MACHINES

One removal — X-Api-Key — and it is not deprecated, it is gone. Two credentials where
one is strictly stronger is not defence in depth: the weaker sets the floor. Everything
else here is additive, and [auth].enabled defaults to false, so an unconfigured
deployment is unchanged.

[auth]. HS256, hand-written rather than taken from a crate so every copy of the secret
is owned (no Debug, zeroized, key file 0600 with O_EXCL) and algorithm confusion is
closed by construction — verify reads kid and nothing else before checking the MAC. Keys:
enabled, signing_key_file, max_token_days (90), leeway_seconds (60). The TLS key is
not reused. No denylist by design: revocation is expiry or deleting a kid.

The token is the mapping; there is no schema change. prj (dashless GUIDs, or exactly
["*"], which must be spelled — an empty list reaches nothing) and act
(search/research/index/delete/admin/mint) are signed in. The rejected
alternative was a tenant_id column; it cost a table rebuild, a trigger, a cache, a startup
warm and a rule for pre-existing rows, and it lost one bug class this keeps: only a caller
whose token already names a GUID can create that project, so POST /index is not an
existence oracle.

Refusals. An out-of-scope project answers 404 project.not_found, byte-identical to
one that never existed
— a distinguishable refusal confirms which GUIDs exist, and a GUID
is a bearer identifier, so auth.forbidden cannot exist on that path. The missing action
is named (403): the caller has already proved it holds the project. Two enforcement
layers: typed scope extractors check covers(guid) then permits(action), in that
order; enforce_route_policy fails closed, so a routed path with no policy row is refused
rather than served. ROUTE_POLICY names every route, and one of its guards drives the whole
refusal table so it stays exhaustive as routes are added.

Public routes are five and each says why: /health and /version (liveness — a probe
needing a credential reports the credential's health, not the server's), /config,
/llms.txt and /.well-known/mindex.json (discovery cannot be discovered from behind a
credential). admin covers /gc, /status, /metrics; there is no gc action, because
POST /gc walks every collection and no project list describes it.

POST /auth/tokens mints a narrower token from the one presented. Containment is
absolute — actions, projects and expiry may never exceed the minter, or a read-only mint
credential becomes admin one call later. --days 0 (non-expiring) is refused here and
allowed only from the local CLI. aud (--for cli,vscode,agent) is the one claim
nothing in the server reads: no part of an HTTP request identifies the process behind it, so
a check there would be theatre. The clients refuse instead, which catches a credential
pasted into the wrong place and catches nothing adversarial. Containment deliberately does
not bind it — audience is not authority, and binding it would refuse the VS Code button
minting an agent token from a vscode one.

Client credential resolution, first wins: --token > $MINDEX_TOKEN >
$MINDEX_TOKEN_FILE (a path to a 0600 file) > token in indexer.toml/watcher.toml >
the per-server entry in ~/.config/mindex/credentials.toml. MINDEX_TOKEN_FILE exists for
a caller configured by an environment block inside somebody else's config file — an MCP
server list lives in an editor's JSON, where a token sits in plaintext under no permission
check and a path does not. Its trap is the precedence: a shell exporting MINDEX_TOKEN
passes it to every child, so such a block must also set MINDEX_TOKEN="".

GET /.well-known/mindex.json — identity, version, transport, the endpoint inventory
and the live /config snapshot. endpoints[] is derived from the OpenAPI spec at first
use
, never written: the route table already had four copies and a hand-kept fifth is the
one nothing checks. Three things stay hand-kept and each has a guard —
UNDOCUMENTED_ROUTES, DESCRIPTOR_HIDDEN_ROUTES (/metrics alone, routed only under
[metrics].enabled, so advertising it would promise a 404) and STREAMING_ENDPOINTS.
authentication serializes as an explicit null: "authenticates nothing" and "too old to
say" must not look the same on the wire. Unlike /llms.txt and /metrics, the descriptor
is in the spec — it is JSON for a client, which is what a spec is for.

POST /v0/{guid}/research and .../{run_id}/challenge answer one JSON body unless
?stream=yes.
The run is identical either way; the query decides only who assembles the
terminal. Every field of ResearchResponse is the data() of the frame of the same name,
produced by the same code — thinking, step and progress are dropped, since they exist
to be watched. Mid-run failures become statuses because they can: 503
ollama.unavailable, 503 ollama.error, 500 research.no_report; under
?stream=yes all three are still error events on a 200. Cancel-on-disconnect is
unchanged — a CancellationGuard across the drain does in JSON mode what the stream's
Drop does in SSE mode. ?stream= is deny_unknown_fields and accepts only yes/no, so
?stream=true is a 400 rather than an hour of GPU spent before anyone notices.

Gateway. The keyless set is now ROUTE_POLICY's public set (GET and HEAD, anchored so
/configuration is not /config). A keyless request to a route that exists gets 401 with
WWW-Authenticate and a problem+json body in the same envelope mindex uses everywhere else;
/.env, /js/config.js and / still get 444. The fail2ban jail counts a keyless 401 on
key_ok:0 alone — mindex issues its own 401s and 403s for a token that is real but expired
or scoped elsewhere, and those carry key_ok:1: a client to re-credential, not an attacker
to ban. The public surface carries X-Robots-Tag: noindex.

Test coverage. The stack now runs a second mindex with [auth].enabled and 39 tests
drive it, because the existing suite asserts the unauthorized behaviour and that is the
coverage worth keeping. Every narrow token is minted from the root one over
POST /auth/tokens and then used, so containment is exercised dozens of times as a side
effect of setting scenes — the shape a containment bug needs, since it does not fail loudly;
it hands somebody a working credential wider than the one they had. Action tests are driven
from Action::ALL in both roles rather than from an example: a hand-written list of
"dangerous" actions rejects admin, passes delete, and looks correct in any test that
only tries admin.