feat(mcp): @memwal/mcp stdio bridge + browser login #141
Merged
Conversation
…bility - packages/mcp: new @memwal/mcp npm package — stdio MCP server, browser wallet login flow, auto-reconnect SSE bridge, env presets (--prod/--dev/--staging/--local), auth-required fallback that serves a friendly inline error when ~/.memwal/credentials.json is missing - services/server/scripts/mcp: Node sidecar MCP routes (/mcp/sse + /mcp/messages), 3s heartbeat keepalive, per-session transport map, idempotent cleanup so late close events on replaced transports don't wipe the active session - services/server/src/mcp_proxy.rs: Rust axum reverse proxy with 24h timeout override on SSE GET; the shared http_client carried a 30s timeout that was killing long tool calls (walrus blob writes take 25-30s). Adds anti-buffering response headers (X-Accel-Buffering: no, cache-control no-transform, connection keep-alive) - apps/app/src/pages/ConnectMcp: /connect/mcp consent page with sponsored add_delegate_key tx + localhost callback. Post-login ClientConfigPanel shows paste-ready config snippets for Cursor / Claude Desktop / Claude Code / Antigravity with copy buttons and env-aware CLI flags - services/server/scripts/package.json: bump @mysten/walrus 1.0.3 -> 1.1.7, @mysten/sui 2.5.0 -> 2.16.2, @mysten/seal 1.1.0 -> 1.1.3 to match the current Walrus contract — older versions hit "balance::destroy_zero" abort on remember - plans: design doc covering Phase A package, Phase B login flow, Phase B.5 deferred OAuth-style UX, Phase D dashboard panel
… H4)
C2 — localhost /callback CSRF + DNS rebinding:
- packages/mcp/src/login.ts: generate 32-byte state token, embed in
connectUrl, require callback to echo it back; constant-time compare
(timingSafeEqual). Reject if missing or mismatch.
- Validate Host header against {127.0.0.1:port, localhost:port} —
defeats DNS rebinding to an attacker-controlled name resolving to
loopback.
- Validate Origin header equals webUrl (no wildcard, no missing).
- Replace `Access-Control-Allow-Origin: *` with exact webUrl. Add `Vary: Origin`.
- Assert `Content-Type: application/json` so simple-request cross-origin
POSTs (text/plain, form-urlencoded) hit preflight and get blocked
by the restricted CORS.
- apps/app/src/pages/ConnectMcp.tsx: read `state` from query params,
validate as 64-hex, include in callback POST payload. Treat missing
state as invalid query (forces bridge upgrade in lockstep).
H3 — bridge creds-wipe DoS on 401:
- packages/mcp/src/bridge.ts: stop calling clearCreds() on 401. A 401
from the relayer is evidence of a problem (revoked key, WAF, proxy
MITM, --local malware) but not proof the saved seed is the cause.
Auto-wipe turned any one of those into a permanent outage. Now we
surface a loud error pointing the user at `memwal-mcp login` and
leave credentials.json untouched.
H4 — --relayer silently rewrote saved creds:
- packages/mcp/src/index.ts: --relayer flag now overrides the relayer
in-memory for the current process only; the saved credentials.json
is no longer mutated. Stops malicious config snippets (`--relayer
https://attacker`) from rewriting saved creds so future spawns ship
the seed to the attacker. To rotate the saved relayer, the user
must explicitly `memwal-mcp --logout` then re-login.
Verified locally: SSE handshake unchanged, memwal_remember E2E 27.5s
with blob_id returned, memwal_recall returns saved memories. Build clean.
Remaining audit findings tracked for follow-up:
- C1 (bearer = seed) — needs signed-request migration, larger refactor
- H1, H2, M1-M7, L1-L6 — see plans/2026-05-11-memwal-mcp-package-with-login.md
![github-code-quality[bot]](https://avatars.githubusercontent.com/u/9919?s=60&v=4){kind=small}
Resolves conflict in services/server/scripts/sidecar-server.ts (both sides added a new top-level import on the same line — kept both). Additional cleanup while resolving: - services/server/scripts/mcp/index.ts: drop the unused default `import express from "express";` runtime binding. Only the type imports (Express, Request, Response) are used in this file. dev brings in: - seal committee aggregator support (PR #136) - seal threshold + sidecar config extraction to seal-config.ts - vite stale-chunk reload fix in apps/app - doc updates Verified: packages/mcp tsc clean, apps/app tsc clean, cargo check on services/server clean. MCP route mount call site unchanged.
CI installed `zod@4.4.3` per the direct dep in sidecar package.json, then
npm rejected the resolution because `@mysten-incubation/memwal@0.0.3`
declares `peerOptional zod@^3.23.0` — incompatible with major 4 even
though `@modelcontextprotocol/sdk@1.29.0` itself accepts `^3.25 || ^4.0`.
Resolved by pinning the sidecar's zod to `^3.25.0` (lockfile picks
3.25.76). Satisfies:
- @modelcontextprotocol/sdk@1.29.0 → "^3.25 || ^4.0" ✓
- @mysten-incubation/memwal@0.0.3 → "^3.23.0" ✓
Our zod usage (services/server/scripts/mcp/tools/{remember,recall,
analyze,restore}.ts) is limited to `z.string()`, `.min()`, `.optional()`,
`.describe()` — no zod-v4-only surface — so the downgrade is mechanical.
`plans/2026-05-11-mcp-client-testing-guide.md` and `plans/2026-05-11-memwal-mcp-package-with-login.md` were committed in 2e788fe by mistake — they are personal/internal design notes, not intended for the public repo. Also adds `plans/` to .gitignore so future commits never reach in. The content is preserved locally in the author's untracked `.local-plans/` directory.
Adds `/api/mcp` — a single endpoint that speaks the MCP 2025-06
Streamable HTTP transport. After this lands, users add MemWal to any
MCP client without installing the `@memwal/mcp` package:
claude mcp add --transport http memwal https://relayer.memwal.ai/api/mcp
Architecture matches MailGate / Linear / Figma's MCP servers — one URL,
session id round-trips through the `mcp-session-id` header, auth on
every request via the existing Bearer scheme (no OAuth dance, no JWT,
no .well-known mess).
Sidecar — services/server/scripts/mcp/index.ts:
- Mounts GET/POST/DELETE /mcp via `StreamableHTTPServerTransport` from
`@modelcontextprotocol/sdk` 1.29 (already a dep).
- Per-session transport keyed by `mcp-session-id`. The SDK's
`onsessioninitialized` callback wires the cleanup so a sidecar restart
doesn't leak transports.
- Auth identical to the existing SSE path — same `resolveAuth(...)` lookup,
same Bearer + X-MemWal-Account-Id headers. SSE routes
(/mcp/sse + /mcp/messages) kept alive for the stdio bridge's backward
compatibility.
Rust relayer — services/server/src/mcp_proxy.rs:
- New `streamable_proxy` handler routed at `/api/mcp` (GET/POST/DELETE/
OPTIONS). Method dispatched on inbound, body forwarded verbatim,
response streamed via `Body::from_stream` so both small JSON bodies
and long-lived SSE upgrades work without buffering.
- Header allowlist extended with `mcp-session-id`, `mcp-protocol-version`,
`last-event-id` — the SDK on both ends reads these to route
requests to the right session and resume after disconnects.
- 24h per-request timeout override carried over from the SSE proxy
fix (commit 8990a88) so walrus blob writes don't trip the shared
http_client's 30s ceiling.
Dashboard — apps/app/src/pages/ConnectMcp.tsx:
- `ClientConfigPanel` adds a transport-mode pill (`stdio + package` vs
`http`). Both modes render the JSON snippet inline with copy buttons;
http mode shows the user's accountId and leaves Bearer as a
placeholder (the seed lives in ~/.memwal/credentials.json — never
surfaced to the dashboard for security reasons).
- HTTP hint block tells the user where to pick up the bearer and
reminds them to treat it like a long-lived API token.
Verified locally (Rust + sidecar both restart cleanly):
- POST /api/mcp initialize → 200, mcp-session-id assigned (~73ms)
- tools/list → 4 tools registered
- tools/call memwal_remember → blob_id returned at T+24.4s end-to-end
through the new transport
Tradeoff vs full OAuth 2.1: no native "Connect" button in client UIs,
user pastes a 7-line config block manually. Same UX shape as the
stdio package (which also requires a paste). Phase B.5 OAuth remains
on the follow-up list for when we want the Linear/Figma-style
one-click connect; this commit unblocks the Henry-confirmed simplest
path in the meantime.
ducnmm
requested changes
May 12, 2026
ducnmm
left a comment
ducnmm
left a comment
Collaborator
There was a problem hiding this comment.
Found blockers before this should merge.
-
services/server/Dockerfile:49only copiesscripts/*.tsinto the runtime image, butsidecar-server.tsnow imports./mcp/index.jsand that directory is not copied. Local source runs can pass, but the deployed Docker image will fail at sidecar startup with module-not-found forscripts/mcp/*. Please copy the MCP directory into the image, e.g.COPY scripts/ ./scripts/or an explicitCOPY scripts/mcp/ ./scripts/mcp/. -
The public MCP routes are long-lived and not rate-limited, while
resolveAuth()only validates bearer/account shapes and derives a pubkey before a session is allocated. A client can open many/api/mcp/sseor/api/mcpsessions with random 64-hex bearer values and valid-looking account ids, causing the relayer to hold 24h proxy streams and the sidecar to allocate transports/heartbeats without any on-chain auth ever happening. Please add an IP/session rate limit or cap before session creation, and preferably force a real credential validation before opening long-lived transports. -
Streamable HTTP sessions are not bound to the authenticated caller on subsequent requests.
handleStreamableHttp()resolves auth for every request, but whenstreamableSessions.get(sessionId)succeeds it directly callsconn.transport.handleRequest(req, res)without checkingconn.sessionKey === auth.sessionKey. Please reject mismatched session/auth pairs with 403 so a leaked/session-id cannot be reused under another bearer/account context.
Verification I ran:
cargo checkinservices/serverpassedpnpm --filter @memwal/app buildpassedpnpm --filter @memwal/mcp buildcould not be run locally because this checkout has not installed the new package dependencies, while CI's Compile & CLI Smoke is passing
…session binding
Three reviewer-flagged blockers, plus tests that exercise the fixes
end-to-end via real HTTP.
1. Dockerfile (services/server/Dockerfile)
sidecar-server.ts now imports ./mcp/index.js, but the runtime stage
only copied scripts/*.ts. The MCP module was missing in the image,
so deployed containers would crash at boot with module-not-found.
Added an explicit COPY scripts/mcp/ ./scripts/mcp/.
2. Rate limit before MCP session creation
(services/server/scripts/mcp/rateLimit.ts + index.ts)
resolveAuth() only validates header shape; a flood of forged
bearers + valid-looking accountIds could open long-lived SSE /
streamable-HTTP transports and hold sidecar memory + 24h proxy
streams indefinitely.
New McpRateLimiter caps:
- total concurrent sessions globally
- concurrent active sessions per source IP
- new session opens per source IP per minute (sliding window)
Limiter runs BEFORE resolveAuth() on /mcp/sse and on POST /mcp
for new sessions. Slot is one-shot released on transport close,
on auth failure, and on handleRequest throw.
Limits env-tunable: MCP_MAX_TOTAL_SESSIONS,
MCP_MAX_SESSIONS_PER_IP, MCP_MAX_NEW_SESSIONS_PER_IP_PER_MIN.
Limiter is lazy-initialized so tests can override env before
first request (ESM hoists imports above top-level env writes).
Relayer (services/server/src/mcp_proxy.rs) now injects the real
client IP via x-forwarded-for on all three MCP routes (sse,
messages, streamable), preserving any inbound chain. Without
this the limiter would bucket per loopback.
3. Streamable HTTP session bound to authenticated caller
(services/server/scripts/mcp/index.ts:handleStreamableHttp)
Previously, on a request carrying an existing mcp-session-id,
we dispatched directly without checking the auth matched the
one that opened the session — anyone who learned the UUID
could drive an existing transport under their own (or random)
credentials. Now we reject 403 unless conn.sessionKey ===
auth.sessionKey (delegate:accountId:delegatePubKey). Mismatch
logged as session.auth_mismatch.
Tests
-----
services/server/scripts/mcp/__tests__/rateLimit.test.ts (16 unit
tests) — McpRateLimiter caps, releaseFn idempotency, multi-IP
independence, clientIpFromRequest x-forwarded-for parsing,
loadRateLimitConfigFromEnv defaults + garbage handling.
services/server/scripts/mcp/__tests__/integration.test.ts (6 real-
HTTP integration tests) — spins up mountMcpRoutes on an ephemeral
port and verifies via fetch:
- POST /mcp initialize creates session, returns mcp-session-id
- 3rd open from same IP within window → 429 ip_burst_cap
- separate XFF IPs get independent buckets
- reusing mcp-session-id under different bearer → 403
- reusing under SAME bearer is accepted (sanity)
- malformed bearer → 401 with www-authenticate
services/server/src/mcp_proxy.rs (7 cfg(test) tests) —
inject_forwarded_for (empty/existing/whitespace-only inbound,
IPv6 peer), should_forward (allow/block lists),
build_forwarded_headers (drops cookies/host, keeps authorization
+ x-memwal-*).
services/server/scripts/package.json — wires `npm test` to
node --test --import tsx (no new dep; built-in node:test runner).
Verification
------------
cargo test → 142 passed (135 prior + 7 new)
npm test (sidecar) → 22 passed (16 unit + 6 HTTP integration)
cargo check → clean
Collaborator
Author
|
HI @ducnmm Thanks for the review addressed all three blockers in . |
Collaborator
|
One gap to cover before merging: auth should be discoverable from MCP clients, not require a separate manual Could we add either a |
Collaborator
Author
|
Yes, i think we should aim to ship both eventually; first can add a However Native MCP OAuth is our long term answer but need more time to implement it (auth code + PKCE). I can create tickets to implement it |
… login command (ENG-1749)
Before this change, first-run setup of the stdio MCP package required two
terminal commands:
codex mcp add memwal -- npx -y @mysten-incubation/memwal-mcp --dev # register
npx -y @mysten-incubation/memwal-mcp login --dev # browser login
Step 2 forced users to leave their MCP client, run a separate CLI command,
then restart the client. PR #141 review (ducnmm) flagged this as a merge
blocker; Henry confirmed on Slack the same friction applies internally.
After this change, only step 1 is needed. The agent calls memwal_login as
a tool inline; the user clicks the URL in chat, approves the wallet, and
the next memwal_* tool call succeeds.
Implementation
--------------
`packages/mcp/src/auth-required.ts`:
- Advertise a 5th tool `memwal_login` in `tools/list` when no creds exist.
- `tools/call memwal_login` returns the dashboard `/connect/mcp` URL
near-instantly (~250ms). The HTTP listener stays alive in the background
for 5 minutes; on user approval, credentials land at
~/.memwal/credentials.json. The tool result text repeats the URL three
times (header, code block, markdown link) so MCP clients that paraphrase
tool output (Claude Code) cannot strip every copy.
- Pass-through `relayerUrl` / `webUrl` / `label` from the entry point so
--dev / --staging / --local correctly route to the matching dashboard
(previously dropped on the auth-required path → always opened prod).
- LOGIN_INSTRUCTION (returned for the other 4 tools when creds missing)
now leads with "call memwal_login from this client" before the CLI
fallback.
`packages/mcp/src/bridge.ts`:
- Same `memwal_login` available in BRIDGE mode (when creds exist) so users
can re-login or switch wallets without removing+re-adding the MCP server.
- New `memwal_logout` tool: clears ~/.memwal/credentials.json and
instructs the user to revoke the on-chain delegate key from the
dashboard if they want full revocation. Local-only, never forwarded.
- `tools/list` responses from the relayer get spliced with the two local
tool definitions on the way back to the client.
- `tools/call memwal_login` / `memwal_logout` intercepted in stdin pump,
handled locally, never sent to the relayer.
`packages/mcp/src/login.ts`:
- New `onUrl` callback option fired as soon as the localhost listener is
bound. Lets callers (the tool wrapper) push the URL inline via the MCP
result before awaiting the callback.
`packages/mcp/src/index.ts`:
- Pass resolved `{relayerUrl, webUrl, label}` through to both
runAuthRequiredServer and runBridge.
`packages/mcp/package.json`:
- Add `engines: { node: ">=20.0.0" }` so npm install warns on older Node.
- Pin all four runtime deps (drop the `^`). For a public stdio package
spawned via `npx -y` (always pulls latest), exact pins eliminate the
supply-chain risk of a transitive dep update introducing a CVE between
publishes. Lockfile updated accordingly.
Local validation
----------------
- pnpm --filter @mysten-incubation/memwal-mcp build: clean (tsc, no errors)
- Direct stdin smoke against fresh HOME, --dev: tools/list returns 5 tools
in auth-required mode (with memwal_login); memwal_login tool call
returns dashboard URL <1s with correct dev.memwal.ai host
- URL routing verified for --prod / --staging / --dev / --local: each
routes to its matching webUrl + relayerUrl (https://memwal.ai,
staging.memwal.ai, dev.memwal.ai, localhost:5173)
- openBrowser disabled in MCP context: prevents macOS `open` from
foregrounding an existing memwal.ai tab instead of navigating to the
full /connect/mcp?... URL. Users click the URL surfaced in chat instead.
Out of scope (separate ticket)
------------------------------
- Native remote MCP OAuth on /api/mcp HTTP transport (ENG-1750). Covers
the second surface (HTTP transport), independent code area, larger
scope. Both flows coexist after both ship.
Closes ENG-1749.
Integrates MEM-35 (multi-wallet → single-wallet collapse) + MEM-37
(SEAL ciphertext Redis cache) + follow-up fixes (single-wallet retry
consistency, concurrent Walrus jobs) from dev.
Conflict in services/server/scripts/sidecar-server.ts:
- HEAD added `mountMcpRoutes(app, ...)` before the shared-secret auth
middleware (this PR's MCP routes).
- dev added `app.get("/metrics/wallet", ...)` in the same pre-auth
position (MEM-35 observability).
Both blocks are independent route mounts that need to coexist — kept
both, MCP routes first then /metrics/wallet, both before the
shared-secret check. No semantic overlap; both serve different
endpoints with different trust models.
Verified:
- cargo test: 149 passed, 0 failed (auto-merged main.rs + routes.rs OK)
- pnpm --filter @mysten-incubation/memwal-mcp build: clean
- tsc on sidecar TS: clean
ducnmm
approved these changes
May 13, 2026
Merged
18 tasks
hungtranphamminh
added a commit
that referenced
this pull request
May 14, 2026
…rage + routes split (#147) Restructures services/server into four focused layers, with zero behavioural regression against dev: - src/engine/ — MemoryEngine trait + WalrusSealEngine (production) and PlaintextEngine (benchmark). Handlers depend on Arc<dyn MemoryEngine> and are mode-blind. - src/services/ — extracts Embedder, Extractor, LlmChat from routes.rs; prompts moved to services/prompts/*.txt versioned text assets bundled via include_str!. - src/storage/ — moves db.rs, seal.rs, sui.rs, walrus.rs from src/ into src/storage/. - src/routes/ — splits the 2,844-line routes.rs into per-endpoint files: analyze.rs, recall.rs, remember.rs, admin.rs, sponsor.rs, mod.rs. Product additions: - BENCHMARK_MODE=true env flag — selects PlaintextEngine so the AI- quality benchmark suite can run without burning Walrus testnet quota. Off by default; loud warning on startup. Auth stays mode-blind. - POST /api/forget — owner-scoped, namespace-wide hard delete on vector_entries (used by the benchmark harness for inter-run cleanup). - POST /api/stats — owner-scoped namespace stats (count + bytes). - GET /health response gains a "mode" field ("production" | "benchmark") — additive, non-breaking. Rebase absorbed three dev-side feature streams that landed in parallel: - MEM-35 — single-wallet Apalis queue collapse + WalletJobError retry classification (dev commit 316cefd, PR #144). - MEM-37 — Redis SEAL ciphertext warm cache + BLOB_CACHE_MAX_BYTES size cap (dev commit 17fc86b, PR #143). Integrated into WalrusSealEngine during conflict resolution so the cap lives next to the cache it bounds. Cache policy factored into pure read_decision / write_decision helpers for unit-testability. - MCP server sidecar + proxy routes (PRs #141, #144). Routes mounted in main.rs; MEMWAL_RELAYER_URL piped to sidecar. Migration-number collision resolved by renaming this PR's benchmark- plaintext migration to 008_*; dev's 007_collapse_wallet_queues.sql is preserved unchanged. Defence-in-depth touch-ups (all in scope of the layer split): - LOW-S1: owner filter on fetch_plaintext_by_blob_id. - LOW-S5: /api/ask body.limit clamped at 100. - F3: upfront SEAL credential probe in /api/ask. Validation: - cargo test — 159/159 pass (+10 new: MEM-37 cache cap policy, /api/ask limit cap, /forget + /stats empty-namespace validation). - cargo check, cargo fmt --check — clean. - cargo clippy --all-targets — 8 warnings (carry-over from dev + 1 expected too_many_arguments on WalrusSealEngine::new after MEM-37 added blob_cache_max_bytes). - AI-quality preserved across rebase: - LOCOMO three-way (BENCHMARK_MODE=true): 54.5 → 54.8 → 53.5 J overall; every category within the ±2-3 J judge-noise floor. - LongMemEval (500 oracle queries): 71.7 J overall vs Mem0 49.0 / Zep 63.8 / Supermemory 85.4. - Four-reviewer panel (architect / backend / security / quality) signed off MERGE READY. Benchmark evidence committed at services/server/review/assessment/ benchmark-runs/2026-05-13-eng1747-quality-validation/ (README + 8 result JSONs). auth.rs is a 1-line namespace-rename diff vs dev; rate_limit.rs is byte-identical. Closes ENG-1747, MEM-40, MEM-41.
3 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds end-to-end MCP (Model Context Protocol) support to MemWal so any MCP-aware editor — Cursor, Claude Desktop, Claude Code, Antigravity can read and write to a user's encrypted Walrus memory via 4 tools (memwal_remember, memwal_recall, memwal_analyze, memwal_restore).
Test plan