Skip to content

test(e2e): frozen MCP contract behavior tier#222

Merged
vreshch merged 1 commit into
masterfrom
feature/e2e-mcp-contract
Jul 6, 2026
Merged

test(e2e): frozen MCP contract behavior tier#222
vreshch merged 1 commit into
masterfrom
feature/e2e-mcp-contract

Conversation

@vreshch

@vreshch vreshch commented Jul 6, 2026

Copy link
Copy Markdown
Member

What

Adds e2e/mcp-contract.test.ts (@p0, offline, ephemeral-port daemon transport) that proves the frozen 6-tool MCP contract BEHAVIORS through the CLI's wrapped /mcp surface. Today mcp-daemon.test.ts only checks the tool surface exists (6 tools + dual-channel output); a bad @agentage/server-memory or @agentage/memory-core bump could regress any behavior and pass CI silently.

Tests (each fails if the behavior regresses)

  • memory__edit str_replace edge cases - new_str omitted deletes the matched old_str; body alongside mode=str_replace returns the canonical refusal; old_str/new_str without the mode is refused; a duplicate old_str surfaces the Multiple occurrences of old_str error THROUGH the MCP transport (asserted at the wire, not just the CLI). Also asserts single-vault instructions omit the @<vault>/ line and stay under 2048 chars.
  • @<vault>/ routing over TWO vaults - list root surfaces each vault as an @<vault> folder; read/write/edit/delete accept @<vault>/<path>; search folder:"@<vault>" scopes and prefixes hits; an unscoped search fans out; a write to @beta is invisible in @alpha. Asserts multi-vault instructions carry the @<vault>/ addressing line and stay under 2048 chars.
  • Bounded search pagination - limit caps the page, nextCursor is emitted, and a cursor round-trip returns page 2 with no overlap and covers every hit (deterministic score ordering).
  • Soft-delete recoverability - after delete the working copy drops the file and a read reports not-found, yet the content is still recoverable from the vault's git history (the product promise), asserted via git.
  • memory__list tree shapes - depth 1 (unexpanded stubs) vs depth 2 (expanded), recursive per-folder file counts, truncated flag type, and the tags filter.

Also corrects the mcp-daemon.test.ts header comment, which falsely claimed it asserts @<vault>/ routing (it uses one vault + plain paths).

Findings (recorded, not faked)

Two claimed behaviors are NOT enforced upstream, so their tests are test.skip with an evidence comment rather than fake-passing assertions:

  1. Secret refusal is description-only. The memory__write/memory__edit tool descriptions say secrets "are refused", but neither register-tools nor memory-core has any runtime enforcement - a credential body writes through.
  2. memory__read has no size budget. Read returns the full body verbatim (doc.body); there is no byte/line truncation on read.

Verification

Offline and deterministic. Ran the new spec 3x consecutively (5 passed / 2 skipped each, ~5s), the full offline set once (28 passed / 2 skipped, no interference), and npm run verify (type-check + e2e type-check + lint + format + 203 unit tests + build) - all green.

@github-actions

github-actions Bot commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

🎉 PR Validation ✅ PASSED

Commit: 6bdbe7445fd1d567c327ab4de2b7123fc7fd6bf9
Branch: feature/e2e-mcp-contract

Checks:

  • ✅ Release guard (no version/changelog changes)
  • ✅ Dependencies installed
  • ✅ Type check passed
  • ✅ Linting passed
  • ✅ Format check passed
  • ✅ Tests + coverage passed
  • ✅ Build successful

Ready to merge!


🔗 View workflow run
⏰ Generated at: 2026-07-06T09:18:17.893Z

@vreshch vreshch marked this pull request as ready for review July 6, 2026 09:19
@vreshch vreshch merged commit ebf8bb1 into master Jul 6, 2026
2 checks passed
@vreshch vreshch deleted the feature/e2e-mcp-contract branch July 6, 2026 09:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant