test(e2e): frozen MCP contract behavior tier#222
Merged
Conversation
Contributor
|
🎉 PR Validation ✅ PASSED Commit: Checks:
Ready to merge! ✨ 🔗 View workflow run |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds
e2e/mcp-contract.test.ts(@p0, offline, ephemeral-port daemon transport) that proves the frozen 6-tool MCP contract BEHAVIORS through the CLI's wrapped/mcpsurface. Todaymcp-daemon.test.tsonly checks the tool surface exists (6 tools + dual-channel output); a bad@agentage/server-memoryor@agentage/memory-corebump could regress any behavior and pass CI silently.Tests (each fails if the behavior regresses)
memory__editstr_replace edge cases - new_str omitted deletes the matched old_str; body alongsidemode=str_replacereturns the canonical refusal; old_str/new_str without the mode is refused; a duplicate old_str surfaces theMultiple occurrences of old_strerror THROUGH the MCP transport (asserted at the wire, not just the CLI). Also asserts single-vault instructions omit the@<vault>/line and stay under 2048 chars.@<vault>/routing over TWO vaults - list root surfaces each vault as an@<vault>folder; read/write/edit/delete accept@<vault>/<path>;search folder:"@<vault>"scopes and prefixes hits; an unscoped search fans out; a write to@betais invisible in@alpha. Asserts multi-vault instructions carry the@<vault>/addressing line and stay under 2048 chars.limitcaps the page,nextCursoris emitted, and a cursor round-trip returns page 2 with no overlap and covers every hit (deterministic score ordering).memory__listtree shapes - depth 1 (unexpanded stubs) vs depth 2 (expanded), recursive per-folder file counts,truncatedflag type, and the tags filter.Also corrects the
mcp-daemon.test.tsheader comment, which falsely claimed it asserts@<vault>/routing (it uses one vault + plain paths).Findings (recorded, not faked)
Two claimed behaviors are NOT enforced upstream, so their tests are
test.skipwith an evidence comment rather than fake-passing assertions:memory__write/memory__edittool descriptions say secrets "are refused", but neitherregister-toolsnormemory-corehas any runtime enforcement - a credential body writes through.memory__readhas no size budget. Read returns the full body verbatim (doc.body); there is no byte/line truncation on read.Verification
Offline and deterministic. Ran the new spec 3x consecutively (5 passed / 2 skipped each, ~5s), the full offline set once (28 passed / 2 skipped, no interference), and
npm run verify(type-check + e2e type-check + lint + format + 203 unit tests + build) - all green.