Skip to content

fix(mcp): clear live validation release blockers#809

Merged
rickylabs merged 7 commits into
mainfrom
fix/808-mcp-live-defects
Jul 17, 2026
Merged

fix(mcp): clear live validation release blockers#809
rickylabs merged 7 commits into
mainfrom
fix/808-mcp-live-defects

Conversation

@rickylabs

@rickylabs rickylabs commented Jul 17, 2026

Copy link
Copy Markdown
Owner

Summary

Clears the three @netscript/mcp beta.10 live-validation release blockers and preserves the paths that already passed.

  • live Aspire Dashboard telemetry is now transported and normalized instead of silently becoming empty;
  • doctor is bounded before schema validation while retaining full family counts/severity;
  • scaffolded consumers get useful package-shipped docs by default, with explicit errors for missing/empty overrides;
  • exact tool names, command discovery, and both execution-policy directions are regression-guarded.

Closes #808

Root cause and owning-layer fixes

1. Aspire telemetry

The false-empty result had four causes, not a casing mismatch:

  1. Deno rejected the generated localhost Dashboard certificate with UnknownIssuer; the query adapter caught the request error and degraded it to empty.
  2. Aspire Dashboard 13.4.6 wraps OTLP JSON as {data:{resourceSpans:[…]},totalCount,returnedCount}, while normalization only inspected top-level traces / spans / items.
  3. Spans live under resourceSpans[].scopeSpans[].spans[]; resource-level service attributes were therefore lost even after flattening.
  4. Numeric OTLP SpanKind values were decoded one position early (1=internal … 5=consumer is the real mapping).

The shared parser fix is owned by packages/telemetry infrastructure. MCP infrastructure only composes the Aspire edge: for loopback HTTPS it loads the generated ASP.NET development PEM into a custom Deno.HttpClient; it never disables verification and never changes non-loopback HTTPS. The doctor probe reuses the same trust policy.

The checked-in fixture is a real capture from 2026-07-17, Aspire Dashboard 13.4.6, GET /api/telemetry/spans and GET /api/telemetry/resources after triggering the scaffolded health-check job. Its provenance is named in the fixture source.

2. Doctor output contract

The CLI-composed doctor flattened every check into $.checks, then validated against maxItems:20. The runner correctly rejected its own oversized output.

The cap remains 20 deliberately. Top-level checks now contains one aggregate per family. Each family retains up to 19 detailed checks plus an explicit overflow summary carrying the worst omitted severity. Overall/family counts and statuses are computed over every original check, so aggregation cannot hide failures. A real CLI-composed 25-plugin-check regression runs through the advertised schema.

3. Docs composition

The previous default was <projectRoot>/docs/site, which scaffolded consumers do not contain. The package README is now imported as a published text asset and indexed under slug mcp. --docs-root / NETSCRIPT_DOCS_ROOT remain filesystem overrides.

A missing or Markdown-empty explicit override returns docs_corpus_not_found, includes the resolved path, and says to pass --docs-root <path>; it never returns silent zero counts.

Live-discovered contract guard

Once telemetry made get_run success reachable, live validation exposed a closed-schema omission for its optional traceId, outcome, and errorMessage fields. The schema and undefined-field construction are corrected and guarded through the captured live fixture.

Commits

  • fb87169c — parse live Aspire telemetry
  • e8bc7210 — bound doctor output by family
  • 9b2fd26d — ship a default docs corpus
  • b04e8f0a — validate live success contracts
  • bd52f4b6 — record live release gates

Live acceptance — local maintainer CLI over newline stdio

Fresh scaffold, foreground Aspire AppHost, fresh health-check trigger, exact 13-name tools/list:

Tool Live evidence Verdict
get_app_status pass; 11 resources; 42 spans PASS
list_runs 2 runs; fresh health-check present PASS
get_run live run returned 13 spans PASS
get_recent_errors valid bounded empty result for the healthy workload PASS
get_last_job_result found:true; completed health-check trace PASS
analyze_service_performance 15 samples; 5 operations for workers PASS
analyze_db_bottlenecks valid bounded zero-sample result for this workload PASS
doctor schema-valid; 4 aggregates / 4 families; telemetry family pass PASS
search_docs 1 match; slug mcp PASS
list_docs 1 package-shipped document; slug mcp PASS
get_doc successful mcp retrieval with bounded content PASS
list_commands 100 live CLI descriptors PASS
execute_command plugin list exit 0; deploy denied before spawn by deny_deploy PASS

Initialize reported @netscript/mcp / protocol 2025-11-25; tools/list returned the exact code-owned 13-name order.

Validation

  • deno task quality:scan — PASS, no findings / no new suppressions
  • deno task arch:check — PASS
  • MCP scoped check — PASS, 60 files / zero diagnostics
  • CLI MCP scoped check — PASS, 6 files / zero diagnostics
  • focused telemetry package — PASS, check + 52 tests
  • focused MCP package — PASS, check + 45 tests
  • CLI MCP composition regression — PASS
  • deno doc --lint packages/mcp/cli.ts — PASS, zero diagnostics
  • MCP publish:dry-run — PASS, README text asset included, no slow types
  • no-cleanup capture scaffold — PASS, 58/58
  • final deno task e2e:cli run scaffold.runtime --cleanup --format pretty — PASS, 60/60; Aspire stopped and aspire ps empty
  • no deno.lock churn

Scoped lint-wrapper note: Deno 2.9.3 rejects the root wildcard workspace before linting. The wrapper reports this honestly as a tooling failure. The identical selected 59 MCP and 6 CLI MCP TypeScript files pass lint and formatting with explicit configs; no workspace edit or suppression was introduced.

Harness / architecture

  • Run: .llm/runs/fix-808-mcp-live-defects--mcp-live/
  • Archetype 6 remains a thin protocol router; external adapters stay in infrastructure.
  • Accepted debt MCP-A6-V2-SHAPE is unchanged; no new architecture debt.
  • Formal PLAN-EVAL / IMPL-EVAL dispatch was explicitly owner-waived. No evaluator verdict is claimed.
  • Keep this PR draft with status:impl-eval; do not merge.

Copy link
Copy Markdown
Owner Author

[PHASE: PLAN]

Research and design are locked in harness run .llm/runs/fix-808-mcp-live-defects--mcp-live/.

Decisions

  1. Normalize the live Aspire 13.4.6 envelope at the shared telemetry infrastructure owner, with an MCP boundary regression generated from a real capture.
  2. Preserve doctor’s 20-item contract by family aggregation and bounded detail.
  3. Use package-shipped embedded documentation by default; explicit filesystem overrides fail visibly when missing.
  4. Retain the 13-name and command allow/deny live matrix.

Process

  • Owner explicitly waived formal evaluator dispatch; no PLAN-EVAL or IMPL-EVAL will be launched or claimed.
  • Draft remains at status:impl-eval; do not merge.
  • Bootstrap commit: 61de3d323e5a53a08c4af2d469e02186fddb93cb.

@rickylabs
rickylabs force-pushed the fix/808-mcp-live-defects branch from 61de3d3 to f2b8c08 Compare July 17, 2026 04:57

Copy link
Copy Markdown
Owner Author

Implementation + live-validation summary

Harness run: .llm/runs/fix-808-mcp-live-defects--mcp-live/
Head: bd52f4b6407086b41d8aef76a1f1bd026959d776
State: implementation complete; draft remains at status:impl-eval; formal evaluator dispatch was owner-waived.

Delivered

  • telemetry: real Aspire 13.4.6 envelope/nesting/resource attributes/OTLP kinds, plus loopback ASP.NET CA trust and captured live fixtures;
  • doctor: family aggregation preserves the 20-item cap and full severity/count truth;
  • docs: package-shipped default corpus, explicit filesystem override, actionable missing/empty-corpus error;
  • regression paths: exact 13 names, populated command catalog, allowed plugin list, denied-before-spawn deploy;
  • live-discovered get_run success-schema omission fixed and regression-guarded.

Verdict evidence

  • all 13 tools: PASS live over local maintainer CLI stdio;
  • telemetry proof: 11 resources / 42 spans, 2 runs, fresh job found, 15 performance samples;
  • docs proof: list/search/get all succeeded on slug mcp;
  • doctor proof: schema-valid 4 aggregates / 4 families;
  • command proof: 100 descriptors; allowed exit 0; denied deny_deploy;
  • final scaffold runtime with cleanup: 60 passed, 0 failed;
  • quality:scan, arch:check, focused checks/tests, raw doc lint, and publish dry-run: PASS;
  • no lockfile churn, new suppressions, or new architecture debt.

The PR body contains the complete per-tool matrix and telemetry root-cause analysis. No merge was attempted.

@rickylabs

Copy link
Copy Markdown
Owner Author

IMPL-EVAL — PASS (supervisor-dispatched; Claude · Fable 5 · medium, review_codex_complex per ladder)

Verdict: .llm/runs/beta10-cli--orchestrator/slices/808-mcp-fixes/evaluate.md. Code-level truth of all five commits verified (adapter parses the live Dashboard envelope at the infrastructure layer with provenance-stamped live-capture fixtures; doctor bounding is deliberate family aggregation with a real-flow schema regression; default docs corpus ships with explicit non-silent empty state; Archetype-6 layering intact; publish dry-run incl. README asset green). Merging on CI green; the independent full-scaffold SHIP re-validation runs post-merge as the final publish gate.

🤖 Generated with Claude Code

@rickylabs

Copy link
Copy Markdown
Owner Author

[PHASE: IMPL] — Round-two doctor registry-layout follow-up

External re-validation verified the original three #808 fixes live, then exposed a new publish blocker in project/plugin_registry.

Root cause: the doctor hardcoded .netscript/generated/plugins.ts, but netscript generate plugins emits registry modules from each generator emission. The canonical E2E scaffold produced nested plugin-ai/{agents,tools}.registry.ts and plugin-workers/job-registry.ts outputs.

Fix (418efd69): the owning project-wiring doctor filesystem adapter now enumerates .netscript/generated recursively and accepts the generator layouts *.registry.ts and *-registry.ts; legacy plugins.ts remains supported. The healthy regression fixture preserves the exact three paths captured on 2026-07-17 from round-two scaffold plugin-smoke-20260717-074108 after generated.plugins-check.

Focused evidence:

  • doctor families + real doctor-flow output/schema regression: 9 passed, 0 failed
  • scoped Deno check: 3 files, 0 diagnostics
  • package-config lint: 3 files checked
  • git diff --check: PASS

No self-evaluation was dispatched and no merge was performed. The new fix is implementation/test verified; a subsequent external live rerun should confirm the doctor verdict on the canonical scaffold.

@rickylabs
rickylabs marked this pull request as ready for review July 17, 2026 06:02
@rickylabs
rickylabs merged commit b90a4ee into main Jul 17, 2026
18 of 19 checks passed
rickylabs added a commit that referenced this pull request Jul 17, 2026
…VAL FAIL_FIX

Proves: preflight-vs-publish outcomes are no longer conflated (33
published / mcp failed / cli skipped, separate from the unauthenticated
10-failed/25-skipped graph preflight); byte-identity of already-published
members is now a MANDATORY precondition for any tag move, with beta.10
recast honestly as a non-byte-identical move (6 plugins changed) that
JSR's skip-existing/fill-unpublished behavior papered over, not a safe
precedent; first-publish live-validation wording now matches #808/#809
(in-tree maintainer CLI against a real scaffolded app) with an explicit
note that #812's canary channel changes this to canary-published surface
+ pinned prod E2E verdict.

Cures findings 1-3 of .llm/runs/beta10-cli--orchestrator/slices/819-relskill-eval/evaluate.md.
Finding 4 (non-blocking, #812 merge-conflict guidance) is untouched.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017LHrkXyMzsQwb9bqr82EFK
rickylabs added a commit that referenced this pull request Jul 17, 2026
…, first-publish checklist (#819)

* docs(release): codify same-semver republish recovery + min-dependency-age handling + first-publish checklist

Owner directive after the v0.0.1-beta.10 cut (2026-07-17): two release patterns were proven live
and need to be reachable before an agent defaults to an unwanted stable-channel semver bump.

- Same-semver republish (partial-publish recovery): JSR version immutability binds only published
  members; a failed/skipped member can still publish at the same semver via fix-forward + tag
  fast-forward + publish.yml re-dispatch. Cites the beta.10 precedent (33/70 members published,
  mcp/cli recovered without a version bump — runs 29558968037 -> 29562537123).
- Minimum-dependency-age at release time: Deno 2.9's ~24h default rejects same-day publishes; the
  harness pins --minimum-dependency-age=0 and keeps fresh-plugin resolution in one Deno process
  (child deno x re-exec drops the flag) — #813 + #817. User-facing 24h window tracked in #818.
- First-publish checklist: manual README/tagline/license/exports/docs-site/live-validation checks
  until #811/#812's publish-readiness automation lands.

Regenerated the .claude mirror via agentic:sync-claude; docs:links, sync-claude:check, and fmt all
green on touched files.

* fix(docs): correct beta.10 same-semver recovery accounting per IMPL-EVAL FAIL_FIX

Proves: preflight-vs-publish outcomes are no longer conflated (33
published / mcp failed / cli skipped, separate from the unauthenticated
10-failed/25-skipped graph preflight); byte-identity of already-published
members is now a MANDATORY precondition for any tag move, with beta.10
recast honestly as a non-byte-identical move (6 plugins changed) that
JSR's skip-existing/fill-unpublished behavior papered over, not a safe
precedent; first-publish live-validation wording now matches #808/#809
(in-tree maintainer CLI against a real scaffolded app) with an explicit
note that #812's canary channel changes this to canary-published surface
+ pinned prod E2E verdict.

Cures findings 1-3 of .llm/runs/beta10-cli--orchestrator/slices/819-relskill-eval/evaluate.md.
Finding 4 (non-blocking, #812 merge-conflict guidance) is untouched.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017LHrkXyMzsQwb9bqr82EFK

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

1 participant