Skip to content

feat(mcp): stage the diagnostics verification release#2674

Draft
reachjalil wants to merge 2 commits into
different-ai:devfrom
reachjalil:feature/mcp-diagnostics-release
Draft

feat(mcp): stage the diagnostics verification release#2674
reachjalil wants to merge 2 commits into
different-ai:devfrom
reachjalil:feature/mcp-diagnostics-release

Conversation

@reachjalil

@reachjalil reachjalil commented Jul 11, 2026

Copy link
Copy Markdown
Collaborator

MCP diagnostics controlled release

Important

Verification is ready to begin, but Jalil has not personally verified any level yet.

This draft is the controlled release ledger. Its branch remains based on
upstream dev and contains none of the three implementation levels.

Current status

Status Result
Three source levels implemented Yes
Source hardening fixes published Yes
Independent agent integration rehearsal Passed in #2675
Verified by Jalil Not started
Integrated into this parent branch None
Live ServiceNow / Microsoft tenant verification Not started

The independent rehearsal deliberately acted out the full release plan before
Jalil begins. It merged the three levels in order, resolved their overlaps,
found and fixed integration defects, and passed one complete operator journey.
That reduces review risk; it does not grant product approval.

What we set out to build

The program has three understandable levels:

  1. Structured failures: tell the operator exactly where an MCP connection
    failed, who owns the next action, and what safe evidence support can use.
  2. Enterprise diagnostic fixture: reproduce ServiceNow and Microsoft-style
    OAuth, MCP, pagination, sessions, and faults without customer data.
  3. Live diagnostic mode: show Den's phases in real time, preserve the
    highest proven health and first failure, and support repair and retry.

Source and approval ledger

Level Source PR Published head Implemented Agent-verified in #2675 Verified by Jalil Integrated here
Structured errors #2669 7b99407c Yes Yes Not started No
Enterprise mock and connection test #2670 e1569ec3 Yes Yes Not started No
Live tracing #2672 3e84bd30 Yes Yes Not started No

Passing tests or an agent-run browser flow never changes Verified by Jalil.
That column changes only when Jalil reviews the selected capability and accepts
its behavior and limitations.

What the rehearsal found

The independent rehearsal found issues that were easy to miss when the three
PRs were viewed separately:

User-facing problem Why it blocked release Rehearsal result
OAuth could return to Den Web instead of Den API. A successful provider consent could end in a 404. Fixed and regression-tested.
A timed-out operation could persist a late token, client, verifier, or grant. Product state could change after the UI reported failure. Fixed with lifecycle persistence fences.
A successful callback could validate with stale credentials. Valid consent could immediately appear unauthorized. Fixed by reloading persisted state.
Credential 401 and provider-policy 403 were conflated. Support could assign the wrong administrator and remediation. Fixed with distinct safe phases/codes/owners.
Structured provider tool errors could be treated as success. Permission denial or throttling could be hidden. Fixed through the real Den execute path.
Deleting a connection could race a diagnostic start and orphan work. Removed connections could retain attempts/events or background runners. Fixed transactionally with cancellation/race tests.
The easy OAuth mock path did not prove the realistic confidential-client path. A demo could pass while ServiceNow-style onboarding failed. Fixed; the proof uses pre-registration, exact callback, client_secret_post, and no DCR.
Repeated fixture runs leaked listener/callback capacity. A long review session could become flaky. Fixed with bounded replacement and fail-closed controls.
Small-screen layout and unstable row targeting obscured the workflow. The maintainer could not reliably review Add, Test, and Diagnose. Fixed and replayed at the automation viewport.
Catalog Ready did not make its scope boundary prominent enough. It could be misread as provider-operation readiness. A visible catalog-only boundary is now tested.

All of these are fixed and agent-verified on their owning source branch and/or
the integrated rehearsal. None is considered accepted in this parent until
Jalil reviews the corresponding level.

Independent evidence

Draft rehearsal: #2675
at b44089ce.

The final combined run proved this sequence:

owned Network TCP failure
-> pre-registered ServiceNow-style confidential OAuth
-> read-only full catalog test (4 tools / 2 pages)
-> live MCP version failure preserving Authorized
-> repair and retry to Catalog Ready
-> separate provider-policy denial through Den execution
-> cleanup

Agent verification includes 271 focused tests / 1,432 expectations, Den
API/Web/types builds and TypeScript checks, supported database bootstrap and
schema checks, and one Fraimz flow with six user-visible steps, 59 assertions,
five screenshots, cleanup, and voice-over coverage.

What remains a decision

  • The deterministic mock is not a live-provider conformance claim. Approved
    nonproduction ServiceNow and Microsoft 365 tenants still need verification.
  • In-flight authorization is not seamlessly portable across a Den restart or
    instance hop; failure is explicit and retryable.
  • An already-running third-party SDK promise is lifecycle-bounded and cannot
    write late state, but it cannot always be forcibly aborted internally.
  • Test connection proves authorization, protocol, catalog, and teardown;
    it intentionally does not invoke tools or prove mutation permissions.
  • The quick test and production catalog loader should eventually share the
    same safety bounds.

These can be accepted for an initial release, fixed before integration, or
deferred explicitly. They must not disappear from the ledger.

Controlled verification process

For each level:

  1. Jalil selects the next capability. Start with structured errors because the
    later levels consume their phase/owner language.
  2. Review only the short visible checkpoint and the focused source PR.
  3. Record verified, needs changes, or deferred, including accepted
    limitations.
  4. Integrate only the accepted source commit into this parent branch.
  5. Rerun the cumulative test gate and combined browser journey.
  6. Update this ledger with the integrated SHA, proof, and next decision.
  7. Repeat, then perform approved live-provider checks.

Branch and PR rules

  • This branch remains on Jalil's fork and targets upstream dev through this
    draft PR.
  • Source feature branches remain on Jalil's forks; no feature branch is pushed
    to the upstream repository.
  • GitHub cannot use a fork-only branch as the literal base of an upstream-repo
    PR. Therefore feat(mcp): add structured connection diagnostics #2669, feat(mcp): add enterprise diagnostic mock server #2670, and feat(mcp): add live connection diagnostic tracing #2672 remain based on upstream dev and link
    to this ledger. Accepted SHAs are integrated here after approval.
  • The rehearsal PR is an independent sibling and must not be merged wholesale
    as a substitute for the controlled process.
  • Tests and agent review cannot silently change Jalil's approval status.
  • Evidence must exclude secrets, OAuth values, session identifiers, customer
    hostnames, and customer content.

Verification log

Date Checkpoint Result
2026-07-11 Parent integration branch created from upstream dev Complete
2026-07-11 Parent draft linked from all three source PRs Complete
2026-07-11 Independent source review and hardening Complete; source fixes published
2026-07-11 Parallel three-level integration rehearsal Passed in #2675
2026-07-11 Jalil-led capability verification Not started
2026-07-11 Implementation integrated into this parent None

Next decision

Jalil chooses the first checkpoint to review. The recommended starting point is
the owned Network TCP failure in Level 1: confirm that the UI names the phase,
the Network Admin owner, a useful action, and no raw fetch failed or secret.

@vercel

vercel Bot commented Jul 11, 2026

Copy link
Copy Markdown
Contributor

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
openwork-landing Ready Ready Preview, Comment, Open in v0 Jul 11, 2026 6:17pm

@vercel

vercel Bot commented Jul 11, 2026

Copy link
Copy Markdown
Contributor

@reachjalil is attempting to deploy a commit to the Different AI Team on Vercel.

A member of the Team first needs to authorize it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant