Skip to content

PDX-468/469: feat(mcp): PROVAR_MCP_SCHEMA_MODE compact descriptions + PROVAR_MCP_TOOLS group filtering#170

Merged
mrdailey99 merged 4 commits into
developfrom
feature/pdx-468-469-startup-tuning
May 14, 2026
Merged

PDX-468/469: feat(mcp): PROVAR_MCP_SCHEMA_MODE compact descriptions + PROVAR_MCP_TOOLS group filtering#170
mrdailey99 merged 4 commits into
developfrom
feature/pdx-468-469-startup-tuning

Conversation

@mrdailey99
Copy link
Copy Markdown
Collaborator

@mrdailey99 mrdailey99 commented May 13, 2026

Summary

  • PDX-468: PROVAR_MCP_SCHEMA_MODE=compact — when set, every tool description switches to a short summary via a new desc(standard, compact) helper. All 19 tool files wrapped (220 call sites). Saves hundreds of tokens per tool at handshake time.
  • PDX-469: PROVAR_MCP_TOOLS=group1,group2 — comma-separated list of tool groups to register at startup. provardx_ping always registered. Groups: nitrox, automation, qualityhub, validation, authoring, inspect, connection, rca.

Changes

  • src/mcp/tools/descHelper.ts — new desc(standard, compact) helper
  • src/mcp/server.ts — TOOL_GROUPS registry, parseActiveGroups() export, group-loop registration
  • 19 tool files — all descriptions and z.describe() calls wrapped with desc()
  • test/unit/mcp/startupTuning.test.ts — 13 unit tests (compact mode + group filtering)
  • docs/mcp.md — env var table updated + new Agent performance tuning section

Test plan

  • mocha startupTuning.test.ts → 13 passing
  • mcp-smoke.cjs → 54/54 PASS (default config)
  • tsc --noEmit → clean
  • Set PROVAR_MCP_SCHEMA_MODE=compact and inspect tool description — should be short
  • Set PROVAR_MCP_TOOLS=nitrox — only NitroX tools + ping visible

Generated with Claude Code

@mrdailey99 @claude
PDX-468: feat(mcp): add PROVAR_MCP_SCHEMA_MODE and PROVAR_MCP_TOOLS s…
36ce286 
…tartup tuning (PDX-469)

RCA: Agents with limited context windows hit budget limits on startup because
full tool descriptions and schema metadata consume hundreds of tokens per tool.
Customers also reported wasted context when only a subset of ProvarDX tools
were relevant to their workflow (e.g. NitroX-only or automation-only sessions).
Fix: Add PROVAR_MCP_SCHEMA_MODE=compact that switches all 19 tool descriptions
to short summaries via desc() helper; add PROVAR_MCP_TOOLS env var that
restricts which tool groups register at startup via TOOL_GROUPS + parseActiveGroups().
Covers nitrox, automation, qualityhub, validation, authoring, inspect, connection,
rca groups. Add 13 unit tests and Agent performance tuning section in docs/mcp.md.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings May 13, 2026 15:38
@github-actions
Copy link
Copy Markdown

github-actions Bot commented May 13, 2026 •
edited
Loading

Quality Orchestrator

🟢 LOW · 22 / 100 · 2 file(s) with no test coverage detected.

🧪 Tests to Run · Running 19 of 44 tests

  • unit/mcp/server.test.ts
  • unit/mcp/antTools.test.ts
  • unit/mcp/automationTools.test.ts
  • unit/mcp/connectionTools.test.ts
  • unit/mcp/defectTools.test.ts
  • unit/mcp/nitroXTools.test.ts
  • unit/mcp/pageObjectGenerate.test.ts
  • unit/mcp/pageObjectValidate.test.ts
  • unit/mcp/projectValidateFromPath.test.ts
  • unit/mcp/propertiesTools.test.ts
  • unit/mcp/qualityHubApiTools.test.ts
  • unit/mcp/qualityHubTools.test.ts
  • unit/mcp/rcaTools.test.ts
  • unit/mcp/testCaseGenerate.test.ts
  • unit/mcp/testCaseStepTools.test.ts
  • unit/mcp/testCaseValidate.test.ts
  • unit/mcp/testPlanTools.test.ts
  • unit/mcp/testPlanValidate.test.ts
  • unit/mcp/testSuiteValidate.test.ts
▶ Run command 
npx vitest run \
  unit/mcp/server.test.ts \
  unit/mcp/antTools.test.ts \
  unit/mcp/automationTools.test.ts \
  unit/mcp/connectionTools.test.ts \
  unit/mcp/defectTools.test.ts \
  unit/mcp/nitroXTools.test.ts \
  unit/mcp/pageObjectGenerate.test.ts \
  unit/mcp/pageObjectValidate.test.ts \
  unit/mcp/projectValidateFromPath.test.ts \
  unit/mcp/propertiesTools.test.ts \
  unit/mcp/qualityHubApiTools.test.ts \
  unit/mcp/qualityHubTools.test.ts \
  unit/mcp/rcaTools.test.ts \
  unit/mcp/testCaseGenerate.test.ts \
  unit/mcp/testCaseStepTools.test.ts \
  unit/mcp/testCaseValidate.test.ts \
  unit/mcp/testPlanTools.test.ts \
  unit/mcp/testPlanValidate.test.ts \
  unit/mcp/testSuiteValidate.test.ts

⚠️ Missing Coverage · 2 files

These source files have no mapped test — consider adding coverage before merge.

  • src/mcp/tools/descHelper.ts
  • src/mcp/tools/projectInspect.ts

💡 Run locally: qo stub src/mcp/tools/descHelper.ts to generate a test scaffold.

⚡ quality-orchestrator  ·  /qo stub <file>  ·  qo analyze-local

Copilot AI reviewed May 13, 2026
View reviewed changes
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR tunes the ProvarDX MCP server startup footprint by (1) adding a “compact schema” mode that shortens tool/parameter descriptions and (2) allowing tool registration to be filtered by group via an environment variable, reducing initial handshake token usage.

Changes:

  • Added desc(standard, compact) helper and wrapped tool + Zod parameter descriptions across MCP tools for PROVAR_MCP_SCHEMA_MODE=compact.
  • Implemented tool-group registry + PROVAR_MCP_TOOLS parsing in createProvarMcpServer() to conditionally register tool groups (with provardx_ping always present).
  • Added unit tests for compact mode and group parsing, and documented both new env vars + performance-tuning guidance.

Reviewed changes

Copilot reviewed 23 out of 23 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
test/unit/mcp/startupTuning.test.ts Adds unit coverage for compact descriptions and PROVAR_MCP_TOOLS parsing behavior.
src/mcp/tools/descHelper.ts Introduces desc() helper to switch between standard/compact strings at runtime.
src/mcp/server.ts Adds tool-group registry, parseActiveGroups(), and group-filtered registration loop; updates ping description.
src/mcp/tools/testSuiteValidate.ts Wraps tool + parameter descriptions with desc().
src/mcp/tools/testPlanValidate.ts Wraps tool + parameter descriptions with desc().
src/mcp/tools/testPlanTools.ts Wraps tool + parameter descriptions with desc().
src/mcp/tools/testCaseValidate.ts Wraps tool + parameter descriptions with desc().
src/mcp/tools/testCaseStepTools.ts Wraps tool + parameter descriptions with desc().
src/mcp/tools/testCaseGenerate.ts Wraps tool + parameter descriptions with desc(); reorders some schema fields while doing so.
src/mcp/tools/rcaTools.ts Wraps tool + parameter descriptions with desc().
src/mcp/tools/qualityHubTools.ts Wraps tool + parameter descriptions with desc().
src/mcp/tools/qualityHubApiTools.ts Wraps tool + parameter descriptions with desc().
src/mcp/tools/propertiesTools.ts Wraps tool + parameter descriptions with desc().
src/mcp/tools/projectValidateFromPath.ts Wraps tool + parameter descriptions with desc().
src/mcp/tools/projectInspect.ts Wraps tool + parameter descriptions with desc().
src/mcp/tools/pageObjectValidate.ts Wraps tool + parameter descriptions with desc().
src/mcp/tools/pageObjectGenerate.ts Wraps tool + parameter descriptions with desc().
src/mcp/tools/nitroXTools.ts Wraps tool + parameter descriptions with desc().
src/mcp/tools/defectTools.ts Wraps tool + parameter descriptions with desc().
src/mcp/tools/connectionTools.ts Wraps tool + parameter descriptions with desc().
src/mcp/tools/automationTools.ts Wraps tool + parameter descriptions with desc().
src/mcp/tools/antTools.ts Wraps tool + parameter descriptions with desc().
docs/mcp.md Documents PROVAR_MCP_SCHEMA_MODE + PROVAR_MCP_TOOLS and adds performance-tuning section.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/mcp/server.ts Outdated
Comment on lines +78 to +83
return new Set(
env
.split(',')
.map((g) => g.trim().toLowerCase())
.filter(Boolean)
);
Comment thread src/mcp/tools/projectValidateFromPath.ts Outdated
.describe(
desc(
'Minimum quality score for a test case to be considered valid (default: 80)',
'int 0–100, optional; minimum quality score threshold'
Comment thread docs/mcp.md Outdated
PROVAR_MCP_SCHEMA_MODE=compact
```

When set to `compact`, every tool description and parameter description is replaced with a short summary (typically ≤15 words). This can save hundreds of tokens per tool in the initial context handshake, at the cost of reduced in-description guidance for the agent.
Comment thread test/unit/mcp/startupTuning.test.ts
Comment on lines +83 to +86
it('returns null when env var is empty string', () => {
process.env['PROVAR_MCP_TOOLS'] = '';
assert.equal(parseActiveGroups(), null);
});
mrdailey99 and others added 3 commits May 13, 2026 12:04
@mrdailey99 @claude
PDX-469: fix(mcp): patch parseActiveGroups comma-only edge case; add …
2714a8a 
…--profile to smoke script

RCA: parseActiveGroups returned an empty Set for inputs like PROVAR_MCP_TOOLS=","
which caused no tool groups to register (silent outage). provardx_ping message
param was not routed through desc(), making the docs/mcp.md claim "every parameter
is replaced" inaccurate. quality_threshold compact desc said "int" but the Zod
schema uses z.number(). Smoke script TOTAL_EXPECTED was hardcoded so --profile
had no way to adjust the expected count for partial runs.
Fix: parseActiveGroups now checks groups.size===0 post-filter and returns null
(all groups) with a warn log. provardx_ping message routes through desc().
projectValidateFromPath quality_threshold compact desc changed from "int" to
"number". docs/mcp.md line 484 softened from "every parameter" to "most".
mcp-smoke.cjs gains --profile flag, inGroup() helper, PROVAR_MCP_TOOLS passthrough
to server env, dynamic expectedCount, and group-conditional callTool wrappers.
Adds desc() unit tests and "," / ",," edge cases to startupTuning.test.ts.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@mrdailey99 @claude
PDX-477: docs(mcp): fix compact description claim and add env var doc…
779edf4 
…s for PR #170

RCA: CLAUDE.md requires docs updates for env var and tool description changes; PR #170 added PROVAR_MCP_SCHEMA_MODE and PROVAR_MCP_TOOLS support with an overclaiming compact description sentence
Fix: Changed tool descriptions and parameter descriptions to most tool and parameter descriptions to accurately reflect that provardx_ping message param is not routed through desc() helper

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@mrdailey99 @claude
PDX-477: chore(release): bump version to 1.5.1
1f68270 
RCA: Version bump required for the 1.5.1 release cycle covering PDX-468 through PDX-475.
Fix: Update package.json and server.json to 1.5.1 in sync per CLAUDE.md convention.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@mrdailey99 mrdailey99 merged commit 273ba77 into develop May 14, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

Copilot code review Copilot Copilot left review comments

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@mrdailey99