Skip to content

Safe-outputs: add schema-driven synonym mapping (incl. camelCase aliases), keep synonym metadata internal to MCP/CLI prompts, and enforce strict unknown-parameter errors; clarify schema constraints#37421

Merged
pelikhan merged 7 commits into
mainfrom
copilot/safeoutputs-disambiguate-field-names
Jun 6, 2026

Conversation

Copilot AI commented Jun 6, 2026

Copy link
Copy Markdown
Contributor

Safe-output runs were incurring avoidable schema rejections due to ambiguous parameter naming (level vs severity, commit_message vs message), undocumented size constraints, and noop payload shape confusion. This change adds schema-driven parameter synonym acceptance, keeps synonym metadata internal (not exposed to LLM-facing MCP/CLI schemas), and upgrades strict-schema validation to reject unknown parameters with nearest valid-field suggestions.

  • Argument normalization: schema-defined synonyms

    • Extended safe-output argument normalization to read per-field x-synonyms from inputSchema.properties and remap aliases to canonical fields before required-field validation.
    • Plumbed tool schema into the normalizer for both stdio and HTTP safe-output MCP server paths.
    • Expanded synonym coverage for likely LLM naming mistakes by adding camelCase aliases for snake_case parameters across safe-output schemas (for example, issueNumberissue_number).
  • Synonym metadata kept internal (no prompt pollution)

    • Added schema sanitization so internal correction metadata (for example x-synonyms) is stripped from MCP-exposed tool schemas (tools/list) and prompt-facing generation paths.
    • Preserved runtime synonym recovery by retaining original schemas internally for argument normalization.
  • Strict unknown-parameter handling

    • Added strict-schema unknown-parameter detection (additionalProperties: false) in MCP core request handling.
    • Unknown keys now fail fast with explicit offending fields plus closest valid parameter suggestions and a bounded preview of supported parameters.
  • Safe-output schema/docs disambiguation and constraints

    • Updated both schema copies (pkg/workflow/js/safe_outputs_tools.json and actions/setup/js/safe_outputs_tools.json) to:
      • clarify create_code_scanning_alert.severity (“NOT level”) and add x-synonyms: ["level"]
      • clarify push_to_pull_request_branch.message (“NOT commit_message”) and add x-synonyms: ["commit_message"]
      • clarify noop.message as the only content field and require serializing structured metrics into that string
      • document create_discussion.body max size (10 KB / 10240 bytes) with overflow guidance
    • Added/normalized synonym metadata where existing docs previously referenced stripped alternates (e.g., categoryId/category_id, issue_number).
  • Focused regression coverage

    • Added tests for synonym remapping and for strict unknown-parameter rejection with closest-field suggestions.
    • Added regression coverage for camelCase alias remapping behavior.
    • Added regression coverage confirming synonym metadata is not exposed via MCP tools/list while synonym arguments are still accepted in tools/call.
// Example: schema-driven synonym remap + strict validation behavior
{
  inputSchema: {
    type: "object",
    additionalProperties: false,
    required: ["severity", "message"],
    properties: {
      severity: { type: "string", "x-synonyms": ["level"] },
      message: { type: "string" }
    }
  }
}

// call args: { level: "warning", message: "..." } -> normalized to { severity: "warning", message: "..." }
// call args: { sev: "warning", message: "..." }   -> rejected: unknown parameter 'sev' (closest: 'severity')
// tools/list output omits internal x-synonyms metadata

Copilot AI and others added 3 commits June 6, 2026 22:05
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copilot AI changed the title [WIP] Disambiguate field names and document constraints for safeoutputs Safe-outputs: add parameter synonym mapping and strict unknown-parameter errors; clarify schema constraints Jun 6, 2026
Copilot AI requested a review from pelikhan June 6, 2026 22:21
@pelikhan

pelikhan commented Jun 6, 2026

Copy link
Copy Markdown
Collaborator

@copilot merge main, populate synonym with likely LLM mistakes

Copilot AI and others added 2 commits June 6, 2026 22:46
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Copilot AI commented Jun 6, 2026

Copy link
Copy Markdown
Contributor Author

@copilot merge main, populate synonym with likely LLM mistakes

Done in 3a27394 (merged main) and ddbd6a8 (expanded likely LLM synonym coverage with camelCase aliases across safe-output parameters, plus regression test update).

Copilot AI changed the title Safe-outputs: add parameter synonym mapping and strict unknown-parameter errors; clarify schema constraints Safe-outputs: add schema-driven synonym mapping (incl. camelCase aliases) and strict unknown-parameter errors; clarify schema constraints Jun 6, 2026
@pelikhan

pelikhan commented Jun 6, 2026

Copy link
Copy Markdown
Collaborator

@copilot make sure the synonyms are not communicated to the LLM through the MCP or cli generation. They are internal correction data, but should not pollute the prompts.

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Copilot AI commented Jun 6, 2026

Copy link
Copy Markdown
Contributor Author

@copilot make sure the synonyms are not communicated to the LLM through the MCP or cli generation. They are internal correction data, but should not pollute the prompts.

Implemented in e7d29ba. x-synonyms now stays internal: safe-output tools still accept synonym arguments during normalization, but MCP-exposed schemas no longer include synonym metadata in tools/list.

Copilot AI changed the title Safe-outputs: add schema-driven synonym mapping (incl. camelCase aliases) and strict unknown-parameter errors; clarify schema constraints Safe-outputs: add schema-driven synonym mapping (incl. camelCase aliases), keep synonym metadata internal to MCP/CLI prompts, and enforce strict unknown-parameter errors; clarify schema constraints Jun 6, 2026
@pelikhan pelikhan marked this pull request as ready for review June 6, 2026 23:31
Copilot AI review requested due to automatic review settings June 6, 2026 23:31
@pelikhan pelikhan merged commit d25afa6 into main Jun 6, 2026
1 check passed
@pelikhan pelikhan deleted the copilot/safeoutputs-disambiguate-field-names branch June 6, 2026 23:33

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR improves safe-outputs tool-call robustness by (1) normalizing tool arguments using schema-defined synonyms (including camelCase aliases), (2) stripping internal synonym metadata from MCP-exposed schemas, and (3) enforcing strict unknown-parameter rejection with “closest match” suggestions when schemas opt in via additionalProperties: false.

Changes:

  • Add schema-driven synonym remapping (x-synonyms) to safe-output argument normalization, and plumb the tool input schema into the normalizer (stdio + HTTP servers).
  • Sanitize tool schemas before exposing them via MCP (tools/list) by stripping x-synonyms, while retaining original schemas internally for runtime normalization.
  • Add strict unknown-parameter validation in MCP core with nearest-valid-field suggestions and a bounded “supported parameters” preview; extend tests accordingly.
Show a summary per file
File Description
pkg/workflow/js/safe_outputs_tools.json Adds x-synonyms + doc clarifications to the safe-outputs tool schemas shipped with the workflow package.
actions/setup/js/safe_outputs_tools.json Mirrors schema updates for the Actions setup copy (synonyms + doc clarifications).
actions/setup/js/safe_outputs_mcp_wrapped_arguments.test.cjs Adds regression tests for synonym remapping and ensuring synonyms are not exposed via tools/list.
actions/setup/js/safe_outputs_mcp_server.cjs Stores original schemas for normalization while stripping internal metadata from MCP-exposed schemas (stdio server).
actions/setup/js/safe_outputs_mcp_server_http.cjs Same schema plumbing/sanitization as above for the HTTP server path.
actions/setup/js/safe_outputs_mcp_arguments.cjs Implements synonym-based argument remapping and schema metadata stripping helper.
actions/setup/js/mcp_server_core.test.cjs Adds coverage for strict unknown-parameter rejection + closest-parameter suggestions.
actions/setup/js/mcp_server_core.cjs Implements unknown-parameter detection + error construction (gated by additionalProperties: false).

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

  • Files reviewed: 8/8 changed files
  • Comments generated: 2

"message": {
"type": "string",
"description": "Status or completion message to log. Should explain what was analyzed and the outcome (e.g., 'Code review complete - no issues found', 'Analysis complete - all tests passing')."
"description": "Status or completion message to log. Should explain what was analyzed and the outcome (e.g., 'Code review complete - no issues found', 'Analysis complete - all tests passing'). This is the only content field noop accepts. Serialize any metrics or structured run summary into this single string; additional named fields (for example processed, skipped) are not accepted and cause a missing-message error."
"message": {
"type": "string",
"description": "Status or completion message to log. Should explain what was analyzed and the outcome (e.g., 'Code review complete - no issues found', 'Analysis complete - all tests passing')."
"description": "Status or completion message to log. Should explain what was analyzed and the outcome (e.g., 'Code review complete - no issues found', 'Analysis complete - all tests passing'). This is the only content field noop accepts. Serialize any metrics or structured run summary into this single string; additional named fields (for example processed, skipped) are not accepted and cause a missing-message error."
@github-actions github-actions Bot mentioned this pull request Jun 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

3 participants