Refactor safe output type validation into data-driven validator engine #5086

Copilot · 2025-11-29T14:53:33Z

Original prompt

Refactor a mini safe output type validation helper/engine from the collect ndjson JavaScript file.

use information in tools.json to perform validation. Extend tools.json with extra fields, like "sanitize: true" to be generic and data driven instead of special handling in code

move validation engine to own JavaScript file, add tests

update compiler to generate new flags

register with bundler

refactor collect JavaScript to use validation engine and reduce duplicated code.

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

Changeset

Type: patch
Description: Refactor safe output type validation into a data-driven validator engine; moves validation into a JS validator, generates validation config from Go (single source of truth), updates the collector to use the new engine, and adds tests.

AI generated by Changeset Generator

Smoke Test Summary

Test Run: 2025-11-29T16:58:05Z
Status: PASS
Summary: All smoke tests passed - GitHub MCP, file writing, bash tools, and Playwright validated

AI generated by Smoke Copilot No Firewall

- Create safe_output_type_validator.cjs with VALIDATION_CONFIG containing all safe output types - Move validation logic from collect_ndjson_output.cjs to validator engine - Add sanitize, maxLength, itemSanitize, itemMaxLength flags in VALIDATION_CONFIG - Register validator with bundler in js.go - Add comprehensive test suite for the validator - Update collect_ndjson_output.cjs to use the new validator - Fix test assertions to match new consistent error message format - Fix Go tests that were naively matching JavaScript object properties as YAML keys Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

pelikhan · 2025-11-29T15:55:15Z

.github/workflows/archie.lock.yml

-                  if (fieldName.includes("create_code_scanning_alert 'line'")) {
+            const MAX_BODY_LENGTH = 65000;
+            const MAX_GITHUB_USERNAME_LENGTH = 39;
+            const VALIDATION_CONFIG = {


@copilot VALIDATON_CONFIG should be part of the Go compiler and use at compile time to generate the safe output tools.json / config.json files. It should NOT be part of the JavaScript code. The entire configuration should be loaded from the config file.

Moved VALIDATION_CONFIG to Go as the single source of truth in pkg/workflow/safe_output_validation_config.go. The Go compiler now generates validation.json at compile time, which JavaScript loads at runtime. The JavaScript code no longer contains any hardcoded validation config. (2672e21)

- Add pkg/workflow/safe_output_validation_config.go with all validation rules - Add pkg/workflow/safe_output_validation_config_test.go with comprehensive tests - Update mcp_servers.go to write validation.json from Go config - Update collect_ndjson_output.cjs to load from validation.json file - Update safe_output_type_validator.cjs to load config from env var - Fix test setup in collect_ndjson_output.test.cjs to provide validation config - Fix safe_output_type_validator.test.cjs sanitization test assertion The VALIDATION_CONFIG is now part of the Go compiler and generates the validation.json file at compile time. JavaScript code loads this config at runtime instead of having duplicate inline config. Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

pelikhan · 2025-11-29T16:27:39Z

@copilot Only generate validation info for activated tools in the safe output MCP server to keep validation.json small. Write indented JSON to avoid merge issues.

…onfig - Upgrade validation config parsing error from warning to error level - Add 'error' mock to safe_output_type_validator tests - Update test assertion to check for error instead of warning Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Copilot · 2025-11-29T16:38:24Z

Just as a heads up, I was blocked by some firewall rules while working on your feedback. Expand below for details.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

https://api.github.com/user
- Triggering command: /usr/bin/gh gh api user --jq .login (http block)

If you need me to access, download, or install something from one of these locations, you can either:

Configure Actions setup steps to set up my environment, which run before the firewall is enabled
Add the appropriate URLs or hosts to the custom allowlist in this repository's Copilot coding agent settings (admins only)

- Update GetValidationConfigJSON to accept enabledTypes filter parameter - Extract enabled types from safeOutputConfig in mcp_servers.go - Use json.MarshalIndent for pretty-printed JSON output - Add tests for filtered and unfiltered validation config generation - validation.json now only contains configs for activated safe output types Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Copilot · 2025-11-29T16:48:47Z

@copilot Only generate validation info for activated tools in the safe output MCP server to keep validation.json small. Write indented JSON to avoid merge issues.

Implemented in c798020. GetValidationConfigJSON now accepts an enabledTypes parameter and only includes validation configs for types present in the workflow's safe-outputs configuration. JSON is now indented with 2-space formatting using json.MarshalIndent.

github-actions · 2025-11-29T16:56:17Z

🔮 The ancient spirits stir... Smoke Codex awakens to divine this pull request...

github-actions · 2025-11-29T16:56:18Z

📰 DEVELOPING STORY: Smoke Copilot reports failed. Our correspondents are investigating the incident...

github-actions · 2025-11-29T16:56:19Z

💥 WHOOSH! Smoke Claude springs into action on this pull request! [Panel 1 begins...]

github-actions · 2025-11-29T16:56:19Z

🎉 Yo ho ho! Changeset Generator found the treasure and completed successfully! ⚓💰

github-actions · 2025-11-29T16:56:20Z

🤖 SYSTEM_INIT: Smoke Copilot No Firewall ACTIVATED. PROCESSING pull request. ALL SUBSYSTEMS ONLINE.

pelikhan · 2025-11-29T16:56:45Z

@copilot update add-safe-output-type agent.md file with update instructions to add a safe output type and the new validation scheme

github-actions · 2025-11-29T16:59:09Z

Merged PRs:

Update changeset generator to use codex gpt-5-mini
Update workflow compiler to generate secure template expansion patterns
Tests:
GitHub MCP ✅
File write ✅
Playwright ✅
Overall: PASS

🔮 The oracle has spoken through Smoke Codex fer issue #5086 🗺️

github-actions · 2025-11-29T16:59:12Z

Smoke Test Summary

Status: PASS ✅

Test Results

✅ GitHub MCP: Last 2 merged PRs retrieved
- Update changeset generator to use codex gpt-5-mini #5088: Update changeset generator to use codex gpt-5-mini
- Update workflow compiler to generate secure template expansion patterns #5084: Update workflow compiler to generate secure template expansion patterns
✅ File Writing: Created /tmp/smoke-test-claude-19786705773.txt
✅ Bash Tool: Verified file contents successfully
✅ Playwright MCP: Navigated to github.com, title contains "GitHub"

💥 [THE END] — Illustrated by Smoke Claude fer issue #5086 🗺️

github-actions · 2025-11-29T16:59:18Z

Smoke Test Results - 2025-11-29T16:58:05Z

Last 2 Merged PRs:

Update changeset generator to use codex gpt-5-mini #5088: Update changeset generator to use codex gpt-5-mini
Update workflow compiler to generate secure template expansion patterns #5084: Update workflow compiler to generate secure template expansion patterns

Test Results:

✅ GitHub MCP: Retrieved PR data successfully
✅ File Writing: Created /tmp/smoke-test-copilot-19786705785.txt
✅ Bash Tools: Verified file content
✅ Playwright MCP: Navigated to GitHub, title confirmed

Status: PASS

🤖 DIAGNOSTIC REPORT GENERATED BY Smoke Copilot No Firewall fer issue #5086 🗺️

Initial plan

604603a

Copilot AI assigned Copilot and pelikhan Nov 29, 2025

Copilot started work on behalf of pelikhan November 29, 2025 14:54 View session

Copilot AI changed the title ~~[WIP] Refactor mini safe output type validation engine~~ Refactor safe output type validation into data-driven validator engine Nov 29, 2025

Copilot AI requested a review from pelikhan November 29, 2025 15:50

Copilot finished work on behalf of pelikhan November 29, 2025 15:50

pelikhan reviewed Nov 29, 2025

View reviewed changes

Copilot started work on behalf of pelikhan November 29, 2025 15:55 View session

Copilot AI and others added 2 commits November 29, 2025 16:30

Refactor safe output type validation into data-driven validator engine

f842be8

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Copilot AI requested a review from pelikhan November 29, 2025 16:38

Copilot finished work on behalf of pelikhan November 29, 2025 16:38

Copilot started work on behalf of pelikhan November 29, 2025 16:39 View session

pelikhan added the smoke label Nov 29, 2025

pelikhan marked this pull request as ready for review November 29, 2025 16:58

github-actions bot added the smoke-codex label Nov 29, 2025

github-actions bot added smoke-claude smoke-copilot-no-firewall labels Nov 29, 2025

Add changeset for PR #5086: refactor safe output validation [skip-ci]

a8ed1fc

pelikhan merged commit d388503 into main Nov 29, 2025
4 checks passed

pelikhan deleted the copilot/refactor-validation-helper-engine branch November 29, 2025 16:59

Copilot stopped work on behalf of pelikhan due to an error November 29, 2025 17:00
Copilot has encountered an error. See logs for additional details.

github-actions bot mentioned this pull request Nov 29, 2025

Add support for loading tool handlers from external files in MCP server core #5085

Merged

21 tasks

Refactor safe output type validation into data-driven validator engine #5086

Refactor safe output type validation into data-driven validator engine #5086

Uh oh!

Conversation

Copilot AI commented Nov 29, 2025 • edited by github-actions bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changeset

Smoke Test Summary

Uh oh!

pelikhan Nov 29, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 29, 2025

Choose a reason for hiding this comment

Uh oh!

pelikhan commented Nov 29, 2025

Uh oh!

Copilot AI commented Nov 29, 2025

I tried to connect to the following addresses, but was blocked by firewall rules:

Uh oh!

Copilot AI commented Nov 29, 2025

Uh oh!

github-actions bot commented Nov 29, 2025

Uh oh!

github-actions bot commented Nov 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Nov 29, 2025

Uh oh!

github-actions bot commented Nov 29, 2025 • edited by agentic-workflows-dev bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Nov 29, 2025

Uh oh!

pelikhan commented Nov 29, 2025

Uh oh!

github-actions bot commented Nov 29, 2025

Uh oh!

github-actions bot commented Nov 29, 2025

Smoke Test Summary

Test Results

Uh oh!

github-actions bot commented Nov 29, 2025

Smoke Test Results - 2025-11-29T16:58:05Z

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Copilot AI commented Nov 29, 2025 •

edited by github-actions bot

Loading

github-actions bot commented Nov 29, 2025 •

edited

Loading

github-actions bot commented Nov 29, 2025 •

edited by agentic-workflows-dev bot

Loading