[Schema Consistency] Schema Consistency Check – 2026-03-26

[github-actions[bot]]

Summary

• Inconsistencies Found: 4 new (plus confirmed persisting findings from prior runs)
• Strategy Used: Strategy-37 — Schema Doc Generator Staleness + New Field Coverage Audit
• Day: 085 (mod 10 = 5 → proven strategy path)
• Key Method: Ran scripts/generate-schema-docs.js against live schema to detect drift, then cross-checked new fields against user-facing docs and changeset-driven artifact changes

---

New Findings

HIGH — observability.job-summary Completely Absent from Reference Docs

The observability top-level frontmatter field with its job-summary sub-field is fully implemented and tested, but is not documented anywhere in the reference docs.

• Schema: pkg/parser/schemas/main_workflow_schema.json — properties.observability.properties.job-summary defined with enum ["on","off"] ✅
• Code: pkg/workflow/compiler_yaml_ai_execution.go:8–22 — getObservabilityJobSummaryMode() reads data.ParsedFrontmatter.Observability.JobSummary; pkg/workflow/frontmatter_types.go:132–134 — ObservabilityConfig struct ✅
• Tests: pkg/workflow/observability_job_summary_test.go — covers both enabled and disabled cases ✅
• docs/src/content/docs/reference/frontmatter.md: ❌ ABSENT — zero mentions of observability
• docs/src/content/docs/reference/frontmatter-full.md: ❌ ABSENT — zero mentions (confirmed by grep -c observability = 0)

Running scripts/generate-schema-docs.js regenerated the file and added the missing section, confirming it was a staleness issue.

Impact: Users cannot discover the observability.job-summary: "on" feature from any reference documentation, despite it being production-ready and tested.

---

HIGH — imports Object Form (aw:/apm-packages: subfields) Not Documented in imports.md

The imports: field supports two forms — array form (array of paths) and object form with aw: (agentic workflow imports) and apm-packages: (APM package dependencies) subfields. The object form is fully implemented but completely absent from the imports reference.

• Schema: pkg/parser/schemas/main_workflow_schema.json:114–148 — imports.oneOf[1] defines object form with aw and apm-packages subfields ✅
• Code: pkg/parser/import_bfs.go:29–63 — handles both array and object forms; pkg/workflow/frontmatter_extraction_metadata.go:247–282 — extracts imports.apm-packages as the preferred APM config location ✅
• CLI codemod: pkg/cli/codemod_dependencies.go:11–12 — migrates old dependencies: field to imports.apm-packages: ✅
• docs/src/content/docs/reference/imports.md: ❌ ABSENT — only documents array form; no mention of object form, aw: subfield, or apm-packages: subfield

Critical gap: The compiler code at frontmatter_extraction_metadata.go:274–275 shows imports.apm-packages is the preferred location for APM packages (vs deprecated dependencies:), yet imports.md gives users no visibility into this. Users migrating from dependencies: follow the codemod but find no documentation explaining the resulting structure.

# Object form of imports — documented only in schema, not in imports.md
imports:
  aw:
    - shared/common-tools.md
  apm-packages:           # Preferred location for APM packages (dependencies: is deprecated)
    - microsoft/apm-sample-package
```

---

#### MEDIUM — `frontmatter-full.md` Is Stale (Not Regenerated After Schema Changes)

`docs/src/content/docs/reference/frontmatter-full.md` is described as "automatically generated from the JSON Schema," but it has not been regenerated after recent schema additions.

Running `scripts/generate-schema-docs.js` produced **77 new lines**, including:
1. The `observability` section (job-summary field)
2. Expanded `imports` documentation showing both array form and object form with `aw`, `apm-packages`, `isolated`, `github-app`, `env`, and `github-token` subfields

**Root cause**: No CI check enforces that `frontmatter-full.md` is regenerated when `main_workflow_schema.json` changes. A `make docs-check` or similar step that runs the generator and fails if the output differs would prevent this class of staleness.

---

#### MEDIUM — `debugging.md` Artifact Table Shows Stale Firewall Artifact Path

`docs/src/content/docs/troubleshooting/debugging.md:308` lists:

```
| `firewall-logs/` | `/tmp/gh-aw/firewall-logs/` | Network access logs |

But the recent patch-upload-firewall-audit-logs-artifact changeset changed this entirely:

• New artifact name: firewall-audit-logs (constant constants.FirewallAuditArtifactName, pkg/constants/constants.go:387)
• New paths uploaded: /tmp/gh-aw/sandbox/firewall/logs/ + /tmp/gh-aw/sandbox/firewall/audit/ (see pkg/workflow/engine_firewall_support.go:124–138)
• Old table entry path /tmp/gh-aw/firewall-logs/ does not exist in the codebase

Users following the debugging guide will look for an artifact named firewall-logs/ that no longer exists and will not find the new firewall-audit-logs artifact.

---

Confirmed Persisting Issues (from Prior Runs)

Previously reported, still unresolved

ID	Severity	Finding
S30-1	CRITICAL	status-comment schema says "Must be explicitly set to true" but compiler_safe_outputs.go:192–197 auto-enables it for slash_command/label_command triggers
S30-2	HIGH	proxy-args MCP server field implemented in mcp_config_custom.go:661 but absent from $defs/stdio_mcp_tool.properties in schema (additionalProperties:false → fails validation)
S31-1	CRITICAL	update-discussion labels/allowed-labels is a dead feature — schema and docs define it, but no LLM tool parameter and no handler code in update_discussion.cjs
S32-1	HIGH	Codemod ID sandbox-agent-false-removal listed in mcp_tools_management.go:171 and gh-aw-as-mcp-server.md:284 but does not exist in fix_codemods.go; actual ID is sandbox-false-to-agent-false
S33-1	HIGH	close-older-key field fully implemented in schema and code but absent from all documentation
S34-1	MEDIUM	safe-outputs.actions.inputs in schema and code (parseActionsConfig) but missing from custom-safe-outputs.md Action Reference table
S35-1	HIGH	set-issue-type safe output absent from safe-outputs.md (exists in schema, code, spec)
S35-2	HIGH	mark-pull-request-as-ready-for-review absent from safe-outputs.md
S35-3	MEDIUM	features.copilot-requests flag fully active in code but completely undocumented
S36-1	CRITICAL	MCPGatewayFeatureFlag = "mcp-gateway" never checked via isFeatureEnabled(); schema self-contradicts between top-level sandbox description and mcp field
S36-2	HIGH	MCPScriptsFeatureFlag = "mcp-scripts" is dead code — activation uses HasMCPScripts() not feature flag
S36-3	HIGH	DisableXPIAPromptFeatureFlag = "disable-xpia-prompt" used in unified_prompt_step.go:106 but completely absent from docs and schema examples

---

Recommendations

1. Run scripts/generate-schema-docs.js in CI — Add a check that fails if frontmatter-full.md differs from the generated output, to prevent staleness whenever the schema changes.

2. Document observability.job-summary in frontmatter.md — Add a section for observability: with job-summary: on/off, explaining it appends a compact observability section to GitHub Actions job summaries.

3. Document imports object form in imports.md — Add a section showing both array and object form, emphasizing that imports.apm-packages: is the preferred way to configure APM packages (and that dependencies: is deprecated).

4. Update debugging.md artifact table — Replace the stale firewall-logs/ row with firewall-audit-logs (paths: /tmp/gh-aw/sandbox/firewall/logs/ and /tmp/gh-aw/sandbox/firewall/audit/).

---

Strategy Performance

• Strategy: Schema Doc Generator Staleness + New Field Coverage Audit
• Method: Run generator script, diff output, cross-check with user-facing docs
• Findings: 4 actionable (2 HIGH, 2 MEDIUM)
• Effectiveness: HIGH — running the generator directly reveals staleness that text grep would miss
• Should Reuse: YES — generalize to other generator scripts when they exist

---

Next Steps

• Add CI step: node scripts/generate-schema-docs.js && git diff --exit-code docs/src/content/docs/reference/frontmatter-full.md
• Document observability.job-summary in frontmatter.md
• Document imports object form (aw/apm-packages) in imports.md
• Update debugging.md artifact table for firewall-audit-logs
• Address persisting critical issues (status-comment auto-enable, update-discussion labels dead code, proxy-args schema gap)

References: §23577506476

AI generated by Schema Consistency Checker · history

• expires on Mar 27, 2026, 4:36 AM UTC

[github-actions[bot]]

🤖 Beep boop! The smoke test agent has arrived and is checking the premises. I've verified your schema consistency findings are noted and filed appropriately. Nothing to see here (except the smoke test passing)! 🚬✅

📰 BREAKING: Report filed by Smoke Copilot · ◷

[github-actions[bot]]

💥 WHOOSH!

The Claude Smoke Test Agent swoops in, cape billowing!

⚡ ZAP! Smoke test agent 23579840642 was here!

🦸 The Claude engine roared to life, tested all systems, and found everything NOMINAL! Schema inconsistencies noted, dependencies validated — the agentic workflows universe remains safe!

POW! 🎉 flies off into the CI/CD sunset

💥 [THE END] — Illustrated by Smoke Claude · ◷

[github-actions[bot]]

This discussion has been marked as outdated by Schema Consistency Checker.

A newer discussion is available at Discussion #23209.