[Schema Consistency] Schema consistency check report - 2026-08-13

[github-actions[bot]]

🔍 Schema Consistency Check - 2026-08-13

Summary

• Inconsistencies Found: 4
• Categories Analyzed: Schema, Parser/Compiler, Documentation, Workflows
• Strategy Used: Field gap triage + targeted parser/doc verification
• New Strategy: No

Critical Issues

1. Precomputed parser field diff is misleading because parser extraction is empty

   • schema-diff.json reports parser_yaml_fields: [], making every schema field appear to be missing from the parser.
   • Actual implementation clearly parses top-level frontmatter via JSON unmarshalling into FrontmatterConfig in pkg/workflow/frontmatter_parsing.go, with fields declared in pkg/workflow/frontmatter_types.go.
   • Example implemented fields include runs-on-slim, resources, max-daily-ai-credits, and check-for-updates.
   • Impact: any analysis workflow that prioritizes field_gaps.in_schema_not_parser will generate false positives unless it compensates for this data quality issue.

2. Schema wording for max-daily-ai-credits contradicts documentation

   • Schema (pkg/parser/schemas/main_workflow_schema.json): description says the guardrail is for "runs triggered by the same user".
   • Docs (docs/src/content/docs/reference/frontmatter.md, rate-limiting-controls.md): describe it as a per-workflow rolling 24-hour cap aggregated across recent runs of the workflow.
   • Implementation/tests also indicate workflow-level semantics, not per-user semantics (for example pkg/workflow/daily_aic_workflow_guardrail_test.go references workflow-wide behavior).
   • Impact: authors may incorrectly assume separate per-user budgets when the system actually gates the workflow globally.

Documentation Gaps

1. Frontmatter reference omits the documented default for check-for-updates in generated reference style

   • Schema includes "default": true for check-for-updates.
   • User-facing guide documents the field, but the generated full reference only shows examples and does not make the default behavior especially visible compared with other operationally important fields.
   • Implementation (pkg/workflow/strict_mode_update_check_validation.go) treats the default-enabled behavior as security-relevant.
   • Impact: workflow authors may disable it without realizing they are deviating from the default safety posture.

2. Workflows do not exercise several documented/schema top-level fields

   • No in-repo workflow currently uses top-level ambient-folders, resources, runs-on-slim, max-daily-ai-credits, check-for-updates, or github-app.
   • These fields are documented and implemented, but absence of real examples reduces confidence and increases drift risk.
   • Impact: future refactors can regress these fields without an obvious real-world fixture in .github/workflows/.

Schema Improvements Needed

1. Fix max-daily-ai-credits description

   • Update the schema text to match docs and implementation: workflow-scoped rolling 24-hour cap, not same-user scoped.

2. Make schema-diff generation parser-aware

   • The current precomputed diff appears to rely on YAML tags in pkg/parser/*.go, but top-level frontmatter is actually represented in pkg/workflow/frontmatter_types.go with json tags and parsed in pkg/workflow/frontmatter_parsing.go.
   • The diff generator should treat these typed frontmatter structs as parser/compiler coverage.

Parser / Compiler Notes

1. Parser/compiler coverage exists for fields the diff marks as missing
   • runs-on-slim → pkg/workflow/frontmatter_types.go
   • resources → pkg/workflow/frontmatter_types.go plus validation in pkg/workflow/compiler_validators.go
   • check-for-updates → pkg/workflow/frontmatter_types.go plus enforcement in pkg/workflow/strict_mode_update_check_validation.go
   • ambient-folders → pkg/workflow/ambient_folders.go
   • github-app → pkg/workflow/workflow_github_app.go
   • Impact: this is not a product bug in those fields; it is a consistency bug in the analysis source-of-truth.

Workflow Violations

1. in_used_not_schema in precomputed data contains many false positives from markdown bodies, not frontmatter
   • Examples include title, repo, authors, try, and else.
   • Spot checks show these appear inside embedded scripts or YAML examples within workflow markdown, not as top-level frontmatter keys.
   • Impact: relying on raw grep-style extraction from workflow files overreports schema violations.

Recommendations

1. Update the schema description for max-daily-ai-credits to say per-workflow rather than same user.
2. Regenerate the precomputed schema diff with a parser that understands pkg/workflow/frontmatter_types.go and frontmatter-only sections of workflow markdown.
3. Add at least one repository workflow fixture using seldom-exercised fields such as ambient-folders, resources, runs-on-slim, or top-level github-app.
4. Tighten workflow-key extraction so only frontmatter keys are counted; ignore markdown body content and embedded scripts.

Strategy Performance

• Strategy Used: Field gap triage + targeted parser/doc verification
• Findings: 4
• Effectiveness: High
• Should Reuse: Yes

Next Steps

• Fix schema wording for max-daily-ai-credits
• Improve schema-diff generation for parser/compiler coverage
• Improve workflow frontmatter key extraction
• Add example workflows covering underused documented fields

Generated by ✅ Schema Consistency Checker · gpt54 · 19.3 AIC · ⌖ 4.9 AIC · ⊞ 8.5K · ◷

• expires on Aug 13, 2026, 9:57 PM UTC-08:00

[github-actions[bot]]

This discussion was automatically closed because it expired on 2026-08-14T05:57:12.738Z.

Closed by Workflow