[Schema Consistency] Schema consistency check - 2026-08-29

[github-actions[bot]]

🔍 Schema Consistency Check - 2026-08-29

Summary

• Inconsistencies Found: 3
• Categories Analyzed: Schema, Parser, Documentation, Workflows
• Strategy Used: Schema-def spot checks + targeted parser/doc/workflow verification
• New Strategy: No
• AWF config-source drift note: Canonical-source retrieval was degraded, so no AWF config-source drift claims below are authoritative.

Critical Issues

1. roles is still documented and typed as a top-level frontmatter field, but the parser only honors on.roles.

   • Schema: roles is present under on and the docs present it as part of trigger configuration.
   • Typed frontmatter: FrontmatterConfig still exposes a top-level roles-adjacent world indirectly through generic On map[string]any, and precomputed field lists include roles among workflow yaml fields.
   • Parser behavior: extractRoles() only reads frontmatter["on"].roles; it does not read top-level frontmatter["roles"] and falls back to default roles otherwise (pkg/workflow/role_checks.go:134-150).
   • Evidence of drift: tests explicitly state “Top-level roles is no longer supported” (pkg/workflow/extract_roles_test.go).
   • Impact: users following older examples or generated docs that place roles: at the top level silently get default [admin, maintainer, write] instead of their intended restriction.

2. stop-after is implemented and documented under on:, but it is not represented in typed frontmatter extraction.

   • Schema/docs: on.stop-after is fully documented (docs/src/content/docs/reference/frontmatter-full.md:865+).
   • Parser/compiler: extractStopAfterFromOn() reads frontmatter["on"]["stop-after"] dynamically and compiles it (pkg/workflow/stop_after.go:16-46).
   • Typed frontmatter gap: FrontmatterConfig.On is only map[string]any; there is no typed field for stop-after, unlike many newer first-class config sections in FrontmatterConfig.
   • Impact: this creates a schema ↔ parser asymmetry where a meaningful supported field bypasses typed frontmatter validation/extraction paths, increasing the chance of future drift or partial feature support.

3. Permissions docs claim support for organization-* scopes, but the main GitHub Actions permissions schema section does not define some documented scopes.

   • Docs: frontmatter.md says GitHub App-only scopes such as organization-* are supported (docs/src/content/docs/reference/frontmatter.md:186).
   • Parser/types: GitHubAppPermissionsConfig includes organization-custom-org-roles and organization-custom-repository-roles (pkg/workflow/frontmatter_types.go:86,88), and parsing code handles them.
   • Schema: these two scopes are missing from $defs.github_actions_permissions.properties as queried directly, even though they appear elsewhere in the big generated schema/doc corpus.
   • Impact: schema-based tooling and validation can reject or fail to autocomplete permission scopes that the parser actually supports.

Documentation Gaps

1. Top-level roles: remains easy to misunderstand.

   • The generated full reference still shows many roles: examples, but the implementation has moved to on.roles semantics.
   • Recommendation: add an explicit “top-level roles is deprecated/ignored; use on.roles” warning in docs/src/content/docs/reference/frontmatter.md.

2. safe-outputs advanced auth/data fields are mostly only visible in the large generated reference.

   • Schema includes safe-outputs.urls, allowed-domains, data, github-token, and github-app.
   • The concise reference page does not give these features much visibility, making the small docs lag practical reality.

Schema Improvements Needed

1. Add/align GitHub App-only permission scopes in the schema definitions used for validation tooling, especially:

   • organization-custom-org-roles
   • organization-custom-repository-roles

2. Model on extension keys more consistently across schema and typed parser surfaces, especially high-value fields like:

   • stop-after
   • roles
   • skip-if-match
   • skip-if-no-match
   • github-token
   • github-app

Parser Updates Required

1. Either restore explicit top-level roles compatibility with a warning, or hard-fail on top-level roles.

   • Current behavior silently ignores it and applies defaults.
   • Silent fallback is more dangerous than a clear validation error.

2. Consider stronger typed extraction for on extension fields.

   • Relying on map[string]any for fields like stop-after and skip-if-* makes drift harder to detect automatically.

Workflow Violations

• No top-level workflow key usage outside the schema was found in the precomputed workflow scan (field_gaps.in_used_not_schema was empty).
• Representative repository workflows already appear to use current patterns such as safe-outputs, sandbox, network, and cache-memory under tools, so the highest-signal problems are schema/parser/doc drift rather than invalid in-repo workflow usage.

Recommendations

1. Add validation that errors when roles: appears at the top level, with a migration hint to on.roles.
2. Extend the permissions schema definition so schema-driven editors and validators match parser support for GitHub App-only scopes.
3. Introduce a typed OnConfig (or at minimum typed extraction helpers) for stop-after, roles, and skip-if auth fields.
4. Add a concise reference section for advanced safe-outputs auth/data configuration, not only the generated full reference.

Strategy Performance

• Strategy Used: Schema-def spot checks + targeted parser/doc/workflow verification
• Findings: 3
• Effectiveness: High
• Should Reuse: Yes

Next Steps

• Fix schema definitions for GitHub App-only permission scopes
• Update parser validation for misplaced top-level roles
• Update documentation to clarify on.roles vs deprecated top-level usage
• Consider typed parser coverage for on extension fields

Generated by ✅ Schema Consistency Checker · pi · gpt54 · 19 AIC · ⌖ 8.54 AIC · ⊞ 8.9K · ◷

• expires on Aug 29, 2026, 9:31 PM UTC-08:00