[Schema Consistency] Schema Consistency Check — 2026-06-08

[github-actions[bot]]

Summary

• Inconsistencies Found: 2 new (1 high-severity, with a related secondary issue) + 2 standing (re-confirmed)
• Categories Analyzed: Schema ↔ Parser/Compiler ↔ Runtime JS ↔ Documentation ↔ Workflows
• Strategy Used: S23 — Enum value reachability at runtime (API/source contract) (NEW)
• New Strategy: YES (day-of-year mod 10 = 9 → new-approach mode). Also ran NEW S22 (enum-vs-docs drift), 0 findings.

This run took a fresh angle: rather than checking whether enum values are validated or branched on, it checked whether enum values can ever match the real data they are compared against at runtime. That surfaced a high-value issue in the on.roles permission gate where several documented enum values appear to be unreachable against GitHub's actual API contract.

---

Critical Issues

🔴 on.roles enum values maintain / maintainer / triage are likely non-functional at runtime

The schema (pkg/parser/schemas/main_workflow_schema.json) and docs (docs/src/content/docs/reference/triggers.md:395) advertise the role values admin, maintainer, maintain, write, triage, read, all. The runtime permission gate, however, only inspects the permission field of the collaborator-permission API response and never reads role_name:

// actions/setup/js/check_permissions_utils.cjs:241-245
const permission = repoPermission.data.permission;
// ...
const hasPermission = requiredPermissions.some(requiredPerm =>
  permission === requiredPerm ||
  (requiredPerm === "maintainer" && permission === "maintain"));

Per GitHub's documented contract, GET /repos/{owner}/{repo}/collaborators/{username}/permission returns permission ∈ admin / write / read / none, while the granular roles (maintain, triage, custom roles) are reported in the separate role_name field. If that contract holds:

• A user with the maintain role reports permission: "write" → roles: [maintain] (or [maintainer]) never authorizes them.
• A user with the triage role reports permission: "read" → roles: [triage] never authorizes them.
• The requiredPerm === "maintainer" && permission === "maintain" alias branch (line 245) is therefore effectively dead code.

⚠️ Verification needed. Web access is unavailable inside this workflow, so the GitHub API contract could not be re-confirmed live. The unit tests mock permission: "maintain" and permission: "triage" (actions/setup/js/check_permissions.test.cjs:92,99), so they pass green regardless of real API behavior — meaning the test suite would not catch this. A maintainer should confirm against the live API whether permission ever returns maintain/triage.

Impact: A workflow author who writes roles: [maintain] or roles: [triage] (both schema-valid and documented) may silently get a gate that authorizes no one, or that behaves differently than the label implies — a security-relevant surprise.

Evidence trail

• Schema enum: properties.on.oneOf[1].properties.roles → [admin, maintainer, maintain, write, triage, read, all]
• Docs: triggers.md:395 — "Available roles: admin, maintainer/maintain, write, triage, read, all."
• Roles parsed verbatim and exported: pkg/workflow/role_checks.go:30 → GH_AW_REQUIRED_ROLES
• Consumed: actions/setup/js/check_permissions_utils.cjs:16 (parse) and :241-245 (compare)
• grep role_name actions/setup/js/*.cjs → no matches (the granular field is never read)
• Test mocks returning maintain/triage: check_permissions.test.cjs:92,99

🟠 Secondary: roles matching is exact-equality, not a permission threshold

The same .some(... permission === requiredPerm ...) logic has no hierarchy. Narrowing the gate to a single lower level silently excludes higher-privileged actors:

• roles: [write] → an admin actor reports permission: "admin", which is !== "write" → admin is rejected.

The default [admin, maintainer, write] masks this because it lists all three explicitly. But the docs describe on.roles as filtering "based on repository permission level" (triggers.md:385), which naturally reads as a minimum threshold ("write or above"), not exact set-membership. This semantic is undocumented.

---

Documentation Gaps

Standing items (re-confirmed, low priority)

• S17 — IgnoredFrontmatterFields = ["user-invokable"] (pkg/constants/constants.go:315) has zero mentions across docs/. It is an intentionally-ignored GitHub Copilot custom-agent field, but its silent acceptance is undocumented.
• S19#1 — dispatch_repository / allowed_repositories use snake_case among otherwise kebab-case siblings (e.g. dispatch-workflow). Docs use snake_case consistently, so this is intentional; flagged only for naming-convention awareness.

Schema Improvements Needed

• If the API-contract finding is confirmed, either (a) remove maintain/triage from the on.roles enum, or (b) keep them and fix the runtime check (see below). Leaving schema-valid-but-unreachable values is the worst outcome.

Parser / Runtime Updates Required

• actions/setup/js/check_permissions_utils.cjs:241-245 — to honor maintain/triage, read repoPermission.data.role_name (fall back to permission) and/or implement a proper permission hierarchy (admin > maintain > write > triage > read) so that requiring a lower role also admits higher ones.
• actions/setup/js/check_permissions.test.cjs:92,99 — replace mocked permission: "maintain"/"triage" with fixtures that match the real API shape (e.g. permission: "write", role_name: "maintain"), so tests reflect production behavior.

Workflow Violations

None found. A sweep of .github/workflows/*.md top-level keys against the schema surfaced only nested-key/expression false-positives from the pre-computed in_used_not_schema list (e.g. judge_path, run_dir, serena-* tool names) — no real schema violations.

---

Recommendations

1. Verify the GitHub API contract for getCollaboratorPermissionLevel: confirm whether permission can ever be maintain/triage. This single check determines whether the critical finding is a real bug or a non-issue.
2. If confirmed, fix the runtime check to consult role_name and/or implement a permission hierarchy, and align the test fixtures with the real API shape.
3. Document the roles matching semantics (exact set-membership vs. threshold) in triggers.md so authors understand that roles: [write] does not implicitly include admin.
4. (Standing) Decide whether to document or remove the decorative experiments.*.analysis_type enum (S20) and the silently-ignored user-invokable field (S17).

Strategy Performance

• Strategy Used: S23 — Enum value reachability at runtime (NEW)
• Findings: 2 (1 high + 1 secondary)
• Effectiveness: HIGH — found a class of issue that schema-validation and branch-coverage strategies structurally cannot catch (values that pass schema and mocked tests but can't match live data).
• Should Reuse: YES — generalize to any enum compared against an external-API response field.
• Companion S22 (enum-vs-docs drift): LOW effectiveness this run (0 findings; docs well maintained).

Next Steps

• Confirm getCollaboratorPermissionLevel permission field values against live GitHub API
• Fix role matching to use role_name and/or a permission hierarchy
• Update permission test fixtures to match real API shape
• Document on.roles matching semantics in triggers.md
• (Standing) Resolve S17 (user-invokable docs) and S20 (analysis_type decorative enum)

References:

• §27121108972

Generated by ✅ Schema Consistency Checker · 233.9 AIC · ⌖ 15.3 AIC · ⊞ 9.7K · ◷

• expires on Jun 8, 2026, 11:06 PM UTC-08:00

[github-actions[bot]]

This discussion has been marked as outdated by Schema Consistency Checker.

A newer discussion is available at Discussion #38059.