Skip to content

Add orchestration governance drift check - #18

Merged
maniczko merged 1 commit into
mainfrom
codex/issue-5-add-rssmaster-orchestration-governance-drift-test
May 25, 2026
Merged

Add orchestration governance drift check#18
maniczko merged 1 commit into
mainfrom
codex/issue-5-add-rssmaster-orchestration-governance-drift-test

Conversation

@maniczko

Copy link
Copy Markdown
Owner

Summary

  • Add scripts/check_orchestration_governance.py to detect GitHub Issue orchestration drift.
  • Cover required files, required/blocking labels, gate commands, AGENTS orchestration section, and issue template snippets.
  • Include self-tests proving the check fails when agent_task.yml is missing or agent:ready is removed.
  • Wire the check into the quick gate via npm run test:unit.

Linked Issue

Closes #5

Evidence

  • python scripts/check_orchestration_governance.py passed.
  • python C:\Users\user\.codex\skills\github-issue-orchestrator\scripts\codex_orchestrate.py doctor --repo-root . passed.
  • npm run test:unit passed after npm ci in the clean worktree.
  • Required gate labels satisfied: area:governance, gate:quick.

Agent Checklist

  • One issue, one branch, one PR.
  • No direct commit to main.
  • No secrets, tokens, raw emails, or private documents in logs/comments.
  • Residual risks stated.

Risks / Rollback

  • npm ci reported existing dependency audit findings: 2 moderate and 2 high. This PR does not change dependencies.
  • The check validates local orchestration policy/config drift; it does not call GitHub to verify live remote labels.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a governance check script, check_orchestration_governance.py, and integrates it into the unit test suite via package.json. The script validates the presence of required repository files and the structure of the orchestration configuration. Feedback from the review highlights two potential issues: the validation logic could be bypassed if the configuration file is an empty dictionary, and the script lacks a type check for the JSON root, which could lead to crashes if the configuration is not an object.


config, config_errors = load_config(root)
failures.extend(config_errors)
if config:

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

The check if config: will evaluate to False if the configuration file is an empty dictionary ({}). This causes the script to skip all validation logic for labels and gates, effectively allowing an empty or minimal configuration to pass the governance check. Changing this to check for the absence of config_errors ensures that the content is validated even if the dictionary is currently empty.

Suggested change
if config:
if not config_errors:

Comment on lines +39 to +43
return json.loads(path.read_text(encoding="utf-8-sig")), []
except FileNotFoundError:
return {}, [f"missing_file:{REQUIRED_FILES['orchestration_config']}"]
except json.JSONDecodeError as error:
return {}, [f"invalid_json:{path}:{error}"]

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The load_config function does not verify that the parsed JSON root is a dictionary. If orchestration.json contains a JSON list or a scalar value at the root, subsequent calls to .get() in list_value and dict_value will raise an AttributeError, causing the script to crash. It is safer to validate the type immediately after loading.

Suggested change
return json.loads(path.read_text(encoding="utf-8-sig")), []
except FileNotFoundError:
return {}, [f"missing_file:{REQUIRED_FILES['orchestration_config']}"]
except json.JSONDecodeError as error:
return {}, [f"invalid_json:{path}:{error}"]
data = json.loads(path.read_text(encoding="utf-8-sig"))
if not isinstance(data, dict):
return {}, [f"invalid_json_root:{path}:expected_object"]
return data, []
except FileNotFoundError:
return {}, [f"missing_file:{REQUIRED_FILES['orchestration_config']}"]
except json.JSONDecodeError as error:
return {}, [f"invalid_json:{path}:{error}"]

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 459f41b9e5

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +100 to +101
for snippet in ("agent:ready", "Cel / Goal", "Kryteria akceptacji / Acceptance Criteria"):
if snippet not in issue_text:

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Validate issue labels structurally, not by text snippet

The governance check only asserts that the string agent:ready appears anywhere in .github/ISSUE_TEMPLATE/agent_task.yml, so it will still pass if the actual labels: list drops agent:ready but the term remains in descriptive text. In that scenario new issues stop being pre-labeled correctly while this drift check reports success, undermining the purpose of the guard. Parse the YAML and verify agent:ready is present under the top-level labels field (or with a stricter pattern tied to that field) to avoid this false negative.

Useful? React with 👍 / 👎.

@maniczko
maniczko merged commit c2b0537 into main May 25, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add RSSmaster orchestration governance drift test

1 participant