feat: Python role selector — works for both Claude and Codex#114
Merged
Conversation
The unified prompt relied on the agent following structured output instructions to pick its role. Codex ignores these 100% of the time (900 sed reads vs 0 cat reads across 23 sessions). The role decision now happens in Python before the agent starts. scripts/pick-role.py: - Reads signal files (eval scores, task counts, session history, etc.) - Computes deterministic scores for 5 roles - Prints winning role to stdout, reasoning to stderr - Works identically for Claude and Codex - 38 unit tests covering 10 stress test scenarios daemon.sh: - pick_session_role() calls pick-role.py, loads the winning role prompt - build_prompt() cats evolve-auto.md + role prompt (not unified.md) - Removed old Python role extraction from logs (always defaulted to "build") - Removed FORCE_ROLE prompt injection (pick-role.py handles env var) - Pentest report wrapped in <pentest_data> XML tags (injection resistance) Also: - unified.md moved to docs/ops/ROLE-SCORING.md (reference doc, not active prompt) - review.md Step 4 fixed: make check replaces individual tool runs - scripts/ added to PROMPT_GUARD_FILES and PROMPT_GUARD_DIRS (trojan detection) - Session index header fixed to 9 columns - Existing pentest test updated for new XML framing
fazxes
added a commit
that referenced
this pull request
Apr 7, 2026
Closed 20 tasks with evidence: - DONE (2): #73 (AGENTS.md created), #181 (docs/prompt/ deleted) - WONTFIX-OBSOLETE (5): #78, #89, #128, #141, #157 (reference docs/prompt/ or docs/ops/ paths deleted in session #103) - WONTFIX-DUPLICATE (1): #88 (subset of #69) - WONTFIX-NEVER-PICKED (12): #66, #69, #90, #96, #112, #114, #120, #123, #132, #133, #138, #145 (low priority, 20-80+ sessions without being picked, speculative) Priority fix: #103 downgraded from urgent to normal (umbrella epic, not an actionable urgent fix).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Replaces the LLM-based unified prompt with a deterministic Python scoring engine.
Why
The unified prompt (unified.md) relied on the agent following structured output to pick its role. Audit found Codex ignores these instructions 100% of the time (900 sed reads vs 0 cat reads). The role decision must happen in Python, not in the LLM.
What changed
scripts/pick-role.py— 372-line scoring engine, reads signal files, computes scores, prints winnertests/test_pick_role.py— 38 unit tests covering 10 stress test scenariosdaemon.sh— calls pick-role.py at cycle start, loads winning role promptunified.md→docs/ops/ROLE-SCORING.md(reference doc, no longer active prompt)make checkHow it works
Both Claude and Codex get the same treatment. No structured output required.
Test plan
make checkpasses (981 tests)