AI agent context quality analyzer. Audit instruction files (AGENTS.md, PROMPT.md, .cursorrules, specs/*) for effectiveness, not just security.
Complementary to agentlint (security) — contextaudit checks whether your instructions will produce good agent output.
The quality of an AI agent's work depends on the quality of its instructions. Bad instructions → bad results, regardless of model capability. This is especially critical for autonomous loops (Ralph Wiggum pattern, HITL workflows) where the agent runs unattended.
contextaudit checks for the patterns that make agent instructions effective:
- Completion criteria — Does the agent know when it's done?
- Backpressure — Are there tests/lints to validate work?
- Scope boundaries — Is the agent's workspace defined?
- Task granularity — Are instructions broken into clear steps?
- Self-correction — Is the agent told to check its own work?
# Just download it
curl -O https://raw.githubusercontent.com/kriskimmerle/contextaudit/main/contextaudit.py
chmod +x contextaudit.py
# Or clone and run
git clone https://github.com/kriskimmerle/contextaudit.git
cd contextaudit
python3 contextaudit.pyNo dependencies. Python 3.9+ required.
# Audit all instruction files in current project
python3 contextaudit.py
# Audit a specific project
python3 contextaudit.py /path/to/project
# Audit a specific file
python3 contextaudit.py -f AGENTS.md
# Verbose output with suggestions
python3 contextaudit.py --verbose
# JSON output
python3 contextaudit.py --json
# CI mode: exit 1 if grade below B
python3 contextaudit.py --ci --threshold B
# Quiet: just the grade
python3 contextaudit.py -q contextaudit v1.0.0
/path/to/project
Files analyzed: 3
• AGENTS.md (system)
• PROMPT.md (prompt)
• specs/auth.md (spec)
Grade: A (94/100)
Issues (3):
🟡 [C09] [PROMPT.md]: Ambiguous language found: make it better
💡 Replace with specific, measurable instructions
🔵 [C07] [AGENTS.md]: No commit/save instructions
💡 Add: 'Commit after each completed task with a descriptive message'
🔵 [C05] [specs/auth.md]: No error handling guidance
💡 Add: 'If tests fail, debug and fix. If stuck, stop and explain.'
Strengths (7):
🟢 Has 1 spec file(s)
🟢 [AGENTS.md] Has backpressure mechanism (tests/lint/build)
🟢 [AGENTS.md] Well-structured task breakdown (12 steps/items)
🟢 [AGENTS.md] Has self-correction/verification instructions
🟢 [AGENTS.md] Specifies concrete tools/commands
🟢 [PROMPT.md] Has task priority/ordering guidance
🟢 [PROMPT.md] Follows single-task-per-iteration pattern
Summary: 1 warning, 2 info
| Rule | Severity | Description |
|---|---|---|
| C01 | Critical/Warning | Missing completion criteria |
| C02 | Critical | No backpressure mechanism (tests/lint/build) |
| C03 | Warning | No scope boundaries defined |
| C04 | Warning | No structured task breakdown |
| C05 | Info | No error handling guidance |
| C06 | Warning | Context size issues (too large or too small) |
| C07 | Info | No commit/save instructions |
| C08 | Info | No self-correction instructions |
| C09 | Warning | Ambiguous/vague language detected |
| C10 | Info | No specific tools/commands mentioned |
| C11 | Bonus | Includes code examples |
| C12 | Bonus | Has task priority/ordering |
| C13 | Bonus | Single-task-per-iteration pattern |
| C14 | Bonus | Subagent/delegation instructions |
| C15 | Info | Repeated/duplicate instructions |
| Rule | Severity | Description |
|---|---|---|
| P01 | Critical | No agent instruction files found |
| P02 | Warning | No system-level instructions (AGENTS.md) |
| P03 | Bonus | Has spec files |
| P04 | Bonus | Has implementation plan |
contextaudit automatically scans for:
| File | Type | Description |
|---|---|---|
| AGENTS.md | System | Project-wide agent instructions |
| CLAUDE.md | System | Claude Code instructions |
| COPILOT.md | System | GitHub Copilot instructions |
| .cursorrules | System | Cursor editor rules |
| .windsurfrules | System | Windsurf editor rules |
| PROMPT.md | Prompt | Task-level prompt (Ralph pattern) |
| PROMPT_plan.md | Prompt | Planning mode prompt |
| PROMPT_build.md | Prompt | Building mode prompt |
| IMPLEMENTATION_PLAN.md | Plan | Task plan |
| specs/*.md | Spec | Requirement specifications |
| Grade | Score | Meaning |
|---|---|---|
| A | 90-100 | Excellent — agent will have clear, effective instructions |
| B | 80-89 | Good — minor improvements possible |
| C | 65-79 | Adequate — notable gaps in instruction quality |
| D | 50-64 | Below average — agent likely to produce poor results |
| F | 0-49 | Poor — critical instructions missing |
# GitHub Actions
- name: Check agent context quality
run: python3 contextaudit.py --ci --threshold B| agentlint | contextaudit | |
|---|---|---|
| Focus | Security risks | Effectiveness/quality |
| Checks | Injection, exfiltration, destruction | Completion criteria, backpressure, scope |
| Question | "Is this safe?" | "Will this work well?" |
| Use together | Yes — run both in CI | Yes — complementary checks |
# Audit this project's own instructions
python3 contextaudit.py .
# Audit any project with AGENTS.md
python3 contextaudit.py ~/my-projectMIT