Problem
charter score on aegis-daemon reports 177 broken path references. The top entries reveal the path resolver is not filtering non-file strings:
AEGIS_LOCAL_URL=http:/localhost:11434 — env var assignment pulled from a code block
C:/Users/kover/Documents/aegis-daemon — Windows absolute path in a markdown table
/chat, /api/runs, /webhooks/voice/complete — HTTP route paths in a table
None of these are file paths, but the grounding checker treats any string that looks like a path as a reference to validate.
False positive categories
- URL strings — anything containing
:// or starting with http should be excluded
- Env var assignments —
KEY=value lines in code blocks should not have the value extracted as a path
- HTTP routes — strings starting with
/api/, /webhooks/, /chat etc. in markdown tables are URL paths, not filesystem paths
- Windows absolute paths —
C:/... paths in documentation tables are reference documentation, not resolvable local paths
- Env var names — bare
ANTHROPIC_API_KEY tokens are not file paths
Impact
These false positives inflate the broken reference count from a likely real count of ~5–10 to 177, dragging the Grounding score from what should be ~18/20 to 7/20. This makes the score misleading and undermines trust in the metric.
Fix
Before running path resolution:
- Strip any token that matches
https?:// (URL)
- Strip any token that matches
^[A-Z_]+= (env var assignment)
- Strip any token that starts with
/api/, /webhooks/, /health, /chat (HTTP route heuristic, or: only validate paths that resolve to a plausible relative file under the repo root)
- Strip Windows absolute paths (
[A-Z]:/)
- Only validate paths containing a file extension (
.ts, .md, .json, etc.) OR paths that start with ./ or ../ (explicit relative references)
Alternatively: require file references to appear in a specific ADF notation (e.g., ref:path/to/file) rather than doing heuristic path extraction from all text.
Problem
charter scoreon aegis-daemon reports 177 broken path references. The top entries reveal the path resolver is not filtering non-file strings:AEGIS_LOCAL_URL=http:/localhost:11434— env var assignment pulled from a code blockC:/Users/kover/Documents/aegis-daemon— Windows absolute path in a markdown table/chat,/api/runs,/webhooks/voice/complete— HTTP route paths in a tableNone of these are file paths, but the grounding checker treats any string that looks like a path as a reference to validate.
False positive categories
://or starting withhttpshould be excludedKEY=valuelines in code blocks should not have thevalueextracted as a path/api/,/webhooks/,/chatetc. in markdown tables are URL paths, not filesystem pathsC:/...paths in documentation tables are reference documentation, not resolvable local pathsANTHROPIC_API_KEYtokens are not file pathsImpact
These false positives inflate the broken reference count from a likely real count of ~5–10 to 177, dragging the Grounding score from what should be ~18/20 to 7/20. This makes the score misleading and undermines trust in the metric.
Fix
Before running path resolution:
https?://(URL)^[A-Z_]+=(env var assignment)/api/,/webhooks/,/health,/chat(HTTP route heuristic, or: only validate paths that resolve to a plausible relative file under the repo root)[A-Z]:/).ts,.md,.json, etc.) OR paths that start with./or../(explicit relative references)Alternatively: require file references to appear in a specific ADF notation (e.g.,
ref:path/to/file) rather than doing heuristic path extraction from all text.