Problem
CodeLens silently skips files above hardcoded size thresholds to avoid a tree-sitter segfault (tracked in #116):
- JavaScript: files > ~100 lines → skipped
- Python: files > ~200 lines → skipped
This is the most critical correctness gap in CodeLens today. The files most worth analyzing are exactly the ones getting skipped.
Tested on Wolfvin/Regrets:
scripts/regret.js: 2,731 lines — cyclomatic complexity 338, cognitive 936 — the most important file in the repo → skippedscripts/validate.py: main function cyclomatic 300 — skipped- ~40% of the codebase invisible to
complexity, entrypoints, dead-code, smell
Result: complexity reports 1,865 functions but the real count is higher. The "untamable" hotspots it finds are from smaller files — the actual worst offenders are invisible.
Root Cause
See #116 — Python GC can collect a tree-sitter Tree while a Node is still alive, causing SIGSEGV. The current fix was to skip large files entirely, which trades correctness for stability.
Goal
Fix the underlying segfault so large files can be analyzed safely, then remove or substantially raise the threshold.
The known fix pattern (from base_parser.py): store a _last_tree reference on the parser instance to prevent GC collection. If this is already in place for small files, it should work for large files too — verify why it is not applied uniformly.
Proposed Fix
- Audit
base_parser.py and all parser subclasses — confirm _last_tree is set before returning any Node objects
- Stress-test with files 500–5000 lines: parse → walk nodes → no SIGSEGV
- If fix holds: remove the file size threshold entirely, or raise to a practical limit (e.g. 10,000 lines) with a warning, not a silent skip
- If segfault persists on very large files: replace silent skip with explicit warning in output:
{ "skipped": [{ "file": "scripts/regret.js", "reason": "file_too_large", "lines": 2731 }] }
Definition of Done
Why This Is P1
Without this fix, CodeLens gives false confidence. A health check that misses 40% of the codebase — specifically the most complex 40% — is worse than no health check, because it implies coverage it doesn't have.
Problem
CodeLens silently skips files above hardcoded size thresholds to avoid a tree-sitter segfault (tracked in #116):
This is the most critical correctness gap in CodeLens today. The files most worth analyzing are exactly the ones getting skipped.
Tested on Wolfvin/Regrets:
scripts/regret.js: 2,731 lines — cyclomatic complexity 338, cognitive 936 — the most important file in the repo → skippedscripts/validate.py: main function cyclomatic 300 — skippedcomplexity,entrypoints,dead-code,smellResult:
complexityreports 1,865 functions but the real count is higher. The "untamable" hotspots it finds are from smaller files — the actual worst offenders are invisible.Root Cause
See #116 — Python GC can collect a tree-sitter
Treewhile aNodeis still alive, causing SIGSEGV. The current fix was to skip large files entirely, which trades correctness for stability.Goal
Fix the underlying segfault so large files can be analyzed safely, then remove or substantially raise the threshold.
The known fix pattern (from
base_parser.py): store a_last_treereference on the parser instance to prevent GC collection. If this is already in place for small files, it should work for large files too — verify why it is not applied uniformly.Proposed Fix
base_parser.pyand all parser subclasses — confirm_last_treeis set before returning anyNodeobjects{ "skipped": [{ "file": "scripts/regret.js", "reason": "file_too_large", "lines": 2731 }] }Definition of Done
_last_treefix applied uniformly across all parser classesscripts/regret.jsfrom Wolfvin/Regrets as test fixture — 2,731 lines, JS)skipped[]list in JSON outputcomplexityon Wolfvin/Regrets now detectsscripts/regret.js:main(cyclomatic ~338) as a hotspotWhy This Is P1
Without this fix, CodeLens gives false confidence. A health check that misses 40% of the codebase — specifically the most complex 40% — is worse than no health check, because it implies coverage it doesn't have.