Summary
_extract_python_rationale never extracts a module docstring when any comment precedes it — including the #! shebang that begins essentially every executable Python script. Function and class docstrings in the same file are extracted normally, so the loss is silent and specific to module-level rationale.
Reproducer
Three files in an empty directory:
# with_shebang.py
#!/usr/bin/env python3
"""MODULE docstring of the shebang-led file, long enough to pass the length filter."""
def f():
"""FUNCTION docstring of the shebang-led file, long enough to pass the filter."""
return 1
# without_shebang.py
"""MODULE docstring of the plain file, long enough to pass the length filter."""
def g():
"""FUNCTION docstring of the plain file, long enough to pass the filter."""
return 2
# with_comment.py
# an ordinary leading comment, not a shebang
"""MODULE docstring behind a plain comment, long enough to pass the filter."""
def h():
return 3
$ GRAPHIFY_OUT=./out graphify update . --no-cluster
Observed (graphify 0.9.53)
file_type: rationale nodes produced:
| file |
module docstring |
function docstring |
without_shebang.py |
✅ L1 |
✅ L5 |
with_shebang.py |
❌ missing |
✅ L6 |
with_comment.py |
❌ missing |
— |
Expected
All three module docstrings extracted as rationale nodes at their own line.
Cause
graphify/extract.py, _extract_python_rationale._get_docstring:
def _get_docstring(body_node) -> tuple[str, int] | None:
if not body_node:
return None
for child in body_node.children:
if child.type == "expression_statement":
...
break # <-- unconditional, after the FIRST child
return None
The break is unconditional, so only the first child of the node is ever considered. At module level the first child of a shebang-led (or comment-led) file is a comment, not the expression_statement holding the docstring, so the function returns None. Function and class bodies are unaffected because their first child is the docstring statement.
A continue past comment children (or skipping non-expression_statement children until the first statement) would fix it. Preserving the "only look at the leading statement" intent matters — you don't want to pick up a string literal from halfway down the module.
Why it's worth fixing
Module docstrings are disproportionately the highest-value rationale in a codebase: they describe why the file exists, and executable entry points — the files people most often ask "what does this do" about — are precisely the ones that carry a shebang and therefore lose it. On a ~25k-node graph across 14 repositories we found 158 files with a module-level (L1) rationale node and not one of them was shebang-led; every shebang-led file with a module docstring was missing it.
Happy to send a PR if the continue shape is the direction you'd want.
Summary
_extract_python_rationalenever extracts a module docstring when any comment precedes it — including the#!shebang that begins essentially every executable Python script. Function and class docstrings in the same file are extracted normally, so the loss is silent and specific to module-level rationale.Reproducer
Three files in an empty directory:
Observed (graphify 0.9.53)
file_type: rationalenodes produced:without_shebang.pywith_shebang.pywith_comment.pyExpected
All three module docstrings extracted as
rationalenodes at their own line.Cause
graphify/extract.py,_extract_python_rationale._get_docstring:The
breakis unconditional, so only the first child of the node is ever considered. At module level the first child of a shebang-led (or comment-led) file is acomment, not theexpression_statementholding the docstring, so the function returnsNone. Function and class bodies are unaffected because their first child is the docstring statement.A
continuepastcommentchildren (or skipping non-expression_statementchildren until the first statement) would fix it. Preserving the "only look at the leading statement" intent matters — you don't want to pick up a string literal from halfway down the module.Why it's worth fixing
Module docstrings are disproportionately the highest-value rationale in a codebase: they describe why the file exists, and executable entry points — the files people most often ask "what does this do" about — are precisely the ones that carry a shebang and therefore lose it. On a ~25k-node graph across 14 repositories we found 158 files with a module-level (
L1) rationale node and not one of them was shebang-led; every shebang-led file with a module docstring was missing it.Happy to send a PR if the
continueshape is the direction you'd want.