Skip to content

Module docstring never extracted when a comment precedes it (shebang-led files lose all module rationale) #3312

Description

@vongohren

Summary

_extract_python_rationale never extracts a module docstring when any comment precedes it — including the #! shebang that begins essentially every executable Python script. Function and class docstrings in the same file are extracted normally, so the loss is silent and specific to module-level rationale.

Reproducer

Three files in an empty directory:

# with_shebang.py
#!/usr/bin/env python3
"""MODULE docstring of the shebang-led file, long enough to pass the length filter."""


def f():
    """FUNCTION docstring of the shebang-led file, long enough to pass the filter."""
    return 1
# without_shebang.py
"""MODULE docstring of the plain file, long enough to pass the length filter."""


def g():
    """FUNCTION docstring of the plain file, long enough to pass the filter."""
    return 2
# with_comment.py
# an ordinary leading comment, not a shebang
"""MODULE docstring behind a plain comment, long enough to pass the filter."""


def h():
    return 3
$ GRAPHIFY_OUT=./out graphify update . --no-cluster

Observed (graphify 0.9.53)

file_type: rationale nodes produced:

file module docstring function docstring
without_shebang.py ✅ L1 ✅ L5
with_shebang.py missing ✅ L6
with_comment.py missing

Expected

All three module docstrings extracted as rationale nodes at their own line.

Cause

graphify/extract.py, _extract_python_rationale._get_docstring:

def _get_docstring(body_node) -> tuple[str, int] | None:
    if not body_node:
        return None
    for child in body_node.children:
        if child.type == "expression_statement":
            ...
        break          # <-- unconditional, after the FIRST child
    return None

The break is unconditional, so only the first child of the node is ever considered. At module level the first child of a shebang-led (or comment-led) file is a comment, not the expression_statement holding the docstring, so the function returns None. Function and class bodies are unaffected because their first child is the docstring statement.

A continue past comment children (or skipping non-expression_statement children until the first statement) would fix it. Preserving the "only look at the leading statement" intent matters — you don't want to pick up a string literal from halfway down the module.

Why it's worth fixing

Module docstrings are disproportionately the highest-value rationale in a codebase: they describe why the file exists, and executable entry points — the files people most often ask "what does this do" about — are precisely the ones that carry a shebang and therefore lose it. On a ~25k-node graph across 14 repositories we found 158 files with a module-level (L1) rationale node and not one of them was shebang-led; every shebang-led file with a module docstring was missing it.

Happy to send a PR if the continue shape is the direction you'd want.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions