Detector: Inline Comment Drift

Detector: Inline Comment Drift

Detects docstrings that no longer accurately describe their function or class implementation.

Property	Value
Name	inline-comment-drift
Tier	LLM_ASSISTED
Languages	Python
External tool	None
LLM required	Yes — minimum basic capability tier
Confidence	0.55 (basic), 0.70 (enhanced)

What it detects

Stale docstrings where the documented behavior no longer matches the code:

• Wrong parameter names or types
• Incorrect return type/value descriptions
• Described behavior that doesn't exist in the implementation
• Wrong examples in docstrings

This detector focuses on factual inaccuracies, not style or completeness.

How it works

1. Discovers Python files, prioritized by risk signals (churn + fix ratio)
2. Parses AST to extract (docstring, code body) pairs for functions and classes
3. Sends each pair to the LLM asking if the docstring is factually wrong
4. Parses structured JSON response (needs_review, reason, optional severity)
5. Logs every LLM call to the database

Limits

Parameter	Value
Min docstring length	30 characters
Max per file	20 docstrings
Max per scan	100 docstrings
Truncation (basic)	600 / 1500 chars (doc / code)
Truncation (enhanced)	1200 / 3000 chars (doc / code)

Capability tiers

Tier	Behavior
basic	Binary "factually wrong or not" judgment, shorter context, confidence 0.55
standard+	5-point checklist (params, return type, side effects, examples, etc.), larger context, confidence 0.70, LLM can report severity

Example finding

[INLINE-COMMENT-DRIFT] src/sentinel/core/runner.py:run_scan (line 45)
  Docstring says it returns a list of findings, but the function
  returns a ScanResult dataclass.
  Severity: MEDIUM, Confidence: 0.55

Configuration

[sentinel]
model = "qwen3.5:4b"     # Works for clear factual errors
# model = "gpt-5.4-nano" # Better for subtle semantic drift