Python analyzer silently swallows multilspy errors → empty CALLS edges on large codebases

[DvirDukhan]

Summary

The Python analyzer's resolve() (in api/analyzers/analyzer.py:55-60) silently swallows every exception from multilspy's request_definition. On larger Python codebases this causes the second-pass to produce zero CALLS / EXTENDS edges while still emitting nodes — so the graph looks "indexed" (thousands of File and Function nodes) but is unusable for any neighbor-based navigation.

How it surfaced

While running the SWE-bench benchmark harness (bench/runners/mini_runner.py) we got wildly inconsistent code_graph results across instances. Direct queries against FalkorDB after indexing:

Repo	Nodes	DEFINES	CALLS	EXTENDS
pytest-dev/pytest-6202	4,711	4,508	1,960	71
sphinx-doc/sphinx-9230	7,607	7,064	0	0
sympy/sympy-20154	42,964	41,525	0	0

Pytest (~200 files, 4.5k functions) indexed fine. Sphinx (~540 files, 6k functions) and sympy (~1.4k files, ~40k functions) got zero call/extends edges. All three repos have no venv directory at the root, so the env path passed to multilspy ({path}/venv) doesn't exist for any of them — yet pytest still resolved.

Root cause

# api/analyzers/analyzer.py:55-60
def resolve(self, files, lsp, file_path, path, node):
    try:
        locations = lsp.request_definition(str(file_path), node.start_point.row, node.start_point.column)
        return [...]
    except Exception as e:
        return []  # ← every error is silently dropped, no log

When multilspy/jedi crashes, hangs, or returns an unexpected payload, the entire per-call resolution fails silently. On larger codebases (sphinx, sympy) this seems to happen for ~100% of calls, producing the empty-CALLS pattern above.

Benchmark impact

The agent's cg get-neighbors returns {"edges": [], "nodes": []} for every symbol → the agent falls back to grep/sed but still pays the cost of the graph queries up front. On sympy-20154 this made code_graph +78.7% vs baseline (vs −21.4% on pytest where the graph was complete).

Suggested fix

1. Log the exception (at WARN level with file/symbol context) instead of returning [] silently. This alone would surface the bug in CI/local indexing logs.
2. Surface a summary at end of second_pass: count of resolved vs unresolved symbols. If <5% resolve, raise — the resulting graph is misleading.
3. Investigate the underlying multilspy failure on bigger codebases. Likely candidates: per-request timeout too short, jedi env detection failing without a venv, or LSP startup race when there are 40k+ files to index.

Repro

export FALKORDB_HOST=127.0.0.1 FALKORDB_PORT=6379
export ALLOWED_ANALYSIS_DIR=/tmp
git clone --depth 1 --branch 1.8 https://github.com/sphinx-doc/sphinx /tmp/sphinx
uv run uvicorn api.index:app --host 127.0.0.1 --port 5000 &
curl -X POST http://127.0.0.1:5000/api/analyze_folder \
  -H 'Content-Type: application/json' \
  -d '{"path":"/tmp/sphinx","ignore":[]}'
# Then query FalkorDB:
redis-cli -p 6379 GRAPH.QUERY sphinx "MATCH ()-[r]->() RETURN type(r), count(r)"
# Expect: only DEFINES, no CALLS

Related

• Surfaced by: bench-mcp-track work (bench: MCP-transport sibling of the code-graph track #684) and the first real SWE-bench 3-way run.
• Affects: any user indexing a Python codebase larger than ~5k functions.

[DvirDukhan]

Fixed by #686 (root cause: hardcoded environment_path={path}/venv killed jedi when no venv existed; analyzer.resolve() silently swallowed every failure).

Verified post-fix:

• sphinx-9230: 0 → 4,931 CALLS edges
• sympy-20154: 0 → 22,803 CALLS edges
• Benchmark impact: code_graph went from +78.7% regression on sympy to −19.3% savings; overall −36.2% tokens vs baseline (was −11.0%).