Skip to content

Extraction pipeline

Arun Soman edited this page Aug 31, 2026 · 1 revision

Two-pass code-graph extraction, plus the TESTS pass

Two-pass code-graph extraction, plus the TESTS pass — migrated from the README on 2026-08-31 and verified against the code at tag v0.1.4; if code and wiki ever disagree, the code wins and this page gets a PR.

Two-pass code-graph extraction & loading (extract.py, callgraph.py, testlink.py)

  • Pass 1 (extract.py): tree-sitter parse of every supported file into file/class/function/method Nodes with signature, line_start/ line_end, docstring, plus the raw inputs for pass 2 — imports and call_sites. Pure: no DB/FS side effects.
  • Pass 2 (callgraph.py): resolves call sites into confidence-tagged calls edges — EXTRACTED (same-file def or import-map resolved), INFERRED (receiver-type heuristic), or AMBIGUOUS (exactly one same-named symbol project-wide). Also resolves inheritance/extends edges and synthesizes external:: stub nodes for unresolved base classes.
  • testlink.py: a third pass that emits TESTS edges from test symbols to the implementation symbols they test, via three heuristics — naming convention (test_foofoo), confidence upgrade when a naming match is backed by a real calls edge, and @patch(...)/@mock.patch(...) decorator resolution.
  • Loaders: cie load <dirs> --project <name> (Neo4j, full replace of one project's nodes) and cie index <path> (embedded SQLite, zero-config). reindex / reindex_file for incremental single-file refresh after a patch; watch for file-system-driven auto-reindex (watchdog).

TESTS heuristics (current, v0.1.4)

  1. Naming convention (test_foofoo; test-class shapes) — INFERRED.
  2. Upgrade: naming match backed by a resolved calls edge → EXTRACTED.
  3. @patch/@mock.patch dotted paths — EXTRACTED.
  4. Direct calls (added 2026-08-31): a resolved calls edge from a test symbol to a FUNC/METHOD/CLASS in production code mints the edge, EXTRACTED — behavioral test names and monkeypatch had starved heuristics 1–3 (dogfooding measured 1 TESTS edge in cie's own 308-test suite; after: 562). Test-glob files and conftest.py are never targets. Still not exhaustive: a test reaching its target only via a helper gets no edge — test_map's hint says so.

Clone this wiki locally