feat(r): extract R with tree-sitter and resolve calls corpus-wide - #2393
feat(r): extract R with tree-sitter and resolve calls corpus-wide#2393fernando-duarte wants to merge 1 commit into
Conversation
R was in CODE_EXTENSIONS but had no entry in _DISPATCH, so every .R file was counted as code and then contributed nothing — it is the language Graphify-Labs#1689's no-AST-extractor warning names as its example. R has no named-function syntax: `f <- function(x)`, `f = \(x)` and `(function(x) x) -> f` are all assignments binding an anonymous function_definition, whose only `name` field is the `function` keyword itself. The config-driven walker reads a name off the function node, so this is a bespoke extractor. Calls are resolved corpus-wide rather than per file. Within a package or an analysis directory R has one shared namespace and no import statement binding a name to a file, so `paste0(x)` and a sibling file's `compute_moments(x)` are the same syntax and the file cannot tell them apart. The extractor emits unresolved calls as raw_calls and graphify.r_resolution links a callee only when the corpus defines it exactly once — the god-node guard the Java and ObjC resolvers use. Everything else is dropped, so base R stays out of the graph without hardcoding ~1,300 base names and no dangling edge reaches build_from_json. Sourced files resolve the same way: any R path literal, including one joined from string literals by a helper (`source_once(paper_path("sub", "b.R"))`), links only when it matches exactly one corpus file. The r-lib grammar has no standalone PyPI wheel, so it comes from tree-sitter-language-pack under a new optional [r] extra, hard-failing like tree-sitter-sql rather than falling back. Two tests in test_extract.py used .r as their example of a code extension with no extractor; they move to .ets, which is now one of the two that remain. Measured on a 541-file R corpus: 2,171 nodes and 7,434 edges in 2.2s, zero dangling edges, and no base-R hub in the top-degree nodes.
![graphify-labs[bot]](https://avatars.githubusercontent.com/in/4356270?s=60&v=4){kind=small}
There was a problem hiding this comment.
Looks safe to merge — no coupling regressions and no blocking issues, checked against the code graph (not a self-assessment).
Graphify reviewed this change.
Looks safe to merge — no coupling regressions and no blocking issues, checked against the code graph (not a self-assessment).
Graphify review — findings
This PR adds R language extraction support to graphify. It introduces a new bespoke extractor (graphify/extractors/r.py) that handles R's assignment-based function definitions (<-/=/<<-/right-assign), lambdas, imports (library/require/pkg::fn), and source() calls, plus a companion r_resolution module that resolves calls corpus-wide against a shared namespace rather than per-file. It wires the extractor into the dispatch tables (.r/.R extensions and the Rscript shebang), registers the R resolver, adds an [r] optional extra, and updates the existing no-AST-extractor warning comments since R is no longer the example of an unsupported-but-counted language. The changeset also touches the changelog and a broad set of test files covering extraction, R resolution, and related behavior. I'm only describing intent and surface area; I haven't assessed whether the extraction/resolution logic is correct.
No blocking issues surfaced. 3 lower-confidence candidates did not survive cross-model review.
Analysis details — impact, health, verification
Impact & health
Graphify review
Impact — 1632 functions depend on the 694 functions this change touches.
Health — this change adds coupling hotspots:
- worse:
extract()— 365 callers, 29 callees - worse:
_get_extractor()— 26 callers, 6 callees - new:
extract_r()— 10 callers, 3 callees - new:
walk_calls()— 0 callers, 8 callees - new:
collect_definitions()— 0 callers, 6 callees
Verification — 1632 functions in the blast radius were not formally verified this run (proofs are advisory here).
Gate & verification
graphify gate
PASS — objectively clean (no health regressions, tests not run — proofs not run this pass (advisory)). Grounded, not self-assessed.
Advisory (not blocking):
- verification_scope: 1504 function(s) in the blast radius were not formally verified this run
· 3 grounded finding(s) anchored inline below; 2 more finding(s) on lines outside this diff (see the check run).
| return joined if joined.lower().endswith((".r",)) else None | ||
|
|
||
|
|
||
| def extract_r(path: Path) -> dict: |
There was a problem hiding this comment.
extract_r()
10 callers depend on it (afferent coupling).
Grounded coupling-delta finding (deterministic), not an LLM guess.
| return rhs, lhs | ||
| return None | ||
|
|
||
| def collect_definitions(node, scope_nid: str) -> None: |
There was a problem hiding this comment.
collect_definitions()
fans out to 6 callees (efferent coupling).
Grounded coupling-delta finding (deterministic), not an LLM guess.
| add_node(imp_nid, name, line) | ||
| add_edge(scope_nid, imp_nid, "imports", line, context="import") | ||
|
|
||
| def walk_calls(node, caller_nid: str) -> None: |
There was a problem hiding this comment.
walk_calls()
fans out to 8 callees (efferent coupling).
Grounded coupling-delta finding (deterministic), not an LLM guess.
1 participant
What
Adds R as an extracted language.
.r/.Rwere already inCODE_EXTENSIONS, so R files were counted as code and then contributed nothing — R is the language the #1689 no-AST-extractor warning uses as its example, and that comment is updated here.Why it is bespoke rather than a
LanguageConfigR has no named-function syntax. Every definition is an assignment whose right-hand side is an anonymous
function_definition:function_definition's onlynamefield is thefunctionkeyword itself, so the config-driven walker labels all of themfunction. Right-assign needs care:function(x) x^2 -> nmbindsnminside the body (R deparses it asfunction(x) nm <- x^2), so only the parenthesised form is a function binding. The extractor and its fixture both follow R here.Why calls resolve corpus-wide
Within a package or an analysis directory, R has one shared namespace and no import statement binding a name to a file.
paste0(x)and a sibling file'scompute_moments(x)are the same syntax, and nothing in the file distinguishes them — so a per-file extractor that emits both produces either a base-R god node or a flood of dangling edges.extractors/r.pytherefore resolves only same-file callees (those are certain) and emits everything else asraw_calls.r_resolution.resolve_r_calls, registered throughresolver_registry, links a callee only when the corpus defines it exactly once — the same god-node guard the Java and Objective-C resolvers use. Anything defined nowhere is base R or a third-party package and is dropped.That keeps base R out of the graph with no hardcoded list of ~1,300 base names, and emits no dangling edges.
Sourced files resolve the same way. A bare
source("b.R")becomesimports; an R path literal joined from string literals by a project helper —source_once(paper_path("sub", "b.R")), the dominant shape in real analysis pipelines — becomesreferences. Both link only when the joined path matches exactly one corpus file, so a wrong separator guess cannot invent an edge.Also covered
library/require/requireNamespace/loadNamespaceandpkg::fnimports,obj$method()calls, nested definitions scoped to their enclosing function, top-level statements attributed to the file node (most of an analysis script), andRscriptshebangs.Grammar dependency
The r-lib grammar has no standalone PyPI wheel, so it comes from
tree-sitter-language-pack(~2 MB platform wheel) under a new optional[r]extra, hard-failing liketree-sitter-sqlrather than falling back.get_language()returns a readyLanguage, unlike the per-grammar modules whoselanguage()returns a raw pointer — noted in the code since it is an easy mis-port.Measured
On a 541-file R corpus (an R package plus a 400-file analysis pipeline): 2,171 nodes, 7,434 edges, 0 dangling, 2.2 s. Relations: 4,312
calls, 1,393defines, 927references, 802imports. Highest-degree targets are all real project functions — no base-R hub. Of 30,804 cross-file call sites, 6,466 resolve, 24,127 are dropped as undefined (base R,test_that, package calls), and 211 (0.7%) are dropped as ambiguous across 25 names.Tests
tests/test_r.py— 14 tests covering every binding form, nested definitions, same-file calls, the absence of base-R nodes, imports, the no-dangling-edges invariant, and the four resolver paths (cross-file call, ambiguity guard, sourced file, wrapper-joined path, out-of-corpus path).Two tests in
tests/test_extract.pyused.ras their example of a code extension with no extractor. They move to.ets, which is now one of the two that remain (.ejs,.ets). The subjects of those tests (#1689's warning and #1693's progress denominator) are unchanged.Full suite: same 15 failures as the base commit
00efd6e(11test_skillgen, 4test_ollama_retry_cap— all pre-existing and unrelated), 3,906 passed..Rmdand.Rdare deliberately out of scope — they are in no extension set today, and each is a different format.