feat(extract): Pine Script (.pine) extractor - #2480
Conversation
TradingView Pine has no tree-sitter grammar, so this is a line-oriented extractor in the same family as the Pascal regex fallback. Pine is small and regular -- declarations sit at column 0 and blocks are indentation scoped -- so a lexer over comment/string-stripped lines recovers the structure an AST would give. Nodes: script declaration (indicator/strategy/library + title), user functions and methods, user-defined types and enums, inputs, imported libraries. Edges: contains, calls (user function -> user function), references (function -> input it reads), uses (script -> notable built-in), imports. Two deliberate choices: - Inputs are first-class nodes. In Pine the inputs are the tunable surface of a strategy, so "which function reads slBuf" is the question people actually ask of a trading script. - Built-ins are a curated whitelist (strategy.entry/exit/close, alertcondition, request.security, box.new, line.new, plot, ...) rather than every dotted call. Edges to math.* / ta.* would hub the graph into god nodes; these are the calls that change what a script does, so they answer cross-script questions without that cost. Built-in and library nodes are unscoped so two scripts share them; user function IDs are file-scoped so a calcQty() defined in two strategies stays two nodes. Validated against a 3,714-line corpus of 9 real strategies: 82/82 function definitions detected, no false positives. Handles multi-line signatures, "//" inside string literals (common in Pine alert payloads), and calls that resemble definitions.
![graphify-labs[bot]](https://avatars.githubusercontent.com/in/4356270?s=60&v=4){kind=small}
There was a problem hiding this comment.
Looks safe to merge — no coupling regressions and no blocking issues, checked against the code graph (not a self-assessment).
Graphify reviewed this change.
Looks safe to merge — no coupling regressions and no blocking issues, checked against the code graph (not a self-assessment).
Graphify review — findings
This PR adds a Pine Script (.pine) extractor to graphify. Since TradingView's Pine has no tree-sitter grammar, it introduces a new line-oriented/regex-based extractor (graphify/extractors/pine.py) in the same style as the existing Pascal fallback, wired into the dispatch table, the extractor registry, and the recognized code extensions in detect.py. The extractor is intended to produce nodes for the script declaration, user functions/methods, user-defined types/enums, inputs, and imported libraries, along with contains/calls/references/uses/imports edges (with uses limited to a curated built-in whitelist and node IDs scoped per file). Supporting changes include README and CHANGELOG documentation updates and a new test suite covering the Pine extractor. Surface area touched: detect.py, extract.py, extractors/__init__.py, the new extractors/pine.py, associated tests, README, and CHANGELOG.
No blocking issues surfaced. 2 lower-confidence candidates did not survive cross-model review.
Analysis details — impact, health, verification
Impact & health
Graphify review
Impact — 2136 functions depend on the 962 functions this change touches.
Health — this change adds coupling hotspots:
- new:
extract_pine()— 16 callers, 8 callees
Verification — 2136 functions in the blast radius were not formally verified this run (proofs are advisory here).
Gate & verification
graphify gate
PASS — objectively clean (no health regressions, tests not run — proofs not run this pass (advisory)). Grounded, not self-assessed.
Advisory (not blocking):
- verification_scope: 1950 function(s) in the blast radius were not formally verified this run
· 1 grounded finding(s) anchored inline below.
| return start | ||
|
|
||
|
|
||
| def extract_pine(path: Path) -> dict: |
There was a problem hiding this comment.
extract_pine()
fans out to 8 callees (efferent coupling); 16 callers depend on it (afferent coupling).
Grounded coupling-delta finding (deterministic), not an LLM guess.
1 participant
What
Adds a Pine Script (
.pine) extractor, following the "Adding a new language extractor" steps in ARCHITECTURE.md.TradingView's Pine has no tree-sitter grammar, so this is a line-oriented extractor in the same family as the regex fallback in
pascal.py. Pine is small and highly regular — declarations live at column 0, blocks are indentation scoped, there is no macro layer — so a lexer over comment/string-stripped lines recovers the structure an AST would give.Nodes: the script declaration (
indicator/strategy/libraryplus its title), user functions and methods, user-defined types and enums, inputs, and imported libraries.Edges:
contains,calls(user function → user function),references(function → input it reads),uses(script → notable built-in),imports.Two decisions worth reviewing
Inputs are first-class nodes. In Pine the inputs are the tunable surface of a strategy. "Which function reads
slBuf" is the question people actually ask of a trading script, and it is unanswerable if inputs are just text.Built-ins are a curated whitelist, not every dotted call. Only the calls that change what a script does:
strategy.entry/exit/close/order,alertcondition,request.security,box.new,line.new,label.new,plot*, container constructors. Edges tomath.*/ta.*would hub the graph into god nodes and drown the real structure. Built-in and library nodes are unscoped so two scripts share them (that shared edge is the point: "which strategies request a higher timeframe?"), while user function IDs are file-scoped, since Pine has no cross-file symbol resolution outside explicit library imports and acalcQty()defined in two scripts is genuinely two functions.Validation
Run against a private corpus of 9 real strategies, 3,714 lines of Pine: 82/82 function definitions detected, no false positives and none missed, checked against a ground-truth scan of the
=>definition lines. On the largest script (1,223 lines) it recovers 59 functions and all 35 inputs exactly.Handles the cases that break a naive regex:
f(float a,\n float b) =>)//inside string literals — common in Pine alert payloads carrying URLs — which is why comment stripping is character-wise rather than a regexma = ta.sma(close, 14)must not become a node)Tests
tests/fixtures/sample.pine+ 6 tests intests/test_languages.py, per the contributing guidelinestests/test_pine.py— 13 focused cases (scoping, comment/string handling, multi-line signatures, empty file, no self-edges)Full suite on this branch: 3910 passed. The 56 failures on my machine (
test_skillgen,test_watch,test_uninstall_scope) are pre-existing on Windows — I ran the suite with-p no:randomlybefore and after the change and the failure set is byte-identical, so nothing here is new. Happy to have CI confirm on Linux.Notes
.pineneeds no entry in_EXTRA_FOR_EXTENSION..pineadded toCODE_EXTENSIONSindetect.py;watch.pypicks it up automatically since_WATCHED_EXTENSIONSderives from it..mq5/.mqh) is the obvious follow-up for the same audience and is C-like enough to likely reuse the generic core rather than a bespoke extractor. Happy to send that separately if this direction looks right.