Skip to content

Dart extractor emits source_location: None for every node — 0% on 3 repos, ~25k nodes unciteable (regex extractor hardcodes the field) #3365

Description

@santoshpy

Summary

Every node the Dart extractor emits has source_location: None. Not "most" — all of
them, in every Dart file, because extractors/dart.py hardcodes the field at all three
node/edge construction sites. Dart graphs therefore cannot cite a line, and no
downstream consumer can verify a node against the file it claims to come from.

This is worth separating from #3307, which asks for source_location on doc/YAML/markdown
and states as its premise that "code already gets this for free from AST (line numbers
are known at parse time)"
. Dart is a code language with an extractor, and it gets 0%.

Minimal reproduction

Two structurally identical files, ten lines each, one graphify extract . --code-only
run:

// sample.dart
class Greeter {
  final String name;
  Greeter(this.name);

  String greet() {
    return 'hello $name';
  }
}

int add(int a, int b) => a + b;
# sample.py
class Greeter:
    def __init__(self, name):
        self.name = name

    def greet(self):
        return f"hello {self.name}"


def add(a, b):
    return a + b

Resulting nodes:

.dart: 5 nodes, 0 with source_location      .py: 5 nodes, 5 with source_location
  None  sample.dart                            L1  Greeter
  None  add                                    L5  .greet()
  None  Greeter                                L2  .__init__()
  None  greet                                  L9  add()
  None  name                                   L1  sample.py

Scale, on real repositories

Measured across nine repos (all --code-only, graphifyy 0.9.55):

repo dominant language declared nodes with source_location
Plugins Dart 286 0 (0%)
Board Dart 531 0 (0%)
Editor Dart 5,625 0 (0%)
App Dart + Rust 31,958 6,961 (22% — exactly the Rust half)
Cloud Rust 6,304 6,223 (99%)
Web TS/TSX 10,143 10,134 (100%)
Auth Go 3,655 3,583 (98%)
ops Python 707 707 (100%)

~25,000 Dart nodes in one repo with no position at all.

Root cause

extractors/dart.py is regex-based (import re, 31 re.* calls, zero tree_sitter
references — its docstring says "using regex"), so there is no start_point to read,
and the field is written as a literal:

nodes.append({"id": file_nid, "label": path.name, "file_type": "code",
              "source_file": str(path), "source_location": None})          # L54

def add_node(nid, label, ftype="code", source_file=str(path)) -> None:
    if nid not in defined:
        nodes.append({"id": nid, "label": label, "file_type": ftype,
                      "source_file": source_file, "source_location": None})  # L61

edge = {..., "source_file": str(path), "source_location": None, ...}         # L67

add_node takes no line parameter, so callers such as add_node(class_nid, class_name)
(L145) have nowhere to pass one.

Suggested fix, and a trap in it

Each match already knows its offset, so the line is src[:m.start(1)].count("\n") + 1.
Two details matter, both verified against the file below:

  1. Anchor on the capture group, not the match. The class pattern begins ^\s*, so
    m.start() sits before the preceding newlines and reports the line too early.
  2. _comment_replace deletes comments (returns ""), so offsets in the cleaned
    text no longer correspond to the original file. Blanking a comment to an equal number
    of newlines keeps every later line correct.
/*
 * A multi-line header comment.
 * It spans several lines.
 */
class AfterComment {      // real line 5
  void method() {}
}
true line                            : 5
today (comments deleted, m.start(1)) : 2     <- off by the comment's height
newlines preserved + m.start(1)      : 5     <- correct

So: return "\n" * token.count("\n") instead of "" for comment tokens, thread a
line argument through add_node/add_edge, and compute it from m.start(1) at each
re.finditer site.

Why it matters beyond citations

We cross-validate graphs against the source they describe (file-coverage ledger, then
per-node line anchoring, then recall against an independent parser). For Rust, Go, TS
and Python the line anchor confirms a node really is the symbol it claims to be at the
line it claims. For Dart that check cannot run at all, so ~78% of our largest repo's
graph is unverifiable — it reports n/a rather than a pass, and a Dart-only repo can
never be more than partially validated.

Environment

graphifyy 0.9.55 (uv tool, Python 3.12.12), macOS 15 arm64. graphify extract --code-only.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions