Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .worktrees/odin
Submodule odin added at 03d8d4
4 changes: 4 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,9 @@ and adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

### New Features

<<<<<<< HEAD
- CodeGraph now indexes **Odin** (`.odin`). Packages, procedures, structs, enums, unions, and the calls between them land in the graph, so Odin projects show up in search, callers/callees, impact analysis, and `codegraph_explore` flow traces like every other supported language.
=======
- Indexing is dramatically faster on slow storage — mechanical HDDs, network folders, and virtualized disks. The database no longer folds its write journal back into the main file thousands of times during a bulk index (that folding was ~95% of all disk activity); it now streams writes sequentially and folds them back in a few large, coalesced passes that run off the main thread. In a disk-throttled benchmark matching the reported hardware, a mid-size Java project went from over 25 minutes to under a minute, and there is no change on fast disks. Opt out with `CODEGRAPH_NO_WAL_DEFER=1`; tune the fold-back threshold with `CODEGRAPH_WAL_VALVE_MB`. (#1231)
- New `CODEGRAPH_PARSE_TIMEOUT_MS` environment variable to raise the per-file parse budget on unusually slow storage, the same way `CODEGRAPH_PARSE_WORKERS` already tunes the worker count. (#1231)

Expand Down Expand Up @@ -51,6 +54,7 @@ and adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
- Erlang behaviour dispatch is now followed through the graph: a framework call through a variable module — cowboy's `Handler:init`/`Middleware:execute` folds, a plugin manager's `Mod:callback(...)` — links to the repo's implementations of the behaviour that declares that callback, so flow traces and impact cross the OTP callback boundary instead of stopping at it. The links are precision-gated: the callback arity must match, exactly one behaviour may own that callback shape (a collision stays unlinked rather than guessed), the implementer must actually export the callback, and the fan-out is bounded — a behaviour with hundreds of implementers stays a visibly dynamic boundary. Every bridged hop is labeled as dynamic dispatch with its wiring site, never shown as a plain static call.
- CodeGraph now indexes **Erlang** (`.erl`, `.hrl`) — functions, with clauses and arities of the same name grouped as one symbol spanning all of them, plus records with their fields, `-type`/`-opaque` aliases, `-define` macros, and `-spec` signatures attached to every function. Cross-module `mod:fn(...)` calls resolve to the target module's function, `fun name/arity` values are captured as references (so callback registrations like `lists:foreach(fun submit/1, ...)` link up), `-include`/`-include_lib` connect to the header files they pull in, `-behaviour` declarations link a callback module to its behaviour (and only ever to a module — a same-named macro or function elsewhere in the repo is never mistaken for one), and `-export` lists (plus `-compile(export_all)`) drive each function's public/private flag. OTP's indirection idioms are followed where the target is static: `spawn`/`apply`/`proc_lib`/`timer`/`rpc` calls that name their target as `(Module, Function, Args)` arguments produce call edges, and `gen_server:call`/`cast` connects to the target module's `handle_call`/`handle_cast` — its own when targeting `?MODULE` (including the `-define(SERVER, ?MODULE)` idiom), and the named module when a registered name follows OTP's name-the-server-after-its-module convention (`gen_server:call(other_mod, ...)`, directly or through a `-define(STORE, other_mod)` macro); a registered name that matches no module stays unlinked. Macros participate in the graph too: a `-define` body's calls belong to the macro, each `?MACRO(...)` use site links into the call chain (and bare `?CONSTANT` reads are tracked as references), so a call path hidden behind a macro — `set_password → ?SQL_UPSERT_T → sql_query_t` — traces end-to-end and "where is this macro used" is answerable. escripts index like any module (the shebang line is understood), and OTP application resource files (`.app.src`, `.app`) join the graph: `{mod, ...}` links an app to its callback module and `{applications, [...]}` connects umbrella sibling apps — resolving only ever to modules, so an OTP app name like `ssl` is never mistaken for a same-named function. Truly dynamic dispatch (`Mod:handle(...)`, message sends, var-module spawns) is deliberately left unlinked rather than guessed. `codegraph_explore` also understands Erlang-native symbol spelling in queries — `mod:fn/3` and `init/2` find the symbols they name. (#635, #648)
- CodeGraph now indexes **Visual Basic .NET** (`.vb`) — classes, Modules, interfaces, structures, enums, properties, events, `MustOverride` abstract members, and `Declare` P/Invoke signatures, with `Inherits`/`Implements` hierarchy edges, call edges (resolved through VB's ambiguous call-vs-index parentheses), and `New`/`As New` instantiation links. Real-world VB styles parse cleanly: WinForms designer files, interpolated and multi-line strings, XML literals (embedded `<%= %>` expressions included), single-line and multi-line LINQ queries, multi-line lambdas, `Handles`/`WithEvents` event wiring, Custom Events, date literals, classic type-character identifiers (`i%`, `name$`), and non-English (Unicode) identifiers. (#648, #639, #170)
>>>>>>> upstream/main
- CodeGraph now indexes **COBOL** (`.cbl`, `.cob`, `.cpy`) — programs, sections and paragraphs with `PERFORM`/`GO TO` call edges, `CALL` cross-program calls, `COPY` copybook imports (standalone copybooks included), and DATA DIVISION records with 88-level condition names, in both fixed and free source format. Impact queries work on data items: every `MOVE`/`ADD`/`COMPUTE`/`SUBTRACT` write-site links back to the field it changes, so "what touches this copybook field" answers across programs. CICS flows connect too: `EXEC CICS LINK`/`XCTL` program targets, `EXEC SQL INCLUDE` copybooks, and pseudo-conversational `RETURN TRANSID(...)` hops resolve to the program owning the transaction id. (#590, #648)
- CodeGraph now indexes **CFML** (`.cfc`, `.cfm`, `.cfs`) — both the classic tag-based style (`<cfcomponent>`/`<cffunction>`) and modern bare-script `component { ... }` syntax, including `extends`/`implements`, embedded `<cfscript>` blocks (at any nesting depth, including inside `<cfif>`/`<cfloop>`/`<cftry>`), call edges, and calls embedded in `#hash#` expressions inside `<cfquery>` SQL bodies. Files saved with a UTF-8 byte-order mark and tags with unquoted attribute values — both common in long-lived CFML codebases — are handled too. Thanks @ghedwards. (#1118)
- CFML inheritance written as a component path now links to the right component. `extends="coldbox.system.web.Controller"` names its supertype by dotted path and `extends="../base"` by relative path (the FW/1 style) — both previously produced no inheritance edge at all, which on framework-style CFML apps hid most of the type hierarchy from impact and blast-radius analysis (on ColdBox's own core, over 90% of inheritance was invisible). Resolution is deliberately conservative: the target's directory layout must corroborate the declared path — so a supertype that lives in an out-of-repo library (testbox, mxunit, an installed framework) correctly stays unlinked rather than being guessed at, and an ambiguous path produces no edge rather than a wrong one. (#1152)
Expand Down
22 changes: 22 additions & 0 deletions src/extraction/grammars.ts
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,7 @@ const WASM_GRAMMAR_FILES: Record<GrammarLanguage, string> = {
r: 'tree-sitter-r.wasm',
luau: 'tree-sitter-luau.wasm',
objc: 'tree-sitter-objc.wasm',
odin: 'tree-sitter-odin.wasm',
cfml: 'tree-sitter-cfml.wasm',
cfscript: 'tree-sitter-cfscript.wasm',
cfquery: 'tree-sitter-cfquery.wasm',
Expand Down Expand Up @@ -166,10 +167,14 @@ export const EXTENSION_MAP: Record<string, Language> = {
// shape as the `.yml` variants — the YAML/properties extractor emits one node
// per leaf key, and the Spring resolver links `@Value("${k}")` references.
'.properties': 'properties',
<<<<<<< HEAD

=======
// Terraform / OpenTofu / HCL config — tree-sitter-terraform dialect of HCL.
'.tf': 'terraform',
'.tfvars': 'terraform',
'.tofu': 'terraform',
>>>>>>> upstream/main
};

/**
Expand Down Expand Up @@ -357,8 +362,24 @@ export async function loadGrammarsForLanguages(languages: Language[], wasmBytes?
// See: https://github.com/tree-sitter/tree-sitter/issues/2338
for (const lang of toLoad) {
try {
<<<<<<< HEAD
// Some grammars ship their own WASMs (not in tree-sitter-wasms, or the
// tree-sitter-wasms build is too old). Lua: tree-sitter-wasms ships an
// ABI-13 build that corrupts the shared WASM heap under web-tree-sitter
// 0.25 (drops nested calls/imports on every file after the first); we
// vendor the upstream ABI-15 wasm instead. C#: the tree-sitter-wasms
// build (ABI 13) has no primary-constructor support and parses
// `class Foo(...)` as an ERROR that swallows the whole class (#237); we
// vendor the upstream ABI-15 tree-sitter-c-sharp 0.23.5 wasm, which parses
// primary constructors natively.
const wasmPath = (lang === 'pascal' || lang === 'scala' || lang === 'lua' || lang === 'luau' || lang === 'csharp' || lang === 'r' || lang === 'cfml' || lang === 'cfscript' || lang === 'cfquery' || lang === 'cobol')
? path.join(__dirname, 'wasm', wasmFile)
: require.resolve(`tree-sitter-wasms/out/${wasmFile}`);
const language = await WasmLanguage.load(wasmPath);
=======
const bytes = wasmBytes?.[lang];
const language = await WasmLanguage.load(bytes ?? resolveWasmPath(lang));
>>>>>>> upstream/main
languageCache.set(lang, language);
} catch (error) {
const message = error instanceof Error ? error.message : String(error);
Expand Down Expand Up @@ -584,6 +605,7 @@ export function getLanguageDisplayName(language: Language): string {
twig: 'Twig',
xml: 'XML',
properties: 'Java properties',
odin: 'Odin',
cfml: 'CFML',
cfscript: 'CFScript',
cfquery: 'CFQuery (SQL)',
Expand Down
2 changes: 2 additions & 0 deletions src/extraction/languages/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@ import { luaExtractor } from './lua';
import { rExtractor } from './r';
import { luauExtractor } from './luau';
import { objcExtractor } from './objc';

import { cfscriptExtractor } from './cfscript';
import { cfqueryExtractor } from './cfquery';
import { cobolExtractor } from './cobol';
Expand Down Expand Up @@ -60,6 +61,7 @@ export const EXTRACTORS: Partial<Record<Language, LanguageExtractor>> = {
r: rExtractor,
luau: luauExtractor,
objc: objcExtractor,

cfscript: cfscriptExtractor,
cfquery: cfqueryExtractor,
cobol: cobolExtractor,
Expand Down
77 changes: 77 additions & 0 deletions src/extraction/languages/odin.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,77 @@
import type { Node as SyntaxNode } from 'web-tree-sitter';
import type { LanguageExtractor } from '../tree-sitter-types';

export const odinExtractor: LanguageExtractor = {
functionTypes: ['procedure_declaration', 'overloaded_procedure_declaration'],
classTypes: [], // Odin has no classes
methodTypes: [], // Procedures are not attached to classes/objects
interfaceTypes: [], // Odin has no interfaces/traits
structTypes: ['struct_declaration', 'union_declaration', 'bit_field_declaration'],
enumTypes: ['enum_declaration'],
enumMemberTypes: ['identifier'], // Enum values are identifiers
typeAliasTypes: [],
importTypes: ['import_declaration'],
callTypes: ['call_expression', 'selector_call_expression'],
variableTypes: ['variable_declaration', 'var_declaration', 'const_declaration', 'const_type_declaration'],
fieldTypes: ['field'], // Struct fields

nameField: 'name',
bodyField: 'body',
paramsField: 'parameters',

resolveName: (node: SyntaxNode, source: string) => {
// In Odin, declarations are structured as: name :: definition or name : type := definition
// The LHS name (an identifier or expression) is the first named child.
const first = node.firstNamedChild;
if (first) {
return source.substring(first.startIndex, first.endIndex).trim();
}
return undefined;
},

resolveBody: (node: SyntaxNode, _bodyField: string) => {
if (node.type === 'procedure_declaration') {
const procNode = node.namedChildren.find(c => c.type === 'procedure');
if (procNode) {
const block = procNode.namedChildren.find(c => c.type === 'block');
if (block) return block;
}
} else if (
node.type === 'struct_declaration' ||
node.type === 'union_declaration' ||
node.type === 'bit_field_declaration' ||
node.type === 'enum_declaration'
) {
// The struct/enum fields are direct children of the declaration node.
// Returning the node itself allows the core extractor to visit its children.
return node;
}
return null;
},

getSignature: (node: SyntaxNode, source: string) => {
const procNode = node.namedChildren.find(c => c.type === 'procedure' || c.type === 'overloaded_procedure');
if (procNode) {
return source.substring(procNode.startIndex, procNode.endIndex).trim();
}
return undefined;
},

extractImport: (node: SyntaxNode, source: string) => {
const pathNode = node.childForFieldName('path');
if (!pathNode) return null;

let modulePath = source.substring(pathNode.startIndex, pathNode.endIndex).trim();
// Strip string quotes
if ((modulePath.startsWith('"') && modulePath.endsWith('"')) ||
(modulePath.startsWith('`') && modulePath.endsWith('`'))) {
modulePath = modulePath.slice(1, -1);
}

const signature = source.substring(node.startIndex, node.endIndex).trim();
return {
moduleName: modulePath,
signature: signature,
};
},
};
3 changes: 2 additions & 1 deletion src/extraction/tree-sitter.ts
Original file line number Diff line number Diff line change
Expand Up @@ -1810,7 +1810,8 @@ export class TreeSitterExtractor {
// Skip forward declarations and type references (no body = not a definition)
// — EXCEPT C# positional records (`record struct M(decimal Amount);`),
// complete definitions with no body block. (#831)
const body = getChildByField(node, this.extractor.bodyField);
const body = this.extractor.resolveBody?.(node, this.extractor.bodyField)
?? getChildByField(node, this.extractor.bodyField);
if (!body && node.type !== 'record_declaration') return;

const name = extractName(node, this.source, this.extractor);
Expand Down
Binary file added src/extraction/wasm/tree-sitter-odin.wasm
Binary file not shown.
1 change: 1 addition & 0 deletions src/types.ts
Original file line number Diff line number Diff line change
Expand Up @@ -98,6 +98,7 @@ export const LANGUAGES = [
'twig',
'xml',
'properties',

'cfml',
'cfscript',
'cfquery',
Expand Down