Skip to content

use tree-sitter for completion and better errors#37

Merged
vito merged 15 commits into
mainfrom
ts-lsp
Feb 27, 2026
Merged

use tree-sitter for completion and better errors#37
vito merged 15 commits into
mainfrom
ts-lsp

Conversation

@vito
Copy link
Copy Markdown
Owner

@vito vito commented Feb 27, 2026

Currently our LSP uses a rube-goldberg approach to determining what syntax node is being completed, working with raw text, because we have to handle states where there isn't a parseable AST (e.g. user typed . and needs completion).

This PR shifts over to using the tree-sitter grammar instead, which gives us a partial AST when there are errors.

Now that we're binding to tree-sitter, we need to build with CGo, which makes things a bit complicated. I opted to use zig to keep static builds in the Dagger SDK, which seems to work pretty well.

vito added 15 commits February 27, 2026 12:57
Dot-completion on a local variable (e.g. ctr.fr where ctr is a
let binding) fails because the text-based path cannot resolve
local variables from the top-level TypeEnv, and FindReceiverAt
misses the Select node when the cursor is one column past its end.

Signed-off-by: Alex Suraci <suraci.alex@gmail.com>
Add tree-sitter-based receiver finding to the LSP completion handler.
When dot-completion is requested, the current buffer text is parsed
with tree-sitter to locate select_or_call nodes or ERROR nodes with
dots at the cursor position. The receiver expression text is extracted
and resolved using a combined type environment that merges the
file-level env with all enclosing scope bindings from the Dang AST.

This fixes completion on local variables (e.g. ctr.fr completing to
ctr.from) where the receiver is not in the top-level TypeEnv, and
also handles multi-line method chains thanks to the external scanner
from the previous commit.

Signed-off-by: Alex Suraci <suraci.alex@gmail.com>
Remove FindNodeAt, FindReceiverAt, FindFunCallAt, and
containsPosition from ast_query.go — these were either unused
or superseded by the tree-sitter-based receiver finding. Remove
the FindReceiverAt fallback from the completion handler since
tree-sitter now handles all member completion cases.

This removes 125 lines of code with no behavior change.

Signed-off-by: Alex Suraci <suraci.alex@gmail.com>
buildCompletionEnv cast the cloned TypeEnv to *Module, but
NewPreludeEnv returns *CompositeModule. The failed cast caused
the function to return the unmodified file-level env without
enclosing scope bindings, so local variable member completion
(e.g. ctr.from) silently returned no results.

Signed-off-by: Alex Suraci <suraci.alex@gmail.com>
The global tsParser is not thread-safe but was called from
concurrent LSP request goroutines. This caused a SIGABRT when
a completion request raced with a didChange notification.
Protect the Parse call with a sync.Mutex.

Signed-off-by: Alex Suraci <suraci.alex@gmail.com>
Proposes replacing the three-fallback completion strategy with a
single tree-sitter-first pipeline: parse → classify cursor context
→ resolve types → emit completions. Keeps the Dang type checker
as the source of truth for types while using tree-sitter for
robust syntax analysis on partial/broken input.

Signed-off-by: Alex Suraci <suraci.alex@gmail.com>
Rewrites the design doc to make REPL/LSP unification a first-class
concern. The shared entry point is dang.Complete(ctx, env, text,
line, col) in pkg/dang, called by both the LSP handler and the
REPL's completion provider. Tree-sitter code moves from pkg/lsp
to pkg/dang so both consumers can use it.

Signed-off-by: Alex Suraci <suraci.alex@gmail.com>
Add dang.Complete() as the single entry point for both LSP and REPL
completion. It uses tree-sitter to parse text and classify the cursor
context (dot-member, argument, or bare-ident), then resolves types
from the provided environment.

Move danglang/ bindings and tree-sitter helpers from pkg/lsp/ to
pkg/dang/ so both callers can share them. The LSP handler simplifies
from three fallback paths to a single dang.Complete call. The REPL
switches from dang.CompleteInput to dang.Complete, gaining multi-line
support and tree-sitter-based context detection.

Argument completion still uses the proven string-based splitArgExpr
as a bridge since tree-sitter doesn't produce arg_values nodes for
incomplete calls. This can move to tree-sitter in a future phase.

Signed-off-by: Alex Suraci <suraci.alex@gmail.com>
Signed-off-by: Alex Suraci <suraci.alex@gmail.com>
Signed-off-by: Alex Suraci <suraci.alex@gmail.com>
Use only the current line up to the cursor for splitArgExpr instead
of the full file text. The full text caused false positives from
unmatched parens on earlier lines, preventing dot-member completion
from working after lines containing function calls.

Signed-off-by: Alex Suraci <suraci.alex@gmail.com>
Replace splitArgExpr with tree-sitter CST analysis for argument
completion. The new findArgContextAtCursor handles three CST
patterns: well-formed arg_values nodes, ERROR siblings of
select_or_call, and root ERROR nodes with flattened children.

Remove CompleteInput, splitDotExpr, splitArgExpr, matchingOpen,
extractProvidedArgNames, and their tests. All completion now flows
through dang.Complete which uses tree-sitter exclusively for cursor
context classification.

Signed-off-by: Alex Suraci <suraci.alex@gmail.com>
Signed-off-by: Alex Suraci <suraci.alex@gmail.com>
the deed is done

Signed-off-by: Alex Suraci <suraci.alex@gmail.com>
When the PEG parser fails, use tree-sitter to parse the same source
and find ERROR/MISSING nodes. These produce SourceErrors with precise
locations and human-readable messages instead of the PEG's opaque
"no match found, expected: ..." output.

Examples of improved messages:
- "unexpected 'let y ='" (incomplete binary operator)
- "unterminated string literal" (missing closing quote)
- "expected '=' after variable name" (missing equals in let)
- "missing closing ')'" (unclosed parenthesis)
- "incomplete 'if' expression" (if without block)

All parse entry points (RunFile, LSP handler, REPL) now use
ParseWithRecovery/ParseFileWithRecovery which fall through to the
enhanced error on PEG failure.

Signed-off-by: Alex Suraci <suraci.alex@gmail.com>
@vito vito changed the title use tree-sitter for completion use tree-sitter for completion and better errors Feb 27, 2026
@vito vito merged commit 5a7b462 into main Feb 27, 2026
6 checks passed
@vito vito deleted the ts-lsp branch May 20, 2026 20:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant