Skip to content

fix(parsers): compile JS highlights and add Dart highlights so modifiers and decorators populate - #779

Merged
vitali87 merged 7 commits into
mainfrom
fix/525-highlights-js-dart
Jul 17, 2026
Merged

fix(parsers): compile JS highlights and add Dart highlights so modifiers and decorators populate#779
vitali87 merged 7 commits into
mainfrom
fix/525-highlights-js-dart

Conversation

@vitali87

Copy link
Copy Markdown
Owner

Closes #525.

What was actually missing

The modifier/decorator extraction pipeline this issue asks for already exists end to end: _create_highlights_query loads a highlights query per language (pip package HIGHLIGHTS_QUERY plus a queries/highlights/*.scm fallback), extract_modifiers_and_decorators turns @keyword.modifier and @attribute/@function.decorator captures into the modifiers and decorators node properties, and function/method/class ingestion attaches them. What remained were two silent per-language holes:

  • JavaScript had no highlights query at all. javascript.scm listed TypeScript-only tokens (public, private, protected, readonly, abstract, declare, override) that are not node kinds in the JS grammar. One invalid token makes the whole concatenated query fail to compile, and the loader degrades to None with only a startup debug log, so JS lost decorators and its real modifiers (static, async, ...) too.
  • Dart had no highlights source. The pip package ships no HIGHLIGHTS_QUERY and no fallback file existed. Dart also needed two shape rules in the generic walk: a class member's static token belongs to the method_signature wrapper (not the captured function_signature), and @override-style metadata precedes the signature as sibling annotation nodes.

Fix

  • javascript.scm trimmed to JS-valid tokens (async, export, default, static, get, set) plus (decorator).
  • New dart.scm (annotations + static/final/abstract/late/external/covariant/get/set).
  • extract_modifiers_and_decorators promotes a Dart function_signature to its wrapping method_signature and collects preceding annotation siblings.

Tests (RED first)

  • test_every_language_loads_a_highlights_query — the loud regression gate: a highlights query that fails to compile can no longer degrade silently for months.
  • test_js_method_modifiers_and_decorators_capturedstatic/async/@dec on a JS class method.
  • test_dart_annotations_and_modifiers_capturedstatic/@override on a Dart class method.

With this, all 12 supported languages (plus TSX) load a compiling highlights query and populate the modifiers/decorators properties, which completes the remaining open sub-issue of #521 (#523 already delivered scope tracking).

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces fallback tree-sitter highlights for Dart and fixes an issue where TypeScript-only modifiers in javascript.scm caused the concatenated highlights query to fail compiling silently. It also adds corresponding tests to verify modifier and decorator extraction for JS and Dart, and ensures all supported languages load a highlights query. The review feedback suggests adding the const modifier to the Dart highlights query and refactoring the language-loading test to iterate over sorted parsers for deterministic execution and complete coverage.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment thread codebase_rag/queries/highlights/dart.scm
Comment thread codebase_rag/tests/test_modifiers_and_decorators.py Outdated
@greptile-apps

greptile-apps Bot commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR fixes highlight-query coverage for JavaScript and Dart. The main changes are:

  • Trimmed the JavaScript fallback highlights query to JS-valid modifier tokens and decorator captures.
  • Added a Dart fallback highlights query for annotations and modifier nodes/tokens.
  • Updated modifier/decorator extraction to include Dart method wrappers and preceding annotation siblings.
  • Added regression tests for highlight-query loading and JS/Dart modifier/decorator extraction.

Confidence Score: 5/5

Safe to merge with minimal risk.

The change is scoped to highlight query definitions, Dart wrapper handling in the existing extraction flow, and targeted regression tests. No blocking or non-blocking issues were identified.

No files require special attention.

T-Rex T-Rex Logs

What T-Rex did

  • T-Rex captured that cmake and cmake3 were not available in the environment by running command -v cmake and command -v cmake3.
  • T-Rex attempted the uv/pytest run for modifiers-decorators and the command exited with status 1 because pymgclient couldn't find a suitable cmake.
  • T-Rex ran a final narrow fallback pytest against the real test file, but the setup failed because codebase_rag.cli patch/import setup could not be completed.
  • T-Rex executed a direct import check that showed codebase_rag.cli requires pydantic_ai in the fallback environment, explaining the blocker.

View all artifacts

T-Rex Ran code and verified through T-Rex

Important Files Changed

Filename Overview
codebase_rag/constants/ast_dart.py Adds the Dart annotation node constant used by the modifier/decorator extraction path.
codebase_rag/parsers/utils.py Extends highlight extraction to include Dart method wrapper tokens and preceding annotation siblings.
codebase_rag/queries/highlights/dart.scm Adds a Dart highlights fallback for annotations and modifier tokens/nodes.
codebase_rag/queries/highlights/javascript.scm Restricts JavaScript highlight modifiers to JS-valid tokens while preserving decorator captures.
codebase_rag/tests/test_modifiers_and_decorators.py Adds regression tests for highlight-query availability and JS/Dart modifier/decorator capture.
uv.lock Updates the editable package version metadata only.

Sequence Diagram

%%{init: {'theme': 'neutral'}}%%
sequenceDiagram
participant Loader as load_parsers/_create_highlights_query
participant SCM as queries/highlights/*.scm
participant Extractor as extract_modifiers_and_decorators
participant Node as Function/Class AST node

Loader->>SCM: Load package HIGHLIGHTS_QUERY plus fallback SCM
SCM-->>Loader: Compiled highlights Query for JS/Dart
Node->>Extractor: Function/method/class node
Extractor->>Extractor: Promote Dart function_signature to method_signature
Extractor->>Extractor: Include preceding decorator/annotation siblings
Extractor-->>Node: modifiers and decorators lists
Loading
%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
sequenceDiagram
participant Loader as load_parsers/_create_highlights_query
participant SCM as queries/highlights/*.scm
participant Extractor as extract_modifiers_and_decorators
participant Node as Function/Class AST node

Loader->>SCM: Load package HIGHLIGHTS_QUERY plus fallback SCM
SCM-->>Loader: Compiled highlights Query for JS/Dart
Node->>Extractor: Function/method/class node
Extractor->>Extractor: Promote Dart function_signature to method_signature
Extractor->>Extractor: Include preceding decorator/annotation siblings
Extractor-->>Node: modifiers and decorators lists
Loading

Reviews (2): Last reviewed commit: "fix(parsers): capture dart const and fin..." | Re-trigger Greptile

Comment thread codebase_rag/tests/test_modifiers_and_decorators.py Outdated
@vitali87

Copy link
Copy Markdown
Owner Author

@greptile review

@sonarqubecloud

Copy link
Copy Markdown

@vitali87
vitali87 merged commit 29bae76 into main Jul 17, 2026
22 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

feat: extract access modifiers and decorators via highlights.scm queries

1 participant