Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[STAL-1960] Add bridge for (TsSymbol <> Name) mappings #387

Merged
merged 1 commit into from
Jun 6, 2024

Conversation

jasonforal
Copy link
Collaborator

What problem are you trying to solve?

One of the fundamental design constraints for ddsa was to represent everything that needed to be passed between Rust and JavaScript with a v8 "smi" (a uint31 on 64 bit platforms). At the same time, we want ergonomic APIs that aren't constrained by performance optimizations like this.

Each tree-sitter CST node (that we care about) has a string name (that it internally calls a TSSymbol) to represent its type:

const myNumber = 123;

The node type names for the nodes in this expression are:

const myNumber = 123;     "lexical_declaration"
myNumber = 123            "variable_declarator"
myNumber                  "identifier"
123                       "number"

We want users to be able to author JavaScript rules that can filter nodes by a string name, like "identifier". We also want to be able to let JavaScript rules request nodes of a given type as well.

In pseudo-code:

const nodes = ddsa.getChildren(targetNode, "variable_declarator");

What is your solution?

In the above example (const nodes = ddsa.getChildren(node, "variable_declarator");), this is a function that will make a call to Rust to request additional tree-sitter nodes. Under the hood, we want to be able to translate this to something like:

const nodes = op_get_children(32 /* nodeId (smi) */, 149 /* TSSymbolId (smi) */);

To achieve this, we create and pre-calculate a v8 map with data for each tree-sitter Language. For example, in JavaScript pseudo-code, this would be like:

const symbolMap = new Map([
    [1, "identifier"],
    ["identifier, 1],
    // ...
    [146, "expression_statement"],
    ["expression_statement", 146],
    // ...
    [225, "decorator"],
    ["decorator", 225],
    // ...
]);

With this map, we can always internally represent the type as a numeric id, but use string names in the user-facing JavaScript API.

This map will be exposed on the JavaScript globalThis on a per-language basis so that ddsa will have access to this map.

Alternatives considered

What the reviewer should know

  • This is just the bridge implementation -- this code is not live. It will be activated when we switch over to ddsa.

Base automatically changed from jf/STAL-1960 to main June 6, 2024 05:54
@jasonforal
Copy link
Collaborator Author

Original PR had #385 as the base, so now there are merge conflicts.

Fixing and then to main we go

@jasonforal jasonforal merged commit 372c94a into main Jun 6, 2024
62 checks passed
@jasonforal jasonforal deleted the jf/STAL-1960-3 branch June 6, 2024 06:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants