Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix/tree sitter parsing expression failure #186

Open
wants to merge 4 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ and this project adheres to [Semantic Versioning](http://semver.org/).
- (Rust) Support for r# raw strings with step definition patterns ([#176](https://github.com/cucumber/language-service/pull/176))
- (Rust) Line continuation characters in rust step definition patterns ([#179](https://github.com/cucumber/language-service/pull/179))
- (Python) Unexpected spaces and commas in generated step definitions [#160](https://github.com/cucumber/language-service/issues/160)
- (TypeScript) Tree sitter parser failing on class decorators ([#186](https://github.com/cucumber/language-service/pull/186))

### Added
- (Python) Support for u-strings with step definition patterns ([#173](https://github.com/cucumber/language-service/pull/173))
Expand Down
12 changes: 10 additions & 2 deletions src/language/SourceAnalyzer.ts
Original file line number Diff line number Diff line change
@@ -1,7 +1,12 @@
import { RegExps, StringOrRegExp } from '@cucumber/cucumber-expressions'
import { LocationLink } from 'vscode-languageserver-types'

import { createLocationLink, makeParameterType, syntaxNode } from './helpers.js'
import {
createLocationLink,
makeParameterType,
stripBlacklistedExpressions,
syntaxNode,
} from './helpers.js'
import { getLanguage } from './languages.js'
import {
Language,
Expand Down Expand Up @@ -136,7 +141,10 @@ language: ${source.languageName}
private parse(source: Source<LanguageName>): TreeSitterTree {
let tree: TreeSitterTree | undefined = this.treeByContent.get(source)
if (!tree) {
this.treeByContent.set(source, (tree = this.parserAdapter.parser.parse(source.content)))
// This is currently necessary since the tree-sitter parser currently errors on certain expressions
const content = stripBlacklistedExpressions(source.content, source.languageName)
tree = this.parserAdapter.parser.parse(content)
this.treeByContent.set(source, tree)
}
return tree
}
Expand Down
71 changes: 70 additions & 1 deletion src/language/helpers.ts
Original file line number Diff line number Diff line change
@@ -1,7 +1,13 @@
import { ParameterType, RegExps } from '@cucumber/cucumber-expressions'
import { DocumentUri, LocationLink, Range } from 'vscode-languageserver-types'

import { Link, NodePredicate, TreeSitterQueryMatch, TreeSitterSyntaxNode } from './types'
import {
LanguageName,
Link,
NodePredicate,
TreeSitterQueryMatch,
TreeSitterSyntaxNode,
} from './types'

export function syntaxNode(match: TreeSitterQueryMatch, name: string): TreeSitterSyntaxNode | null {
const nodes = syntaxNodes(match, name)
Expand Down Expand Up @@ -70,3 +76,66 @@ export function filter(
function flatten(node: TreeSitterSyntaxNode): TreeSitterSyntaxNode[] {
return node.children.reduce((r, o) => [...r, ...flatten(o)], [node])
}

/**
*
* This constant represents a record of language names that contain lists
* of regular expressions that should be stripped from the content of a file.
*/
export const BLACKLISTED_EXPRESSIONS: {
[key in LanguageName]: RegExp[]
} = {
tsx: [
/*
* This regular expression matches sequences of decorators applied to a class,
* potentially including type parameters and arguments.
* The regex supports matching these patterns preceding
* an optionally exported class definition.
*/
/(@(\w+)(?:<[^>]+>)?\s*(?:\([^)]*\))?\s*)*(?=\s*(export)*\s+class)/g,
],
java: [],
c_sharp: [],
php: [],
python: [],
ruby: [],
rust: [],
javascript: [],
}

/**
*
* Strips blacklisted expressions from the given content.
*
* @param content The content to strip blacklisted expressions from.
* @param languageName The name of the language to use for stripping.
*
* @returns The content with blacklisted expressions stripped.
*
* @example
*
* ```typescript
* const content =
* "@decorator\n" ++
* "export class Foo {\n" ++
* "@decorator\n" ++
* "public bar() { }\n" ++
* "}"
*
* const strippedContent = stripBlacklistedExpressions(content, 'tsx')
* console.log(strippedContent)
*
* // Output:
* "export class Foo {\n" ++
* "@decorator\n" ++
* "public bar() { }\n" ++
* "}"
*
* ```
*/
export function stripBlacklistedExpressions(content: string, languageName: LanguageName): string {
return BLACKLISTED_EXPRESSIONS[languageName].reduce(
(acc, regExp) => acc.replace(regExp, ''),
content
)
}