generated from JetBrains/intellij-platform-plugin-template
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
fix(parsing): improve regex patterns for DevInParser and DevInLexer t…
…o support more complex identifiers and whitespace handling #101 The commit addresses issues with the DevInParser and DevInLexer in the ext/devin-lang package. It introduces more flexible regex patterns for identifiers, allowing characters other than the initial '$', '@', or '/'. Additionally, the commit refactors the lexer to handle whitespace more efficiently, using a dedicated WHITE_SPACE token type. The parser definition is updated to leverage the new token types, and the build script is modified to reflect the changes in the generated parser and lexer classes. Finally, a test file is updated to demonstrate the new parsing capabilities.
- Loading branch information
Showing
6 changed files
with
46 additions
and
23 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1 +1 @@ | ||
Gen hello, world @gen | ||
@gen what's this? |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,12 @@ | ||
FILE | ||
DevInUseAgentImpl(USE_AGENT) | ||
PsiElement(DevInTokenType.@)('@') | ||
PsiElement(DevInTokenType.IDENTIFIER)('gen') | ||
PsiWhiteSpace(' ') | ||
PsiErrorElement:DevInTokenType.$, DevInTokenType./, DevInTokenType.@, DevInTokenType.NEWLINE, DevInTokenType.TEXT_SEGMENT or DevInTokenType.WS expected, got 'what' | ||
PsiElement(DevInTokenType.IDENTIFIER)('what') | ||
PsiElement(DevInTokenType.TEXT_SEGMENT)(''') | ||
PsiElement(DevInTokenType.IDENTIFIER)('s') | ||
PsiWhiteSpace(' ') | ||
PsiElement(DevInTokenType.IDENTIFIER)('this') | ||
PsiElement(DevInTokenType.TEXT_SEGMENT)('?') |