Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor(parser): remove TokenValue::RegExp from Token #1926

Merged
merged 1 commit into from
Jan 8, 2024

Conversation

Boshen
Copy link
Member

@Boshen Boshen commented Jan 7, 2024

This PR is part of #1880.

Token size is reduced from 48 to 40 bytes.

To reconstruct the regex pattern and flags within the parser , the regex string is
re-parsed from the end by reading all valid flags.

In order to make things work nicely, the lexer will no longer recover
from a invalid regex.

Copy link
Member Author

Boshen commented Jan 7, 2024

@github-actions github-actions bot added A-parser Area - Parser A-ast Area - AST labels Jan 7, 2024
@Boshen Boshen force-pushed the 01-07-refactor_parser_remove_TokenValue_RegExp branch from aa6aa76 to 9fac4e4 Compare January 7, 2024 15:15
@Boshen Boshen changed the title refactor(parser): parse BigInt lazily refactor(parser): remove TokenValue::RegExp from Token Jan 7, 2024
@Boshen Boshen requested a review from Dunqing January 7, 2024 15:16
Copy link

codspeed-hq bot commented Jan 7, 2024

CodSpeed Performance Report

Merging #1926 will not alter performance

Comparing 01-07-refactor_parser_remove_TokenValue_RegExp (27c3023) with main (73a5b7f)

Summary

✅ 14 untouched benchmarks

@Dunqing
Copy link
Member

Dunqing commented Jan 8, 2024

It looks like this change breaks some cases.

Base automatically changed from 01-07-refactor_parser_parse_BigInt_lazily to main January 8, 2024 04:37
This PR is part of #1880.

`Token` size is reduced from 48 to 40 bytes.

To reconstruct the regex pattern and flags within the parser , the regex string is
re-parsed from the end by reading all valid flags.

In order to make things work nicely, the lexer will no longer recover
from a invalid regex.
@Boshen Boshen force-pushed the 01-07-refactor_parser_remove_TokenValue_RegExp branch from 9fac4e4 to 27c3023 Compare January 8, 2024 04:51
@Boshen Boshen merged commit 08438e0 into main Jan 8, 2024
18 checks passed
@Boshen Boshen deleted the 01-07-refactor_parser_remove_TokenValue_RegExp branch January 8, 2024 05:48
@Boshen Boshen changed the title refactor(parser): remove TokenValue::RegExp from Token refactor(parser): remove TokenValue::RegExp from Token Jan 8, 2024
@Boshen Boshen mentioned this pull request Jan 8, 2024
Boshen pushed a commit that referenced this pull request Jan 12, 2024
#1926 produced a small performance regression because when parsing a
regexp, some work is repeated.
Boshen pushed a commit that referenced this pull request Jan 12, 2024
As discussed in
#1999 (comment),
this PR restores some of regex parsing behavior to as it was prior to
#1926.
IWANABETHATGUY pushed a commit to IWANABETHATGUY/oxc that referenced this pull request May 29, 2024
…#1926)

This PR is part of oxc-project#1880.

`Token` size is reduced from 48 to 40 bytes.

To reconstruct the regex pattern and flags within the parser , the regex
string is
re-parsed from the end by reading all valid flags.

In order to make things work nicely, the lexer will no longer recover
from a invalid regex.
IWANABETHATGUY pushed a commit to IWANABETHATGUY/oxc that referenced this pull request May 29, 2024
oxc-project#1926 produced a small performance regression because when parsing a
regexp, some work is repeated.
IWANABETHATGUY pushed a commit to IWANABETHATGUY/oxc that referenced this pull request May 29, 2024
As discussed in
oxc-project#1999 (comment),
this PR restores some of regex parsing behavior to as it was prior to
oxc-project#1926.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-ast Area - AST A-parser Area - Parser
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants