Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: use strip-literal for more correct comments removal #60

Merged
merged 19 commits into from May 31, 2022

Conversation

leo-buneev
Copy link
Contributor

When applying unimport on huge codebase, I've encountered many issues when global identifiers were not detected.
Some of them were fixed in #58 , but after that fix I've found several more complicated cases, e.g.:

const url = `http://${ref()}`

"//" was detected as single-line comment, discarded rest of the line, and first "`" quote was left unclosed, which lead to discarding of entire code.

This is complex to fix via replacing regexes, because many combinations are possible depending on the order - `str` // comment vs `str // not comment`.

To fix it properly we probably need to parse the syntax tree, but if I understood correctly Vite doesn't want to do it for performance reasons. So I've written small "state machine" to correctly detect strings and comments regardless of nesting in in O(n) time.

@antfu antfu changed the title Rewrite logic of removing comments and strings using state machine fix: use strip-literal for more correct comments removal May 31, 2022
@antfu antfu merged commit 087407e into unjs:main May 31, 2022
@antfu
Copy link
Member

antfu commented May 31, 2022

Thanks for the effort on this. Rather than maintaining the complexity in the repo, I have switched to using https://github.com/antfu/strip-literal, a library that Vite is relying on to remove the comments internals.

@leo-buneev
Copy link
Contributor Author

Yes, that's better, thanks

@pi0
Copy link
Member

pi0 commented May 31, 2022

This is so nice with strip-literal but as an effect, I see we are now using acorn parser on code now. Using with rollup, it would be nice to find a way reusing parsed AST...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants