-
Notifications
You must be signed in to change notification settings - Fork 34
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
perf: alias scanning in files #408
Conversation
@@ -0,0 +1 @@ | |||
EVEN_WILDER = 'wildFlag' |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
😂
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can't say I understand entirely how this all works but the code seems 💯
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Approving to allow work to proceed. The Docs team is codeowners of *.md
files in this repo, so are autotagged here, but I did not review the vendor *.md
files carefully.
unfortunately the build is failing now that we're using re2, so i'm going to revert that change |
Improve performance of alias scanning in files in two ways
doublestar.FilepathGlob(glob)
so that consecutive calls with same pattern can use resultsswap out golang regexp with https://github.com/wasilibs/go-re2With just change 1 I still found the tools running over 30 min (i stopped it before it finished) in one repo, but by adding change 2 it cut the time to under 10 min. Similarly, just implementing change 2 saw run times over 20 min (i stopped it before it finished).
This isn't a total fix for performance of scanning, depending on size of repo and wildcard patterns used, but it certainly helps.
UPDATE:
pulling back on the re2 usage since the build fails - we can revisit that later