Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

x/tools/cmd/guru: consider avoiding reading entire file in globalReferrersPkgLevel #24274

josharian opened this issue Mar 6, 2018 · 0 comments


Copy link

@josharian josharian commented Mar 6, 2018

cmd/guru/referrers.go:globalReferrersPkgLevel currently reads the entire file into memory, in order to do a bytes.Contains check followed by two separate parses.

If #24273 is fixed, it might be nice to avoid reading the entire file into memory. This could be done by writing a bytes.Contains implementation that works on an io.Writer. When a []byte is written, check for the needle in those bytes. If not found, check for a prefix of the needle at the end of the []byte, and save it for the next Write, and check for corresponding suffix at the beginning. (Extra care must be taken for slices that have repeating prefixes!) And of course, once a match is found, all future Writes can be ignored.

Then globalReferrersPkgLevel could use a TeeReader to write to the streaming bytes.Contains while parsing the underlying file. If the relevant import is missing, bail. If the whole file is parsed, but it doesn't contain the target name, bail. Then inspect the AST.

cc @keegancsmith @alandonovan @DanielMorsing

@gopherbot gopherbot added this to the Unreleased milestone Mar 6, 2018
@gopherbot gopherbot added the Tools label Sep 12, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
2 participants
You can’t perform that action at this time.