New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: only rebuild dependent packages when export data has changed #15752

Open
josharian opened this Issue May 19, 2016 · 7 comments

Comments

Projects
None yet
7 participants
@josharian
Contributor

josharian commented May 19, 2016

@mwhudson suggested in #15734:

If the export data hasn't changed, can you skip the compilation of dependent packages?

Moving this idea here to be tracked separately.

My 2c: Interesting idea, seems worth pursuing. Not sure how often it happens that code changes don't impact export data (or how cheap that is to detect), but when it does happen, that could save a bunch of computation.

@dr2chase

This comment has been minimized.

Contributor

dr2chase commented May 19, 2016

It's not trivial. Years a go I worked on an ahead-of-time compiler for Java bytecodes that did just this, and it was a good thing (back in the days of 200Mhz CPUs), but it required substantial amounts of dependence information (stored on disk in gzip format) and it also required that every reference to information from another package log an entry in the internal dependence database for later storage -- in particular, every field reference or interface method reference, because those involve offsets, and offsets might change as interface declarations or field declarations are changed (granularity matters, too -- do you record the actual dependence that field F is at offset 8, or do you just record that the SHA of the struct had a particular value?

@josharian

This comment has been minimized.

Contributor

josharian commented May 19, 2016

IIUC, the plan would be to do something like check the SHA of the export data. If that hasn't changed, then the downstream compilation won't change.

@mwhudson

This comment has been minimized.

Contributor

mwhudson commented May 19, 2016

Yeah, I was just thinking that you'd store the SHA1 of the export data of
all imported packages in the .a file and only recompile if any of them had
changed. No sense trying to be too sophisticated.

I don't know often this would be useful. Certainly sometimes (especially in
a "add more and more debug prints" cycle), but it will obviously also have
a small cost all the time. Seems worth trying out, I'll try to hack
something up between now and 1.8 early.

On 20 May 2016 at 06:56, Josh Bleecher Snyder notifications@github.com
wrote:

IIUC, the plan would be to do something like check the SHA of the export
data. If that hasn't changed, then the downstream compilation won't change.


You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
#15752 (comment)

@josharian josharian added the ToolSpeed label May 20, 2016

@griesemer

This comment has been minimized.

Contributor

griesemer commented Jun 2, 2016

Régis Crelier (currently at Google) has investigated this issue for Oberon in very much detail - I believe much of his work should be directly applicable to Go. For details see:
http://e-collection.library.ethz.ch/eserv/eth:39386/eth-39386-02.pdf

@josharian

This comment has been minimized.

Contributor

josharian commented Jul 6, 2017

@rsc will this be a part of your cmd/go re-building improvements?

@rsc

This comment has been minimized.

Contributor

rsc commented Oct 25, 2017

Yes the plan is that we'll be set up well to take advantage of this.
We probably won't do it initially, but as a followup.

@bcmills bcmills added the GoCommand label Oct 9, 2018

@josharian

This comment has been minimized.

Contributor

josharian commented Oct 9, 2018

Note that if we do much with #25999, the set of code changes that leave export data unchanged might shrink significantly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment