wire: caching fails to make tests run instantaneously

[ijt]

issactrotts-macbookpro:go-cloud issactrotts$ time vgo test ./wire/internal/wire/...

ok      github.com/google/go-cloud/wire/internal/wire   35.005s

real    2m29.450s
user    1m45.080s
sys     1m41.907s
issactrotts-macbookpro:go-cloud issactrotts$ time vgo test ./wire/internal/wire/...
ok      github.com/google/go-cloud/wire/internal/wire   (cached)

real    0m37.605s
user    0m13.383s
sys     0m24.832s

It should take no more than a second to run the tests in the cached case, but really it should be on the order of 100 milliseconds or less.

The problem appears to be that there are many subtests that are dynamically generated. The dynamic generation takes a while, and apparently has to happen regardless of whether there are cache hits for all the tests.

An alternative that would probably allow caching to operate normally would be to generate the test code.

[ijt]

Looking at the source code for the go test command, the computeTestInputsID function apparently computes the hash of every file that got opened during the first test run. That would explain why the time to run the test varies with the number of subtests being run (taking a slice) even for a cache hit. So it's possible that the overhead for computing all these hashes is making the tests slow on cache hits. If so, it might be possible to speed up that case by putting all the test data into a zip file.

[ijt]

Update: I tried putting the testdata directory into a zip file and updating the test to work with that, but got no significant speedup on cache hits. It's puzzling.

[ijt]

@zombiezen is saying the slowdown could be the result of the tests opening a bunch of Go standard library packages for type checking. go test would have to check the hashes of all those files to see if it's a cache hit, and that could take a while.

[ijt]

This is not a wire issue. It's an issue in go test. See golang/go#26726.