cmd/dist: no obvious way to disable test caching

[mundaym]

There doesn't appear to be a way to disable the new test caching behaviour when running go tool dist test. This is a bit inconvenient when trying to recreate/resolve intermittent issues.

What version of Go are you using (go version)?

go version devel +7781fed24e Thu Nov 16 02:24:37 2017 +0000 linux/s390x

Does this issue reproduce with the latest release?

No, 1.9.x does not have test caching.

What did you do?

go tool dist test

What did you expect to see?

##### Testing packages.
ok  	archive/tar	0.033s
ok  	archive/zip	0.749s
ok  	bufio	0.126s
ok  	bytes	0.934s
ok  	compress/bzip2	0.068s
ok  	compress/flate	0.833s
ok  	compress/gzip	0.010s
ok  	compress/lzw	0.020s
ok  	compress/zlib	0.100s
ok  	container/heap	0.035s
ok  	container/list	0.015s
ok  	container/ring	0.014s
ok  	context	1.008s
...

What did you see instead?

##### Testing packages.
ok  	archive/tar	(cached)
ok  	archive/zip	(cached)
ok  	bufio	(cached)
ok  	bytes	(cached)
ok  	compress/bzip2	(cached)
ok  	compress/flate	(cached)
ok  	compress/gzip	(cached)
ok  	compress/lzw	(cached)
ok  	compress/zlib	(cached)
ok  	container/heap	(cached)
ok  	container/list	(cached)
ok  	container/ring	(cached)
ok  	context	(cached)
...

The output of go tool dist test -h is currently:

usage: go tool dist test [options]
  -banner string
    	banner prefix; blank means no section banners (default "##### ")
  -compile-only
    	compile tests, but don't run them. This is for some builders. Not all dist tests respect this flag, but most do.
  -k	keep going even when error occurred
  -list
    	list available tests
  -no-rebuild
    	overrides -rebuild (historical dreg)
  -race
    	run in race builder mode (different set of tests)
  -rebuild
    	rebuild everything first
  -run string
    	run only those tests matching the regular expression; empty means to run all. Special exception: if the string begins with '!', the match is inverted.
  -v	verbosity

The -rebuild flag does not affect the caching.

[mundaym]

/cc @rsc

[laboger]

Did you try GOCACHE=off.

[mundaym]

Did you try GOCACHE=off.

Ah I wasn't aware of that. Thanks, yes that does the trick.

If GOCACHE is going to be the general solution to this it might be nice to add to the dist help output.

[rsc]

Why are you running go tool dist test at all instead of go test? Just curious. Normally go tool dist test is basically only for all.bash/run.bash/builders.

[mundaym]

Normally I just use all.bash and go test when I'm testing. I sometimes use go tool dist test when I want to run the cgo and link mode tests without rebuilding everything. I could use run.bash instead, I just use go tool ... out of habit.

In this particular case I was trying to recreate a 'too many open files' failure on the builder and was changing ulimit -n in between runs to see what failed.

I guess the only other reason to use go tool dist test directly rather than run.bash would be to avoid the -rebuild flag. Useful when testing an install of Go without write access.

Since run.bash is just a thin wrapper around go tool dist test -rebuild it does have the same issue. I sometimes (rarely these days) need to run the tests a few times to be confident I've fixed an intermittent issue. In that case I don't want to see cached results.

GOCACHE=off is fine for me. Happy for this issue to be closed if you think this an obscure use case.

[laboger]

I think there needs to be some clarification of the -rebuild option. Because IMO it does not behave the way it used to, i.e., it doesn't rebuild everything if the cache is active.

…

On Tue, Nov 21, 2017 at 4:41 AM, Michael Munday ***@***.***> wrote: Normally I just use all.bash and go test when I'm testing. I sometimes use go tool dist test when I want to run the cgo and link mode tests without rebuilding everything. I could use run.bash instead, I just use go tool ... out of habit. In this particular case I was trying to recreate a 'too many open files' failure on the builder and was changing ulimit -n in between runs to see what failed. I guess the only other reason to use go tool dist test directly rather than run.bash would be to avoid the -rebuild flag. Useful when testing an install of Go without write access. Since run.bash is just a thin wrapper around go tool dist test -rebuild it does have the same issue. I sometimes (rarely these days) need to run the tests a few times to be confident I've fixed an intermittent issue. In that case I don't want to see cached results. GOCACHE=off is fine for me. Happy for this issue to be closed if you think this an obscure use case. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#22758 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AI_wjDQpHvqikxA1ptWHYy9aAKYAl3TVks5s4qjpgaJpZM4QgaPN> .

[rsc]

@laboger, I believe that using -rebuild does rebuild everything even when the cache is active. cmd/dist says:

if t.rebuild {
	t.out("Building packages and commands.")
	// Force rebuild the whole toolchain.
	goInstall("go", append([]string{"-a", "-i"}, toolchain...)...)
}

The -a should be forcing a rebuild of everything. If not, that's a bug in -a.

[rsc]

@mundaym, thanks for elaborating. I will send a CL that uses -count=1 on all go test invocations from cmd/dist, so that those results are never cached.

[gopherbot]

Change https://golang.org/cl/80735 mentions this issue: cmd/dist: disable test caching during run.bash

[laboger]

Sorry I misunderstood Michael's comment above where he said "The -rebuild flag does not affect the caching". I have had no personal experience of the -rebuild option not working as expected.

…

On Wed, Nov 29, 2017 at 11:20 AM, Russ Cox ***@***.***> wrote: @laboger <https://github.com/laboger>, I believe that using -rebuild *does* rebuild everything even when the cache is active. cmd/dist says: if t.rebuild { t.out("Building packages and commands.") // Force rebuild the whole toolchain. goInstall("go", append([]string{"-a", "-i"}, toolchain...)...) } The -a should be forcing a rebuild of everything. If not, that's a bug in -a. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#22758 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AI_wjJLPfP2FMwBcKI-iBjrQRgCPXj5fks5s7ZJOgaJpZM4QgaPN> .