x/build: detect leftover temporary files?

[minux]

Could the builders detect that there isn't any leftover temporary
files in $TMPDIR after testing each repo? (We need to whitelist
a few, for example gopath-api-*)

Ideally this should be treated a build failure and reported back
to the CL that triggered this.

Each time I logged into the netbsd-386 builder, I find a lot
temporary files in /tmp. However, as I don't know which build
generated them, I have to manually check if the bug is still
there.

[bradfitz]

Yes, we could do something like this. Maybe we add a new flag to the buildlet exec handler like "?detect-tmp-leak=1" and if so, it scans tmp before & after the run, and only complains if the command exited with success and any files in $TMPDIR (or equivalent) are new from the beginning of the run.

/cc @adg

[minux]

It seems if we only concerned about Go code leaking temporary files, we just need to create a new directory for each sub-repo, and set that as $TMPDIR (%TMP% on Windows), so that we don't need to record files in global $TMPDIR before each run (and we can also run multiple tests in parallel.)

[bradfitz]

SGTM

[minux]

The added benefit is that if we remove the created tmp directory after each run, we also solve the leftover files accumulation problem.

[bradfitz]

As of https://go-review.googlesource.com/c/144637 (for #27182 and #28041) we at least now set TMPDIR to $WORKDIR/tmp (a fresh directory per run).

So we could make the coordinator scan that directory after each dist test run so we fail as quickly as possible and figure out which dist test was responsible.

[bradfitz]

@toothrot, this might be a good warm-up coordinator job. It should just involve adding to:

func (st *buildStatus) runTestsOnBuildlet(bc *buildlet.Client, tis []*testItem, goroot, gopath string) {

Somewhere after it runs a test successfully, then run buildlet.Client's list files method on $WORKDIR/tmp and make sure it's empty.