-
Notifications
You must be signed in to change notification settings - Fork 18.5k
Description
In reviewing the testing package, I noticed that it currently contains some tests that call testing.Benchmark, where the benchmark function calls Error, Fatal, or SkipNow.
(https://cs.opensource.google/go/go/+/master:src/testing/benchmark_test.go;l=94-127;drc=cecaa25bd31622f5ae0df7fd5c8c7f02afde8715)
Those tests do not currently emit any output when run, and as far as I can tell the testing.BenchmarkResult type does not provide a way for them to do so. As a result, I believe that failures in calls to testing.Benchmark in general tend to go undetected.
~/go/src$ go1.21rc2 test testing -run='TestRunParallel.*' -v -count=1
=== RUN TestRunParallel
--- PASS: TestRunParallel (0.16s)
=== RUN TestRunParallelFail
--- PASS: TestRunParallelFail (0.00s)
=== RUN TestRunParallelFatal
--- PASS: TestRunParallelFatal (0.00s)
=== RUN TestRunParallelSkipNow
--- PASS: TestRunParallelSkipNow (0.00s)
PASS
ok testing 0.178s
To allow users of testing.Benchmark to write more robust benchmarks, I propose the addition of the following fields to the BenchmarkResult struct:
type BenchmarkResult struct {
…
Log []byte // The output produced by calls to b.Log, b.Error, b.Skip, and similar during the benchmark run.
Failed bool // The value returned by b.Failed at the end of the run.
Skipped bool // The value returned by b.Skipped at the end of the run.
}Since these are new fields, I believe this change is strictly backward-compatible.
Metadata
Metadata
Assignees
Labels
Type
Projects
Status