proposal: testing: profile flags to capture only during benchmark loop

[yonderblue]

Proposal Details

Would be great to have at least the option to restrict the flags like -cpuprofile to only capture during the b.Loop or equivalent, since that is the target of the benchmark. Setup code can often be expensive and make the relative percentages and other inspection harder.

[gabyhelp]

Related Issues

• testing: support pausing cpu profiling in benchmarks #8213 (closed)
• proposal: cmd/go: allow -cpuprofile in tests with multiple packages #71996
• cmd/pprof: Feature: profiling support for regular tests #23811 (closed)
• testing: show rusage statistics for benchmarks #24905
• cmd/compile: per-phase compiler benchmarking #16169 (closed)

_{(Emoji vote if this was helpful or unhelpful; more detailed feedback welcome in this discussion.)}

[nsrip-dd]

As a workaround, you could move the setup to its own function. Then you can use the -ignore and/or -focus flags of go tool pprof to filter down to what you want to see. For example, suppose you write your benchmark like this:

func BenchmarkFoo(b *testing.B) {
    data := benchmarkFooSetup()
    for b.Loop() {
        // use data
    }
}

Then go tool pprof -focus=BenchmarkFoo -ignore=benchmarkFooSetup ... will show the samples going through BenchmarkFoo but exclude samples going through benchmarkFooSetup.