[FR] How to scale statistical time proportionally？

[0400H]

I want use the ->MinWarmUpTime(0.5)->MinTime(1) to sub whole pipeline time.

but How to get the avg time of Func in pipeline without any user-defined metric?

for (auto _ : state) {
    timer.start();
    // cuda kernel pipeline
    for (int i = 0; i < pipeline_size; i++) {
        Func();
    }
    timer.stop();
    state.SetIterationTime(timer.elapsed_ms() / 1000.0);
}
state.SetIteration(state.Iterations() * pipeline_size); // not support

#define REG_BENCHMARK(DATA, FUNC, T) \
    BENCHMARK(FUNC) \
        ->UseManualTime() \
        ->Unit(benchmark::kMillisecond) \
        ->MinWarmUpTime(0.5)->MinTime(1)

Maybe add the state.SetIteration mutable API make it work?

Or

?

[dmah42]

i assume that in your example timer is something that is measuring GPU time rather than CPU time. When you do this, and you use manual time, what is being reported and what do you want to be reported, exactly?

[0400H]

我假设在你的例子中 timer 是测量 GPU 时间而不是 CPU 时间的东西。当你这样做时，如果你使用手动时间，报告的是什么以及你希望报告的是什么，具体是什么？

@dmah42

1. Report the SetIterationTime(pipeline time / pipeline_size == one cuda kernel time).

2. The MinTime should reduce by the whole pipeline running time, rather than the SetIterationTime(one cuda kernel running time).

If the code is written as follows, then the benchmark test cases will run for pipeline_size times the MinTime.

for (auto _ : state) {
    timer.start();
    // cuda kernel pipeline
    for (int i = 0; i < pipeline_size; i++) {
        Func();
    }
    timer.stop();
    state.SetIterationTime(timer.elapsed_ms() / 1000.0 / pipeline_size);
}

I want to report state.SetIterationTime(timer.elapsed_ms() / 1000.0 / pipeline_size);, and not run pipeline_size multiplied by MinTime.

[LebedevRI]

Perhaps you want https://github.com/google/benchmark/blob/main/docs/user_guide.md#custom-counters ?

[0400H]

Perhaps you want https://github.com/google/benchmark/blob/main/docs/user_guide.md#custom-counters ?

@LebedevRI

Use custom-counters could do it.

But if reduce the Mintime by pipeline and get statistics with SetIterationTime, we can reuse many basic capabilities.

BENCHMARK(FUNC) \
   ->ComputeStatistics("max", [](const std::vector<double>& v) -> double { \
      ...
  }) -> \
  ComputeStatistics("min", [](const std::vector<double>& v) -> double { \
      ....
  })->ComputeStatistics("median", [](const std::vector<double>& v) -> double { \
       ...
  })->DisplayAggregatesOnly(false)

[LebedevRI]

I'm sorry, there seems to be a bit of a language barrier going on here. It is not clear what the actual question here is.

[0400H]

I'm sorry, there seems to be a bit of a language barrier going on here. It is not clear what the actual question here is.

Just find a simple method to scale times in statistics log.

state.SetIteration(state.Iterations() * pipeline_size); // not support

If added support for above API, will it be possible to scale the statistical time in log?

---------------------------------------------------------------------------------------------------------------------------------------------------
Benchmark                                                                                         Time             CPU   Iterations
---------------------------------------------------------------------------------------------------------------------------------------------------
xxxxxxx       <scale this data to 1/pipeline_size> ms        0.069 ms        100 

[LebedevRI]

The Walltime/CPU time columns represent the average time one iteration of the workloop took.
If you want to produce some stats, you want to use Custom User counters interface.