Add new ConsumerSettings::enableOptimisticResume setting (true by default) to allow users to disable the "optimistic resume" optimisation && Write benchmarks where this optimisation is disabled

[guizmaii]

Benchmark results

Before

zioKafkaBench/Jmh/run -wi 10 -i 10 -r 1 -w 1 -t 1 -f 3 -foe true .*ConsumersComparisonBenchmark*
...
[info] Benchmark                                        Mode  Cnt     Score     Error  Units
[info] ConsumersComparisonBenchmark.kafkaClients        avgt   30   645.361 ±   3.004  ms/op
[info] ConsumersComparisonBenchmark.manualKafkaClients  avgt   30   641.042 ±   2.877  ms/op
[info] ConsumersComparisonBenchmark.manualZioKafka      avgt   30  2651.134 ± 100.297  ms/op
[info] ConsumersComparisonBenchmark.zioKafka            avgt   30   739.566 ±  21.019  ms/op

After

zioKafkaBench/Jmh/run -wi 10 -i 10 -r 1 -w 1 -t 1 -f 3 -foe true .*comparison.*
...
[info] Result "zio.kafka.bench.comparison.KafkaClientBenchmarks.kafkaClients":
[info]   641.438 ±(99.9%) 3.349 ms/op [Average]
[info]   (min, avg, max) = (624.941, 641.438, 650.032), stdev = 5.012
[info]   CI (99.9%): [638.090, 644.787] (assumes normal distribution)
...
[info] Result "zio.kafka.bench.comparison.KafkaClientBenchmarks.manualKafkaClients":
[info]   642.514 ±(99.9%) 3.090 ms/op [Average]
[info]   (min, avg, max) = (632.069, 642.514, 656.932), stdev = 4.625
[info]   CI (99.9%): [639.424, 645.605] (assumes normal distribution)
...
[info] Result "zio.kafka.bench.comparison.ZioKafkaBenchmarks.manualZioKafka":
[info]   2605.392 ±(99.9%) 229.199 ms/op [Average]
[info]   (min, avg, max) = (1584.064, 2605.392, 2818.016), stdev = 343.055
[info]   CI (99.9%): [2376.193, 2834.592] (assumes normal distribution)
...
[info] Result "zio.kafka.bench.comparison.ZioKafkaBenchmarks.zioKafka":
[info]   734.987 ±(99.9%) 22.830 ms/op [Average]
[info]   (min, avg, max) = (683.740, 734.987, 826.528), stdev = 34.170
[info]   CI (99.9%): [712.158, 757.817] (assumes normal distribution)
...
[info] Result "zio.kafka.bench.comparison.ZioKafkaNoOptimisticResumeBenchmarks.manualZioKafka":
[info]   819.937 ±(99.9%) 120.082 ms/op [Average]
[info]   (min, avg, max) = (683.373, 819.937, 1243.105), stdev = 179.733
[info]   CI (99.9%): [699.855, 940.019] (assumes normal distribution)
...
[info] Result "zio.kafka.bench.comparison.ZioKafkaNoOptimisticResumeBenchmarks.zioKafka":
[info]   1311.588 ±(99.9%) 212.646 ms/op [Average]
[info]   (min, avg, max) = (755.915, 1311.588, 2329.325), stdev = 318.279
[info]   CI (99.9%): [1098.942, 1524.234] (assumes normal distribution)
...
[info] Benchmark                                            Mode  Cnt     Score     Error  Units
[info] KafkaClientBenchmarks.kafkaClients                   avgt   30   641.438 ±   3.349  ms/op
[info] KafkaClientBenchmarks.manualKafkaClients             avgt   30   642.514 ±   3.090  ms/op
[info] ZioKafkaBenchmarks.manualZioKafka                    avgt   30  2605.392 ± 229.199  ms/op
[info] ZioKafkaBenchmarks.zioKafka                          avgt   30   734.987 ±  22.830  ms/op
[info] ZioKafkaNoOptimisticResumeBenchmarks.manualZioKafka  avgt   30   819.937 ± 120.082  ms/op
[info] ZioKafkaNoOptimisticResumeBenchmarks.zioKafka        avgt   30  1311.588 ± 212.646  ms/op

Observations

Interestingly, the optimistic resume optimisation seems to be very detrimental to the manual consumption performances

[erikvanoosten]

There is no reason to inline Chunk.foreach and I prefer positive wording for the feature name. Otherwise this PR LGTM.

BTW, I didn't check the benchmark related code in detail.

[guizmaii]

BTW, I didn't check the benchmark related code in detail.

I just re-organised and copy/pasted the already existing benchmarks.
It's the only way to be able to have different sets of settings if we want to keep the instantiation of the Consumer outside of the benchmarked code (which I absolutely want)

[erikvanoosten]

I am still wondering if we should call the setting optimisticResume instead of enableOptimisticResume 🤷

[guizmaii]

I am still wondering if we should call the setting optimisticResume instead of enableOptimisticResume 🤷

enableOptimisticResume is more explicit

[svroonland]

Good for users to be able to toggle this feature to see the effect on performance