⚠️ CI failed ⚠️

[github-actions]

Workflow Run URL

[fjetter]

This is due to a detected regression

    raise Exception(
Exception:  Regressions detected 1: 
runtime = 'coiled-upstream-py3.9', name = 'test_q3[5 GB (parquet)]', category = 'benchmarks', last_three_duration [s] = (80.41926574707031, 78.423574924469, 79.43234944343567), duration_threshold [s] = 78.3170660380969 

Not entirely convinced this should count as a regression

[jrbourbeau]

xref #270

[ncclementi]

The average now shows a smaller difference, but few days ago, this poped up in regressions as
runtime = 'coiled-upstream-py3.9', name = 'test_q3[5 GB (parquet)]', category = 'benchmarks', last_three_duration [s] = (78.423574924469, 79.43234944343567, 78.1142942905426), duration_threshold [s] = 74.33889042616087

That is a bigger jump. I'm not sure it's not a regression.

As more runs get added the average will hide a potential regression. @jrbourbeau I agree we need to revisit the logic, but in this case, I can see a jump on right after Wednesday 17 and it is sustained. I noticed this in #271 .

Thoughts?

[ncclementi]

Closing in favor of #271