⚠️ CI failed ⚠️

[github-actions]

Workflow Run URL

[j-bennet]

Exception:  Regressions detected 9: 
runtime = 'coiled-latest-py3.9', name = 'test_join_big[1-tasks]', category = 'benchmarks', last_three_duration [s] = (661.3068561553955, 661.3068561553955, 664.7528762817383), duration_threshold [s] = 645.971376454088 

runtime = 'coiled-latest-py3.9', name = 'test_memory_efficient', category = 'benchmarks', last_three_duration [s] = (14.592434644699097, 14.592434644699097, 14.425075054168701), duration_threshold [s] = 14.400530576705933 

runtime = 'coiled-latest-py3.9', name = 'test_q8[0.5 GB (csv)-p2p]', category = 'benchmarks', last_three_duration [s] = (23.257978677749634, 23.257978677749634, 22.042306423187256), duration_threshold [s] = 21.1036319050846 

runtime = 'coiled-latest-py3.9', name = 'test_q8[0.5 GB (csv)-tasks]', category = 'benchmarks', last_three_duration [s] = (23.347721576690674, 23.347721576690674, 21.66010046005249), duration_threshold [s] = 20.842681142124796 

runtime = 'coiled-latest-py3.9', name = 'test_q8[0.5 GB (parquet)-p2p]', category = 'benchmarks', last_three_duration [s] = (38.65501809120178, 38.65501809120178, 37.90650749206543), duration_threshold [s] = 28.452760344775673 

runtime = 'coiled-latest-py3.9', name = 'test_q8[0.5 GB (parquet)-tasks]', category = 'benchmarks', last_three_duration [s] = (38.17823791503906, 38.17823791503906, 37.13248825073242), duration_threshold [s] = 27.552817533184218 

runtime = 'coiled-latest-py3.9', name = 'test_q8[5 GB (parquet)-p2p]', category = 'benchmarks', last_three_duration [s] = (190.76829409599304, 190.76829409599304, 179.65282678604126), duration_threshold [s] = 168.1402059770922 

runtime = 'coiled-latest-py3.9', name = 'test_q8[5 GB (parquet)-tasks]', category = 'benchmarks', last_three_duration [s] = (193.05822157859802, 193.05822157859802, 179.8635790348053), duration_threshold [s] = 179.30978843531042 

runtime = 'coiled-latest-py3.9', name = 'test_q9[0.5 GB (parquet)-tasks]', category = 'benchmarks', last_three_duration [s] = (4.537719488143921, 4.537719488143921, 4.642199754714966), duration_threshold [s] = 4.443351241496266 

This doesn't seem valid, in charts only 2 last points were higher than median, not 3.

test_memory_efficient:

test_join_big[1-tasks]:

test_q8[0.5 GB (csv)-p2p]:

[j-bennet]

For some reason, in the table of runtimes, the last 2 records are a dupe:

From https://benchmarks.coiled.io/details/coiled-latest-py3.9-benchmarks-test_h2o-test_q8[0.5%20GB%20(csv)-p2p].html, same cluster, same everything. What's up with that? How did these get into benchmarks db?

cc @crusaderky

[crusaderky]

Weird. Looks like something caused combine-dbs to pick up the same run twice
https://github.com/coiled/benchmarks/blob/3062113a0a33e69c049379817ac7de49ef76004f/.github/workflows/tests.yml#L162C11-L165