Getting "This run could not be processed" after successfully run the benchmark

[XSAM]

Hi, I am getting "This run could not be processed" error on the result page after a successful benchmark run.

Context:

• Commit: XSAM/opentelemetry-go@84f0877
• Benchmark action: https://github.com/XSAM/opentelemetry-go/actions/runs/21151656939/job/60828760257
• Benchmark result: https://codspeed.io/XSAM/opentelemetry-go/runs/696ea25d478d8b1c1dcd3897

[not-matthias]

Hey, thanks for reaching out. The report couldn't be created because there were too many benchmarks in a single upload to CodSpeed (as seen in the status of the CodSpeed action - https://github.com/XSAM/opentelemetry-go/runs/60829572412).

We recommend sharding your benchmarks into smaller uploads, see the documentation for more information. In your case, you have a bit over 1k benchmarks which is barely over our limit. So adding a single shard should already fix this issue.

I'll close this ticket for now, but feel free to ping me in case you have further questions.

[XSAM]

Hey @not-matthias, thanks for the rapid response.

I don't know how the codespeed action counts the number of benchmarks, but we have 898 benchmarks run on the CI, which shouldn't reach the 1k limit. You can count allocs/op from CI logs to verify. https://github.com/XSAM/opentelemetry-go/actions/runs/21151656939/job/60828760257

After reading shared benchmark docs, I think codespeed can process benchmarks more than 1k totally after sharding the benchmarks into smaller pieces. So, does the 1k limitation come from a single call of codespeed API? If so, could codespeed action split the benchmark into pieces that the codespeed API can accept and upload them separately within one github step?

I think it would be a better user experience, as it does not need users to change their existing testing methods. Not every language has sharding tools come in handy.