Benchmark results

[phofl]

I ran the benchmarks yesterday. Many of them are still failing because of #159

Link to the results: https://github.com/coiled/benchmarks/actions/runs/5313713615

[mrocklin]

I'm not sure exactly how to interpret these results, but in general it seems like modest improvements at best, is that correct?

[phofl]

Yep that's my interpretation as well, but since this surprised me (kind of), so I looked a bit deeper:

• test_csv: calls persist before grouping and selecting single columns -> won't see anything here
• test_filter -> odd, have to investigate
• test_q*: already restrict the DataFrame to the columns that are needed, only advantage we can expect is the push up to read_parquet
• test_join: Probably related to P2P not implemented yet?
• setitem not pushing optimizations through yet looks like a bottleneck in some benchmarks.

So

1. I think the benchmarks don't capture the things we already have very well
2. We should probably add more _simplify_up steps in Blockwise ops

[rjzamora]

Probably related to P2P not implemented yet?

Just a note that I'll probably push on this today.

[mrocklin]

Woo!

[mrocklin]

It would be interesting to see a branch of the benchmarks that were simpler / dumber. This might include things like the following:

1. Not specifying columns at the outset
2. Not persisting data in memory