Currently we have (some) microbenchmarks, but measuring performance of our various readers (CSV, JSON, IPC, Parquet, ORC) over "real world" files would also be interesting and hopefully more illustrative of the use cases we actually care about. Such benchmarks may be expensive, though.
Ideally, we would do this in a variety of scenarios: in-memory (to focus on CPU optimization), on-disk (though such measurements would likely be extremely noisy?), and over the network (perhaps with something like Minio + Toxiproxy to try to have a consistent, reproducible setup) so that we can also judge the I/O characteristics of the readers.
Reporter: David Li / @lidavidm
Note: This issue was originally created as ARROW-16944. Please see the migration documentation for further details.
Currently we have (some) microbenchmarks, but measuring performance of our various readers (CSV, JSON, IPC, Parquet, ORC) over "real world" files would also be interesting and hopefully more illustrative of the use cases we actually care about. Such benchmarks may be expensive, though.
Ideally, we would do this in a variety of scenarios: in-memory (to focus on CPU optimization), on-disk (though such measurements would likely be extremely noisy?), and over the network (perhaps with something like Minio + Toxiproxy to try to have a consistent, reproducible setup) so that we can also judge the I/O characteristics of the readers.
Reporter: David Li / @lidavidm
Note: This issue was originally created as ARROW-16944. Please see the migration documentation for further details.