arrow2's estimated_size_bytes performance issues

[teh-cmc]

The problem is so severe that it is pretty much impossible to use it anywhere in the store without everything slowing down to a crawl, which makes implementing statistics really, really painful.

Even for incremental measurements this is way too slow, orders of magnitudes slower than everything else on the write path.

See #1743 for detailed benchmarks.

---

If we cannot optimize it any further, my proposal is to move the problem upstream: compute byte sizes within the batching system, therefore:

1. distributing the load to the clients
2. making its cost irrelevant, since batching happens on a separate thread

[emilk]

Can't we fix this in arrow2? It sounds like a performance bug.

[teh-cmc]

Can't we fix this in arrow2? It sounds like a performance bug.

Yep, I want to investigate this tomorrow after a good night's sleep.
Sounds like a good opportunity to become really intimate with arrow2's internals too, which I've been meaning to do for a while anyhow.

[teh-cmc]

Long-winded explanation here, but the TL;DR is that, as far as I can tell, A) there is nothing wrong with how estimated_bytes_size is implemented and B) performance is getting absolutely wrecked by nested virtual calls.

Due to the way arrow2 is structured, the number of virtual calls necessary to execute Array::clone or estimated_bytes_size scales linearly with the number of fields in a type, while at the same time the performance of these methods decreases pretty much linearly with the number of virtual calls involved.
I.e. cloning or estimating the size of a Point3D will be much slower than doing the same thing on an InstanceKey.

At least in the short-term I think the solution stays the same: get these operations out of the fast path, cache the results in DataCells, use DataCells everywhere, avoid working with raw arrow arrays as much as possible.