Efficient batch operations

[amotl]

Dear Josh,

I recognize that we are currently using the crate-python driver together with the BatchSizeAutomator in order to balance throughput saturation vs. overload situations.

https://github.com/crate/ts-data-generator/blob/9c64608d2cd9618e9d3b8d9be387c199b1fe4a4e/requirements.txt#L3

https://github.com/crate/ts-data-generator/blob/67330181e262d069f09e27bc64eb18951384b2f4/src/data_generator/data_generator.py#L60-L62

On this matter, I would also like to point out asyncpg's upcoming MagicStack/asyncpg#295, which improves executemany() to batch args for performance. It will probably be available with asyncpg>=0.22.0. Thanks to @mfussenegger for sharing this.

With kind regards,
Andreas.