Conversation
lhoguin
left a comment
lhoguin
left a comment
There was a problem hiding this comment.
I don't know if the strategy works well and if so under which scenarios / use cases (but I've only ever looked at gen_batch_server, not used, so there's also that).
It does lack the option added to the README/documentation in any case.
It looks fine otherwise.
Introduce a `batch_size_growth` start option (default: `exponential`,
which preserves the existing doubling behaviour) that accepts
`{aimd, Step}` to switch to Additive Increase / Multiplicative Decrease
growth: the batch size grows by `Step` each time a full batch is
completed, and halves whenever the mailbox is found empty.
yes this is a fair question. I have run some benchmarks which suggest that aimd can't beat exponential for throughput but aimd is gentler on heap size growth (and thus GC perhaps). So it probably shouldn't be a default but good to have option to experiment for certain roles. Parameters: 100k messages, 4 KB payload, 3 ms base delay + 3 ms/MB variable, min_batch=32, max_batch=8192
|
|
Big difference there indeed. |
2 participants
Introduce a
batch_size_growthstart option (default:exponential, which preserves the existing doubling behaviour) that accepts{aimd, Step}to switch to Additive Increase / Multiplicative Decrease growth: the batch size grows byStepeach time a full batch is completed, and halves whenever the mailbox is found empty.Fixes #23