Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Loading…

make mongos batch inserts into larger chunks #912

Closed
leifwalsh opened this Issue · 1 comment

1 participant

@leifwalsh

In 1.4.0, the mongos insert batch grouping algorithm takes intervals of consecutive documents from the insert batch that all have the same shard key, and inserts each of those intervals separately. With a hashed shard key (or a write batch with high entropy in the shard key), these intervals are tiny in expectation, compared to the total size of the insert batch.

Instead, we should divide the batch fully into separate groups, and then insert those groups, so that the batches sent to each shard are all bigger and amortize away more of the network/locking/transaction overhead.

For extra credit, we could parallelize sending the batches to their respective mongods.

@leifwalsh leifwalsh was assigned
@leifwalsh leifwalsh modified the milestone: 1.5.0, 1.4.1
@leifwalsh

this is done by 2.6 sharding code and will be resolved by merging that code

@leifwalsh leifwalsh closed this
@leifwalsh leifwalsh was unassigned by joelaflamme
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.