You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
An unordered table scan in Rethink is a very slow operation because we do a b-tree traversal without any awareness of disk layout. This is a problem because database dumps are extremely slow on rotational storage because of seeks.
We can improve this by making unordered scans disk-layout aware (this is obviously hard), or by somehow signaling the serializer to allow returning some blocks in a different order.
Leaving this in backlog.
The text was updated successfully, but these errors were encountered:
This isn't actually tough from a traversal perspective. This is basically already what backfilling does. We just need to make a way to hook that up to a ReQL API. So it's still kind of tough, but not as tough as OP makes it out to be.
@jdoliner Can you explain a bit more how backfilling does that? Does it bypass the serializer interface or is there another trick which makes disk access more sequential (on average)?
I imagine that the solution to this would essentially be some kind of smart rethinkdb extract, probably on a per-extent level.
Well, actually I'm overselling this quite a bit. It's just that backfilling works correctly with blocks returned in whatever order the serializer finds convenient. So that would probably already be a bit faster, and with some more optimization could request big batches of blocks and try to read them sequentially. It seems like we can only do so much here without changing the layout on disk though.
An unordered table scan in Rethink is a very slow operation because we do a b-tree traversal without any awareness of disk layout. This is a problem because database dumps are extremely slow on rotational storage because of seeks.
We can improve this by making unordered scans disk-layout aware (this is obviously hard), or by somehow signaling the serializer to allow returning some blocks in a different order.
Leaving this in backlog.
The text was updated successfully, but these errors were encountered: