lowers time to host ondemand tablets #4581
Conversation
This change lowers the time it takes to host ondemand tablets by moving this functionality into TabletGroupWatcher. The client RPC thread processing the hosting request can now directly call a function in TGW that will immediately start on the work of hosting the tablets. Updated SplitMillionIT to request hosting of 200 tablets all at once instead of one by one. This was done by using a BatchScanner instead of lots of scanners.
|
In SplitMillionIT this changes lowered the time to read 100 tablets after a clone operation from 20 -30 secs to 1-2 secs. This is because the hosting operation is no longer impeded by the hosting operation. There may still be further room for improvement, need to dig in and understand where the code is spending its time after making this improvement. |
dlmarion
left a comment
dlmarion
left a comment
There was a problem hiding this comment.
This looks good. I also noticed that manageTablets could be called from different threads. I tried to parallelize the calls to it from the EventCoordinator. But, it didn't provide any real speed increase.
Working through this made me wonder what will happen when like 1000 clients processes all try to get some tablets hosted at around the same time. Will it work through it quickly, slowly, or get jammed up. Need to test this eventually. |
2 participants
This change lowers the time it takes to host ondemand tablets by moving this functionality into TabletGroupWatcher.
The client RPC thread processing the hosting request can now directly call a function in TGW that will immediately start on the work of hosting the tablets.
Updated SplitMillionIT to request hosting of 200 tablets all at once instead of one by one. This was done by using a BatchScanner instead of lots of scanners.