Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
Faster Up-scaling Based on Consumed Read/Write #227
I just started using Dynamic DynamoDB, and it's grand! Already I see a lot of benefit from it.
This is not an issue, but rather a feature. I have very steep, unpredictable consumed read/write activity after periods of not much activity at all - such that I've encountered a number of times now where my table's provisioned reads is at, say, 50, and consumption jumps up to, say, 250. At present, Dymamic DymamoDB will use my increase-reads-with 100% to increase the provisioned reads to 100, but then has to wait another 5 minute cycle to see that consumption is still above the threshold and increase by another 100%, and so on.
My question is, since Amazon no longer limits increases to 100% of the current provisioned amount, can Dynamic DynamoDB use the consumed percentage to modify the increase necessary to place the consumption under the upper threshold in a single step?
Using the numbers of my earlier example, if my table has 50 provisioned, and my consumption jumps to 250, Dynamic DynamoDB knows that consumption is at 500% and an increase of 100% won't cover it. One way to solve this is to loop through the possible increases until the right one to implement is found - 50 increase by 100% = 100; no; 100 increase by 100% = 200; no; 200 increase by 100% = 400; yes!
With a feature like this, I would be able to keep my tables' minimum provisioned amounts very low for the periods of time they're not being used much, but then have them scale up to the correct amount in a single cycle when steep-sided plateaus of activity start.
Again, absolutely brilliant work. Thank you for your time.
referenced this issue
Dec 20, 2014
Really awesome! Thanks for the good news. Can't wait to get back from
Bumping this issue because I think it needs to be documented. I ran into weird behavior when I was implementing the granular upscaling, where I had the following settings:
If I saw something like 400% utilization I was seeing a ridiculously high increase in throughput that I could not understand until I saw this code. Had I have known that I would have had a different increase at 100%