Join GitHub today
GitHub is home to over 31 million developers working together to host and review code, manage projects, and build software together.Sign up
When shard becomes active again, immediately increase its indexing buffer #13918
Spinoff from #13802.
Today, when an index goes from inactive to active, because indexing ops suddenly arrive to an inactive index, we (IndexingMemoryController) take up to 30 seconds to notice this and then increase the indexing buffer from the tiny idle 512KB to "its fair share". This is somewhat dangerous because it could write many, many segments in those 30 seconds...
This PR changes that so the inactive -> active transition instead causes us to immediately re-visit the indexing buffer for all shards.
It also shifts responsibility of tracking active/inactive into
I love how this turned out, especially around moving away from the translog checks. I left some suggestions here and there. I also think we should move the controller to work on IndexShard objects rather than shard ids (sorry for the scope creep - can be done in a different change). At some point I got worried that a new shard will not be seen as "added" if the original shard was destroyed and a new one created. I think it's OK now (because we only use current state when allocating memory) but it's more correct to use physical instances.
But I thought we used