Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
An inactive shard is activated by triggered synced flush #13802
When a shard becomes inactive we trigger a sync flush in order to speed up future recoveries. The sync flush causes a new translog generation to be made, which in turn confuses the IndexingMemoryController making it think that the shard is active. If no documents come along in the next 5m, the shard is made inactive again , triggering a sync flush and so forth.
To avoid this, the IndexingMemoryController is changed to ignore empty translogs when checking if a shard became active. This comes with the price of potentially missing indexing operations which are followed by a flush. This is acceptable as if no more index operation come in, it's OK to leave the shard inactive.
A new unit test is introduced and comparable integration tests are removed.
I think we need to push this all the way to 1.7.3 ...
changed the title from
An inactive shard is temporarily activated by triggered synced flush
An inactive shard is activated by triggered synced flush
Sep 25, 2015
This was a great catch @bleskes, and the cutover from IT to unit test is wonderful.
I left a bunch of small comments, but otherwise LGTM.
I still find that massive if statement in
I was thinking about this some more (morning run, he he) - I think we should fold all the decisions and state about being active or not into the indexshard. Then it will be easy to reset on every index operation (immediately). We also already have the notion of an inactivity listener - we can extend it to have an active event as well. We can also add an checkIfInactive method on IndexShard which check if there was any indexing ops (we already have stats) since the last time it was checked. To avoid making IndexShard more complex, we can push this functionality ShardIndexingService - which is already in charge of stats.
+1, this would be cleaner. E.g. I don't like how
I can try to tackle this.
Or, why not absorb
I personally don't mind folding it into indexshard though having a dedicate stats/active component has benefit (clearer testing suite) . I know Simon has an opinion on this.
On 27 sep. 2015 4:39 PM +0200, Michael McCandlessnotifications@github.com, wrote: