Join GitHub today
GitHub is home to over 40 million developers working together to host and review code, manage projects, and build software together.Sign up
Avoid CPU spikes when abandoning skipped sequences #3823
Before abandoning skipped sequences, Sync Gateway issues a query as a final check for existence of the sequence in the bucket. When there are a large number of skipped sequences being abandoned at once, this is creating a spike in CPU that has the potential to impact other operations.
The skipped sequence query check needs to be treated as a low-priority background task that should not impact normal SG operations.
One thing that's come to mind in the past (particularly when looking at View ops) is that we do (did?) this individually per sequence number.
As I understand it, we do this checking on an interval, and then check for any that have been in the list for longer than
Say you're waiting for
Should also refactor to minimize locking on skippedSeqLock during CleanSkippedSequenceQueue, to avoid blocking normal DCP processing during clean.
Should also ensure there aren't any feedback loops associated with pushing to the skipped sequence queue during Clean. Expectation is that this should generate back pressure on the DCP feed overall (and so TimeReceived isn't set), but should validate.