New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ensure replicator _active_tasks entry reports recent pending changes #623

Merged
merged 1 commit into from Jun 28, 2017

Conversation

Projects
None yet
2 participants
@nickva
Contributor

nickva commented Jun 28, 2017

Previously there was a race between reporting the source update sequence
between the the workers and the changes readers. Each one used separate
incrementing timestamp sequences.

In some cases that lead to pending changes being stuck. For example, if changes
reader reported the highest sequence with timestamp 10000, then later workers
reported it with sequences 5000, 5001, 5002, then all those reports would be
ignored and users would see an always lagging pending changes value reported
with timestamp 1000.

The fix is to thread the last_sequence update through the changes queue to
the changes manager, so only its timestamp sequence will be used. This removes
the race condition.

@davisp

Looks really good. I definitely like that use of lists:partition/2. Though I think for sanity we should look at routing the last_seq to the worker so that we're changing this logic as little as possible.

@davisp

Still seems like it'd be easier to find the ReportSeq from the list, strip out last_seq entries, and then in the ignored empty clause just send a report_seq_done message than your two extra maybe functions.

@davisp

Minor style issue and need to remove that maybe function but pretty close otherwise.

Ensure replicator _active_tasks entry reports recent pending changes …
…value

Previously there was a race between reporting the source update  sequence
between the the workers and the changes readers. Each one used separate
incrementing timestamp sequences.

In some cases that lead to pending changes being stuck. For example, if changes
reader reported the highest sequence with timestamp 10000, then later workers
reported it with sequences 5000, 5001, 5002, then all those reports would be
ignored and users would see an always lagging pending changes value reported
with timestamp 1000.

The fix is to thread the last_sequence update through the changes queue to
the changes manager, so only its timestamp sequence will be used. This removes
the race condition.
@davisp

davisp approved these changes Jun 28, 2017

+1

@nickva nickva merged commit 571a2fc into apache:master Jun 28, 2017

1 check was pending

continuous-integration/travis-ci/pr The Travis CI build is in progress
Details

@nickva nickva deleted the cloudant:fix-replicator-progress-reporting-2 branch Jun 28, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment