Ensure replicator _active_tasks entry reports recent pending changes #623
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
davisp
requested changes
Jun 28, 2017
| ok; | ||
| maybe_report_last_sequence(Parent, LastSeqs, Ts) -> | ||
| {last_seq, Seq} = lists:last(LastSeqs), | ||
| Msg = {report_seq_done, {Ts, Seq}, couch_replicator_stats:new()}, |
There was a problem hiding this comment.
Will this work if we don't call {report_Seq, {Ts, Seq}} first?
There was a problem hiding this comment.
Should be handled by this case:
{NewThroughSeq0, NewSeqsInProgress} = case SeqsInProgress of
[] ->
{Seq, []};
There was a problem hiding this comment.
But I'll switch the order. I think it would make more sense.
|
|
||
|
|
||
| maybe_process_changes(_Parent, [] = _Changes, _From, _Ts) -> | ||
| ok; |
There was a problem hiding this comment.
Pretty sure this will dead lock since you're sometimes not sending a message back to the From client.
nickva
force-pushed
the
fix-replicator-progress-reporting-2
branch
from
254ce3d
to
bc1ca8a
Compare
davisp
requested changes
Jun 28, 2017
|
|
||
|
|
||
| maybe_report_in_progress(_Parent, [] = _Changes, _From, _Ts) -> | ||
| nil; |
There was a problem hiding this comment.
ReportSeq could end up being nil here, will that not break a worker? Or are you relying on the empty clause match?
There was a problem hiding this comment.
It relies on {changes, ChangesManager, [], nil} -> clause match in the worker
nickva
force-pushed
the
fix-replicator-progress-reporting-2
branch
2 times, most recently
from
b1f2307
to
780baef
Compare
davisp
requested changes
Jun 28, 2017
| fun(#doc_info{}) -> | ||
| true; | ||
| ({last_seq, _Seq}) -> | ||
| false |
| ({last_seq, _Seq}) -> | ||
| false | ||
| end, ChangesOrLastSeqs), | ||
| maybe_report_in_progress(Parent, Changes, From, Ts), |
There was a problem hiding this comment.
There's no maybe here any more. You need to just hoist gen_server:cast(Parent, {report_Seq, ReportSeq}) out now.
nickva
force-pushed
the
fix-replicator-progress-reporting-2
branch
2 times, most recently
from
52b84e9
to
f3dfa73
Compare
…value Previously there was a race between reporting the source update sequence between the the workers and the changes readers. Each one used separate incrementing timestamp sequences. In some cases that lead to pending changes being stuck. For example, if changes reader reported the highest sequence with timestamp 10000, then later workers reported it with sequences 5000, 5001, 5002, then all those reports would be ignored and users would see an always lagging pending changes value reported with timestamp 1000. The fix is to thread the last_sequence update through the changes queue to the changes manager, so only its timestamp sequence will be used. This removes the race condition.
nickva
force-pushed
the
fix-replicator-progress-reporting-2
branch
from
f3dfa73
to
719436a
Compare
lag-linaro
pushed a commit
to lag-linaro/couchdb
that referenced
this pull request
Oct 25, 2018
Prepare release 2.6.4
nickva
pushed a commit
to nickva/couchdb
that referenced
this pull request
Sep 7, 2022
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Previously there was a race between reporting the source update sequence
between the the workers and the changes readers. Each one used separate
incrementing timestamp sequences.
In some cases that lead to pending changes being stuck. For example, if changes
reader reported the highest sequence with timestamp 10000, then later workers
reported it with sequences 5000, 5001, 5002, then all those reports would be
ignored and users would see an always lagging pending changes value reported
with timestamp 1000.
The fix is to thread the last_sequence update through the changes queue to
the changes manager, so only its timestamp sequence will be used. This removes
the race condition.