Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Channel cache prepend can fail for some seq, validFrom combinations #3872

Closed
adamcfraser opened this issue Dec 18, 2018 · 1 comment

Comments

@adamcfraser
Copy link
Contributor

commented Dec 18, 2018

There are scenarios where a view query incorrectly fails to prepend its results to the channel cache. These don't appear to be related to the prepend changes in #3475 - these scenarios are also failing to prepend the cache in 1.5.1.

Scenario 1

  • Channel cache is empty
  • No documents have been written to the channel since change cache initialization (SG startup)
  • Channel cache is initialized via changes request w/ since=0, limit !=0

Cause: Query results aren't prepended to the cache because the high sequence value in the query results is lower than the cache validFrom, and the non-zero limit marks the request as non-open-ended.

Scenario 2

  • Change cache is initialized via DCP at sequence n
  • Channel cache is initialized via DCP with entry having sequence m > n
  • Query issued with since value < n

Cause: Query results aren't prepended to the cache because sequence m isn't included in query results, and so there's no overlap.

Scenario 3

  • Channel has non-continguous values between 1 and n
  • Channel cache is prepended via changes w/ since=m, 0<m<n, where sequence m isn't in the channel. (cache[0].sequence = m+2, say)
  • Issue a second changes w/ since=0. Queries for 0..m, but fails to prepend.

Cause: Second changes query doesn't prepend to the cache because there's no overlap (sequence m isn't in the channel, cache[0] isn't in the result set

@adamcfraser adamcfraser self-assigned this Dec 19, 2018

@adamcfraser adamcfraser added this to the Iridium milestone Dec 19, 2018

@adamcfraser

This comment has been minimized.

Copy link
Contributor Author

commented Dec 19, 2018

In general, validFrom/validTo evaluations being done during query prepend shouldn't be based on either the sequences in the cache or the sequences returned by the query - it should be based on the current cache validFrom and the query bounds.

@adamcfraser adamcfraser added backlog ready and removed backlog labels Jan 7, 2019

@adamcfraser adamcfraser added in progress review and removed ready labels Jan 9, 2019

adamcfraser added a commit that referenced this issue Apr 6, 2019

Improve cache validFrom handling
Avoids scenarios where query range and cache validFrom result in query results not being prepended to the cache, or redundant queries being issued because cache validFrom isn't associated with a sequence in the cache.

Backport of #3872 to 2.1.3.1

adamcfraser added a commit that referenced this issue Apr 6, 2019

Improve cache validFrom handling (#4056)
Avoids scenarios where query range and cache validFrom result in query results not being prepended to the cache, or redundant queries being issued because cache validFrom isn't associated with a sequence in the cache.

Backport of #3872 to 2.1.3.1
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
1 participant
You can’t perform that action at this time.