smarter ordering for queries that don't use zk first#935
Merged
ssalinas merged 3 commits intotask_searchfrom Mar 10, 2016
Merged
smarter ordering for queries that don't use zk first#935ssalinas merged 3 commits intotask_searchfrom
ssalinas merged 3 commits intotask_searchfrom
Conversation
Contributor
Author
|
Going to merge this into task_search since both are now in qa branch |
ssalinas
added a commit
that referenced
this pull request
Mar 10, 2016
smarter ordering for queries that don't use zk first
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Currently, when searching anything where
queryUsesZkFirstreturnsfalse, all results from zk are still initially in our list of returned items before fetching from history. So, after sorting, regardless of page, the zk results will likely be on the page and much of the persisted history will not be shown. In short, anything that hasn't been persisted yet will likely show up on every page of task search results.This PR implements two things:
taskHistoryQueryUsesZkFirstoption -> iftruethis would have every query for task history use zk items first. This will only require a single history query, and thus be much faster, with the caveat that anything not persisted will always appear before anything that has already persisted in the general order (but paging will be correct)taskHistoryQueryUsesZkFirstisfalsewe will make similar initial queries for items from zk and from persisted history. But, after ordering them, we will check that we have fetched all relevant items from the history. If we have not (i.e. some non-persisted items in the limitStart/limitCount range are beyond the range of all persisted items we have fetched), then we will fetch additional items to ensure correct ordering of all items, but at the added cost of additional queries needed.My overall thought on this was that we can at least give the user the choice of efficiency/speed vs ensuring correct ordering.
Only other thing I wasn't sure on was whether to make
taskHistoryQueryUsesZkFirstavailable on the query as well to give each individual user making a query the option as well as being abel to set the system-wide default./cc @wsorenson @tpetr @Calvinp