-
Notifications
You must be signed in to change notification settings - Fork 13.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix cache for multiple time comparisons #5828
Conversation
Codecov Report
@@ Coverage Diff @@
## master #5828 +/- ##
==========================================
+ Coverage 63.7% 63.73% +0.02%
==========================================
Files 366 368 +2
Lines 23143 23220 +77
Branches 2599 2600 +1
==========================================
+ Hits 14744 14800 +56
- Misses 8384 8405 +21
Partials 15 15
Continue to review full report at Codecov.
|
@@ -327,7 +327,7 @@ def get_json(self): | |||
self.get_payload(), | |||
default=utils.json_int_dttm_ser, ignore_nan=True) | |||
|
|||
def cache_key(self, query_obj): | |||
def cache_key(self, query_obj, **extra): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It'd be nice to add something in the docstring about what extra is used for currently, and could be used for in the future.
Minor comment on improving the |
* Fix cache for multiple time comparisons * Remove simple cache * Improve docstring (cherry picked from commit 299e20a)
* Fix cache for multiple time comparisons * Remove simple cache * Improve docstring (cherry picked from commit 299e20a)
* Fix cache for multiple time comparisons * Remove simple cache * Improve docstring (cherry picked from commit 299e20a)
* Fix cache for multiple time comparisons * Remove simple cache * Improve docstring (cherry picked from commit 299e20a)
* Fix cache for multiple time comparisons * Remove simple cache * Improve docstring (cherry picked from commit 299e20a)
* Fix cache for multiple time comparisons * Remove simple cache * Improve docstring (cherry picked from commit 299e20a)
* Fix cache for multiple time comparisons * Remove simple cache * Improve docstring (cherry picked from commit 299e20a)
* Fix cache for multiple time comparisons * Remove simple cache * Improve docstring (cherry picked from commit 299e20a)
* Fix cache for multiple time comparisons * Remove simple cache * Improve docstring (cherry picked from commit 299e20a)
* Fix cache for multiple time comparisons * Remove simple cache * Improve docstring (cherry picked from commit 299e20a)
* Fix cache for multiple time comparisons * Remove simple cache * Improve docstring (cherry picked from commit 299e20a)
* Fix cache for multiple time comparisons * Remove simple cache * Improve docstring (cherry picked from commit 299e20a)
* Fix cache for multiple time comparisons * Remove simple cache * Improve docstring
* Fix cache for multiple time comparisons * Remove simple cache * Improve docstring (cherry picked from commit 299e20a)
* Fix cache for multiple time comparisons * Remove simple cache * Improve docstring (cherry picked from commit 299e20a)
* Fix cache for multiple time comparisons * Remove simple cache * Improve docstring (cherry picked from commit 299e20a)
* Fix cache for multiple time comparisons * Remove simple cache * Improve docstring (cherry picked from commit 299e20a)
* Fix cache for multiple time comparisons * Remove simple cache * Improve docstring (cherry picked from commit 299e20a)
* Fix cache for multiple time comparisons * Remove simple cache * Improve docstring (cherry picked from commit 299e20a)
* Fix cache for multiple time comparisons * Remove simple cache * Improve docstring (cherry picked from commit 299e20a)
* Fix cache for multiple time comparisons * Remove simple cache * Improve docstring (cherry picked from commit 299e20a)
* Fix cache for multiple time comparisons * Remove simple cache * Improve docstring (cherry picked from commit 299e20a)
* Fix cache for multiple time comparisons * Remove simple cache * Improve docstring (cherry picked from commit 299e20a)
* Fix cache for multiple time comparisons * Remove simple cache * Improve docstring (cherry picked from commit 299e20a)
* Fix cache for multiple time comparisons * Remove simple cache * Improve docstring (cherry picked from commit 299e20a)
* Fix cache for multiple time comparisons * Remove simple cache * Improve docstring (cherry picked from commit 299e20a)
* Fix cache for multiple time comparisons * Remove simple cache * Improve docstring (cherry picked from commit 299e20a)
* Fix cache for multiple time comparisons * Remove simple cache * Improve docstring (cherry picked from commit 299e20a)
* Fix cache for multiple time comparisons * Remove simple cache * Improve docstring
Currently we have these two conflicting behaviors:
from_dttm
andto_dttm
from the query object when generating the cache dict.from_dttm
andto_dttm
in the query object toinner_from_dttm
andinner_to_dttm
, and update the original keys with the shifted values.When the second query runs, the cache key will be different from the first one even though
from_dttm
andto_dttm
are stripped from both query objects, because the second one has theinner_from_dttm
andinner_to_dttm
keys set.The problem is that multiple time shifts will have the same cache key, since the only thing differing them is
from_dttm
andto_dttm
. This results in a false positive cache hit. I fixed it by allowing passing extra cache keys when fetching a dataframe. This way each time comparison will add, eg,time_compare: "1 week"
to the cache dict.