Cloud Caching #885
Cloud Caching #885
Conversation
| cd prefect | ||
| yarn install | ||
| pip install "prefect[all_extras]" | ||
| pip install ".[all_extras]" |
There was a problem hiding this comment.
I updated this because otherwise it would break if new classes / methods were added in the master branch vs. the latest release
src/prefect/client/client.py
Outdated
| "where": { | ||
| "state": {"_eq": "Cached"}, | ||
| "task_id": {"_eq": task_id}, | ||
| "created": {"_gte": created_after.isoformat()}, |
There was a problem hiding this comment.
Sorry I didn't catch this last night -- this field "created" is when the task_run itself was created. You'll want to check the state_timestamp field instead, which is the time the current state was set for the task_run.
As an example, a task run might be created today in anticipation of a scheduled flow run, but that run might not start for a month. In a month's time, the task might enter a cached state, and that time (in a month) is the quantity of interest.
There was a problem hiding this comment.
That makes too much sense -- good catch and sorry for the oversight!
src/prefect/client/client.py
Outdated
| "task_id": {"_eq": task_id}, | ||
| "created": {"_gte": created_after.isoformat()}, | ||
| }, | ||
| "order_by": {"created": EnumValue("desc")}, |
There was a problem hiding this comment.
Same comment regarding created vs state_timestamp
|
🚀 |
Thanks for contributing to Prefect!
Please describe your work and make sure your PR:
CHANGELOG.md(if appropriate)docs/outline.tomlfor API reference docs (if appropriate)What does this PR change?
This PR implements a new
clientmethod for extracting stored cached states, and additionally implements the necessary queries so that cached tasks will work across flow runs in Cloud.