Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Raise informative error whenever context is pickled #1711

Merged
merged 2 commits into from
Nov 7, 2019
Merged

Conversation

cicdw
Copy link
Member

@cicdw cicdw commented Nov 7, 2019

Thanks for contributing to Prefect!

Please describe your work and make sure your PR:

  • adds new tests (if appropriate)
  • updates CHANGELOG.md (if appropriate)
  • updates docstrings for any new functions or function arguments, including docs/outline.toml for API reference docs (if appropriate)

Note that your PR will not be reviewed unless all three boxes are checked.

What does this PR change?

This PR raises an informative error whenever context is pickled. Closes #1710

Why is this PR important?

Users are tempted to use context like:

from prefect import context

@task
def my_fn():
    logger = context['logger']

and submit such tasks to dask. This is problematic, because if we freeze context (which is what this import pattern effectively does), we lose all the dynamic information that is placed into context by the various runners.

Consequently, users should always access context as an attribute of the top-level module so that dynamically added keys will be available.

This PR raises an informative error for how to access context in the event that it is pickled.

@codecov
Copy link

codecov bot commented Nov 7, 2019

Codecov Report

Merging #1711 into master will decrease coverage by 0.01%.
The diff coverage is 100%.

@cicdw cicdw merged commit b5e8749 into master Nov 7, 2019
@cicdw cicdw deleted the context-pickle branch November 7, 2019 02:29
zanieb added a commit that referenced this pull request May 12, 2022
Fix bug where futures would be marked incorrectly marked as async
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Make context pickleable
2 participants