Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add ability to use a module path entrypoint when using .serve or .deploy #12134

Merged
merged 10 commits into from
Mar 4, 2024

Conversation

desertaxle
Copy link
Member

@desertaxle desertaxle commented Mar 1, 2024

Adds the ability to use an entrypoint in the form path.to.module.function_name when creating a deployment. This is useful for users who have flows contained in Python packages and can be used as an alternative code storage method. It also makes it easy to create deployments for flows from Prefect's integration packages.

Supports the implementation of #11998

Note: support for module path entrypoints for the prefect deploy CLI is not included in this PR. Support will be added in another PR and will likely require a refactor to the flow registration process used in prefect deploy.

Example

When deploying or serving a flow, users can specify that they'd like to use a module path entrypoint by providing a value to entrypoint_type:

from prefect_dbt.cloud.jobs import trigger_dbt_cloud_job_run_and_wait_for_completion

from prefect.deployments import EntrypointType

if __name__ == "__main__":
    trigger_dbt_cloud_job_run_and_wait_for_completion.deploy(
        name="demo-deploy-from-module",
        work_pool_name="olympic",
        entrypoint_type=EntrypointType.MODULE_PATH,
        job_variables={"env": { "EXTRA_PIP_PACKAGES": "prefect-dbt" } }
    )

This will update the entrypoint discovery logic to return the module path to the function instead of a filepath to the function.

Checklist

  • This pull request references any related issue by including "closes <link to issue>"
    • If no issue exists and your change is not a small fix, please create an issue first.
  • This pull request includes tests or only affects documentation.
  • This pull request includes a label categorizing the change e.g. maintenance, fix, feature, enhancement, docs.

For documentation changes:

  • This pull request includes redirect settings in netlify.toml for files that are removed or renamed.

For new functions or classes in the Python SDK:

  • This pull request includes helpful docstrings.
  • If a new Python file was added, this pull request contains a stub page in the Python SDK docs and an entry in mkdocs.yml navigation.

Copy link

netlify bot commented Mar 1, 2024

Deploy Preview for prefect-docs-preview ready!

Name Link
🔨 Latest commit a724f4f
🔍 Latest deploy log https://app.netlify.com/sites/prefect-docs-preview/deploys/65e6145eab2a710008bb3326
😎 Deploy Preview https://deploy-preview-12134--prefect-docs-preview.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

@desertaxle desertaxle added the enhancement An improvement of an existing feature label Mar 1, 2024
@desertaxle desertaxle changed the title Add ability to provide a module path entrypoint for a deployment Add ability to provide a module path entrypoint when using .serve or .deploy Mar 4, 2024
@desertaxle desertaxle marked this pull request as ready for review March 4, 2024 16:38
@desertaxle desertaxle requested a review from a team as a code owner March 4, 2024 16:38
@desertaxle desertaxle changed the title Add ability to provide a module path entrypoint when using .serve or .deploy Add ability to use a module path entrypoint when using .serve or .deploy Mar 4, 2024
test.py Outdated Show resolved Hide resolved
Copy link
Contributor

@zzstoatzz zzstoatzz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice!

src/prefect/deployments/runner.py Outdated Show resolved Hide resolved
@desertaxle desertaxle merged commit 7cbec50 into main Mar 4, 2024
45 checks passed
@desertaxle desertaxle deleted the deployment-module-path-entrypoint branch March 4, 2024 19:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement An improvement of an existing feature
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants