Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(sdk): add local execution skeleton #localexecution #10292

Conversation

connor-mccarthy
Copy link
Member

Description of your changes:
Local execution PR 2.

Adds the skeleton code for invoking local execution (behind a feature flag).

If a component is called outside of a pipeline, it will be executed by task_dispatcher.run_single_component. The PipelineTask has handling for local execution.

Checklist:

Copy link

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

@connor-mccarthy
Copy link
Member Author

/assign @chensun

if self.state == TaskState.FINAL:
raise Exception(
custom_message or
f"Task configuration methods are not supported for local execution. Got call to '.{method_name}()'."
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@chensun, I tend to think having an error message that is specific to local-execution is helpful for the user since that's the only case when a task will be FINAL, but please let me know if you prefer we make the error message less specific.

Also, preferring the general Exception to NotImplementedError, which doesn't seem to fit based on its description.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SGTM

Copy link

@connor-mccarthy: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
kubeflow-pipelines-sdk-execution-tests 71f36f9 link true /test kubeflow-pipelines-sdk-execution-tests

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

Copy link
Member

@chensun chensun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm
/approve

Thanks!

if self.state == TaskState.FINAL:
raise Exception(
custom_message or
f"Task configuration methods are not supported for local execution. Got call to '.{method_name}()'."
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SGTM

@google-oss-prow google-oss-prow bot added the lgtm label Dec 8, 2023
Copy link

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: chensun

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@connor-mccarthy
Copy link
Member Author

kubeflow-pipelines-sdk-execution-tests failures are independent of these changes and tracked separately. Manually merging.

@connor-mccarthy connor-mccarthy merged commit 5cd708d into kubeflow:master Dec 9, 2023
19 of 21 checks passed
petethegreat pushed a commit to petethegreat/pipelines that referenced this pull request Dec 10, 2023
stijntratsaertit pushed a commit to stijntratsaertit/kfp that referenced this pull request Feb 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants