Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Added source parameter for spark_python_task in databricks_job #2157

Merged
merged 1 commit into from Mar 28, 2023
Merged

Added source parameter for spark_python_task in databricks_job #2157

merged 1 commit into from Mar 28, 2023

Conversation

nkvuong
Copy link
Contributor

@nkvuong nkvuong commented Mar 24, 2023 •
edited by nfx

Changes

Add support for using spark_python_task with a git_source by adding the source parameter, same as notebook_task

Closes #2138

Tests

  • make test run locally
  • relevant change in docs/ folder
  • covered with integration tests in internal/acceptance
  • relevant acceptance tests are passing
  • using Go SDK

@nkvuong nkvuong force-pushed the feature/add-source-python-task branch from 70732ec to b32641c Compare March 24, 2023 15:06
@nkvuong
Copy link
Contributor Author

nkvuong commented Mar 24, 2023

acceptance tests passed on all 3 clouds

@nkvuong nkvuong marked this pull request as ready for review March 24, 2023 17:08
alexott
alexott approved these changes Mar 24, 2023
View reviewed changes
Copy link
Contributor

@alexott alexott left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@nfx nfx changed the title add source parameter for spark_python_task in databricks_job Added source parameter for spark_python_task in databricks_job Mar 28, 2023
@nfx nfx merged commit eb4fe2a into master Mar 28, 2023
4 checks passed
@nfx nfx deleted the feature/add-source-python-task branch March 28, 2023 08:13
@nfx nfx mentioned this pull request Mar 31, 2023
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@alexott alexott alexott approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[FEATURE] Support "source" in resource databricks_job task spark_python_task
3 participants
@nkvuong @alexott @nfx