Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Bitbucket storage #3711

Merged
merged 22 commits into from Dec 2, 2020
Merged

Conversation

philz-catz
Copy link

@philz-catz philz-catz commented Nov 25, 2020

Summary

Add support for Bitbucket as a file-based storage option alongside GitHub and GitLab.

Changes

Uses atlassian-python-api package ([https://github.com/atlassian-api/atlassian-python-api]) to allow Bitbucket to be used as a storage option.

Implementation is identical to GitLab but includes additional argument necessary for Bitbucket:

  • 'project' : Bitbucket organizes repositories into Projects and each repo requires an associated project

Importance

Bitbucket is a popular Git distribution and support has been requested (Issue #3343).

Checklist

This PR:

  • adds new tests (if appropriate)
  • adds a change file in the changes/ directory (if appropriate)
  • updates docstrings for any new functions or function arguments, including docs/outline.toml for API reference docs (if appropriate)

Choi added 3 commits November 24, 2020 21:23
Add support for Bitbucket as a file-based storage option alongside
GitHub and GitLab.

Prefect already supports registering and running of flows from
GitHub/GitLab. Since Bitbucket is also a Git distribution and is
fairly popular, the ability to run flows from a Bitbucket repo
would be very useful and has an open issue (PrefectHQ#3343).

The new Bitbucket option utilizes the atlassian-python-api package
to connect to and retrieve flows from a Bitbucket repo on either
Cloud or Server.

Usage and installation is almost identical to GitLab and GitHub,
except Bitbucket requires an additional argument, `project`, that
is Bitbucket specific. Otherwise, installation occurs through the
bitbucket extra and authentication is done through the
BITBUCKET_ACCESS_TOKEN secret.
Add support for Bitbucket as a file-based storage option alongside
GitHub and GitLab.

Prefect already supports registering and running of flows from
GitHub/GitLab. Since Bitbucket is also a Git distribution and is
fairly popular, the ability to run flows from a Bitbucket repo
would be very useful and has an open issue (PrefectHQ#3343).

The new Bitbucket option utilizes the atlassian-python-api package
to connect to and retrieve flows from a Bitbucket repo on either
Cloud or Server.

Usage and installation is almost identical to GitLab and GitHub,
except Bitbucket requires an additional argument, `project`, that
is Bitbucket specific. Otherwise, installation occurs through the
bitbucket extra and authentication is done through the
BITBUCKET_ACCESS_TOKEN secret.
Add support for Bitbucket as a file-based storage option alongside
GitHub and GitLab.

Prefect already supports registering and running of flows from
GitHub/GitLab. Since Bitbucket is also a Git distribution and is
fairly popular, the ability to run flows from a Bitbucket repo
would be very useful and has an open issue (PrefectHQ#3343).

The new Bitbucket option utilizes the atlassian-python-api package
to connect to and retrieve flows from a Bitbucket repo on either
Cloud or Server.

Usage and installation is almost identical to GitLab and GitHub,
except Bitbucket requires an additional argument, `project`, that
is Bitbucket specific. Otherwise, installation occurs through the
bitbucket extra and authentication is done through the
BITBUCKET_ACCESS_TOKEN secret.
@marvin-robot
Copy link
Member

Here I am, brain the size of a planet and they ask me to welcome you to Prefect.

So, welcome to the community @philz-catz! 🎉 🎉

@codecov
Copy link

codecov bot commented Nov 30, 2020

Codecov Report

Merging #3711 (7feb8d3) into master (901f705) will decrease coverage by 0.07%.
The diff coverage is 82.55%.

@philz-catz philz-catz marked this pull request as ready for review November 30, 2020 19:31
@marvin-robot
Copy link
Member

Here I am, brain the size of a planet and they ask me to welcome you to Prefect.

So, welcome to the community @philz-catz! 🎉 🎉

@philz-catz philz-catz marked this pull request as draft December 2, 2020 01:13
@philz-catz philz-catz marked this pull request as ready for review December 2, 2020 01:30
@@ -143,6 +143,28 @@ GitLab storage uses a [personal access token](https://docs.gitlab.com/ee/user/pr
GitLab server users can point the `host` argument to their personal GitLab instance.
:::

## Bitbucket
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you move this to /orchestration/flow_config/storage.html? That is a new page where these details are organized

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I went ahead and copied it into flow_config/storage.md. Should I delete it from the exectution/storage_options.md?

src/prefect/agent/local/agent.py Outdated Show resolved Hide resolved
src/prefect/environments/storage/bitbucket.py Outdated Show resolved Hide resolved
Choi added 3 commits December 2, 2020 10:22
- Removed bitbucket-storage-label from local agent labels
- Removed reference to label in bitbucket.py
- Moved bitbucket storage documentation to flow_config
@@ -57,12 +57,6 @@ def __init__(
self.path = path
self.ref = ref

super().__init__(**kwargs)
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oops looks like an accidental delete of the super 🙂

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yup, I goofed that one. Also need to update the unit tests to remove the "bitbucket-flow-storage" label.

@joshmeek joshmeek merged commit 835e73d into PrefectHQ:master Dec 2, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants