Skip to content
This repository has been archived by the owner on Apr 26, 2024. It is now read-only.

Commit

Permalink
Add cruft for automated template sychronization (#125)
Browse files Browse the repository at this point in the history
  • Loading branch information
ahuang11 committed Oct 14, 2022
1 parent e393728 commit 11aedc4
Show file tree
Hide file tree
Showing 11 changed files with 157 additions and 15 deletions.
20 changes: 20 additions & 0 deletions .cruft.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
{
"template": "https://github.com/PrefectHQ/prefect-collection-template",
"commit": "ff7c2498c022e0722e9ed99a09e7bc58d22b94d9",
"checkout": null,
"context": {
"cookiecutter": {
"full_name": "Prefect Technologies, Inc.",
"email": "help@prefect.io",
"github_organization": "PrefectHQ",
"collection_name": "prefect-aws",
"collection_slug": "prefect_aws",
"collection_short_description": "Prefect integrations for interacting with AWS.",
"_copy_without_render": [
".github/workflows/*.yml"
],
"_template": "https://github.com/PrefectHQ/prefect-collection-template"
}
},
"directory": null
}
36 changes: 29 additions & 7 deletions .github/PULL_REQUEST_TEMPLATE.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,32 @@
<!-- Thanks for contributing to prefect-aws! 🎉-->
<!--
Thanks for opening a pull request to prefect-aws 🎉!
## Summary
<!-- A brief summary explaining the purpose of this PR -->
We've got a few requests to help us review contributions:
## Relevant Issue(s)
<!-- If this PR addresses any open issues, please let us know which one here -->
- Make sure that your title neatly summarizes the proposed changes.
- Provide a short overview of the change and the value it adds.
- Share an example to help us understand the change in user experience.
- Run `pre-commit install && pre-commit run --all` for linting.
## Checklist
- [ ] Summarized PR's changes in the **Unreleased** section of the [CHANGELOG.md](https://github.com/PrefectHQ/prefect-aws/blob/main/CHANGELOG.md)
Happy engineering!
-->

<!-- Include an overview here -->

<!-- Link to issue -->
Closes #

### Example
<!--
Share an example of the change in action.
A code blurb is best. Changes to features should include an example that is executable by a new user.
-->

### Checklist
<!-- These boxes may be checked after opening the pull request. -->

- [ ] This pull request references any related issue by including "Closes #<ISSUE_NUMBER>"
- If no issue exists and your change is not a small fix, please [create an issue](https://github.com/PrefectHQ/prefect-aws/issues/new/choose) first.
- [ ] This pull request includes tests or only affects documentation.
- [ ] Summarized PR's changes in [CHANGELOG.md](https://github.com/PrefectHQ/prefect-aws/blob/main/CHANGELOG.md)
24 changes: 24 additions & 0 deletions .github/workflows/add-to-project.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
name: Add issues to integrations board

on:
issues:
types:
- opened

jobs:

add-to-project:
name: Add issue to project
runs-on: ubuntu-latest
steps:
- uses: tibdex/github-app-token@v1
id: generate-token
name: Generate GitHub token
with:
app_id: ${{ secrets.SYNC_APP_ID }}
private_key: ${{ secrets.SYNC_APP_PRIVATE_KEY }}

- uses: actions/add-to-project@v0.3.0
with:
project-url: ${{ secrets.ADD_TO_PROJECT_URL }}
github-token: ${{ steps.generate-token.outputs.token }}
46 changes: 46 additions & 0 deletions .github/workflows/template-sync.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
name: Template Synchronization
on:
schedule:
- cron: "0 0 * * *"
workflow_dispatch:

jobs:
submit-update-pr:
name: Submit update PR
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3

- name: Set up Python
uses: actions/setup-python@v4
with:
python-version: 3.9

- name: Install cruft
run: pip install "cookiecutter>=1.7.3,<2.0.0" cruft

- name: Perform updates
run: cruft update -y

- uses: tibdex/github-app-token@v1
id: generate-token
name: Generate GitHub token
with:
app_id: ${{ secrets.SYNC_APP_ID }}
private_key: ${{ secrets.SYNC_APP_PRIVATE_KEY }}

- name: Submit PR
uses: peter-evans/create-pull-request@v4
with:
commit-message: Updating collection with changes to prefect-collection-template
token: ${{ steps.generate-token.outputs.token }}
branch: sync-with-template
delete-branch: true
title: Sync Collection with changes to prefect-collection-template
body: |
Automated PR created to propagate changes from prefect-collection-template to this collection
Feel free to make any necessary changes to this PR before merging.
labels: |
template sync
automated pr
16 changes: 11 additions & 5 deletions MAINTAINERS.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ pip install -e ".[dev]"
pre-commit install
```

To verify the set up was successful you can run the following:
To verify the setup was successful you can run the following:

- Run the tests for tasks and flows in the collection:
```bash
Expand All @@ -69,23 +69,29 @@ This collection has been setup to with [mkdocs](https://www.mkdocs.org/) for aut

### CI Pipeline

This collection comes with [GitHub Actions](https://docs.github.com/en/actions) for testing and linting. To add additional actions, you can add jobs in the `.github/workflows` folder. On pull request, the pipeline will run linting via [`black`](https://black.readthedocs.io/en/stable/), [`flake8`](https://flake8.pycqa.org/en/latest/), [`interrogate`](https://interrogate.readthedocs.io/en/latest/), and unit tests via `pytest` alongside `coverage`.
This collection comes with [GitHub Actions](https://docs.github.com/en/actions) for testing and linting. To add additional actions, you can add jobs in the `.github/workflows` folder. Upon a pull request, the pipeline will run linting via [`black`](https://black.readthedocs.io/en/stable/), [`flake8`](https://flake8.pycqa.org/en/latest/), [`interrogate`](https://interrogate.readthedocs.io/en/latest/), and unit tests via `pytest` alongside `coverage`.

`interrogate` will tell you which methods, functions, classes, and modules have docstrings, and which do not--the job has a fail threshold of 95%, meaning that it will fail if more than 5% of the codebase is undocumented. We recommend following the [Google Python Style Guide](https://google.github.io/styleguide/pyguide.html#38-comments-and-docstrings) for docstring format.

Simiarly, `coverage` ensures that the codebase includes tests--the job has a fail threshold of 80%, meaning that it will fail if more than 20% of the codebase is missing tests.

### Track Issues on Project Board

To automatically add issues to a GitHub Project Board, you'll need a [secret added](https://docs.github.com/en/actions/security-guides/encrypted-secrets#creating-encrypted-secrets-for-an-environment) to the repository. Specifically, a secret named `ADD_TO_PROJECT_URL`, formatted like `https://github.com/orgs/<GITHUB_ORGANIZATION>/projects/<PROJECT_NUMBER>`.

### Package and Publish

GitHub actions will handle packaging and publishing of your collection to [PyPI](https://pypi.org/) so other Prefect users can your collection in their flows.

In order to publish to PyPI, you'll need a PyPI account and generate an API token to authenticate with PyPI when publishing new versions of your collection. The [PyPI documentation](https://pypi.org/help/#apitoken) outlines the steps needed to get an API token.
To publish to PyPI, you'll need a PyPI account and to generate an API token to authenticate with PyPI when publishing new versions of your collection. The [PyPI documentation](https://pypi.org/help/#apitoken) outlines the steps needed to get an API token.

Once you've obtained a PyPI API token, [create a GitHub secret](https://docs.github.com/en/actions/security-guides/encrypted-secrets#creating-encrypted-secrets-for-a-repository) named `PYPI_API_TOKEN`.

To create publish a new version of your collection, [create a new GitHub release](https://docs.github.com/en/repositories/releasing-projects-on-github/managing-releases-in-a-repository#creating-a-release) and tag it with the version that you want to deploy (e.g. v0.3.2). This will trigger workflow to publish the new version on PyPI and deploy the updated docs to GitHub pages.
To publish a new version of your collection, [create a new GitHub release](https://docs.github.com/en/repositories/releasing-projects-on-github/managing-releases-in-a-repository#creating-a-release) and tag it with the version that you want to deploy (e.g. v0.3.2). This will trigger a workflow to publish the new version on PyPI and deploy the updated docs to GitHub pages.

Upon publishing, a `docs` branch is automatically created. To hook this up to GitHub Pages, simply head over to https://github.com/PrefectHQ/prefect-aws/settings/pages, select `docs` under the dropdown menu, keep the default `/root` folder, `Save`, and upon refresh, you should see a prompt stating "Your site is published at https://PrefectHQ.github.io/prefect-aws". Don't forget to add this link to the repo's "About" section, under "Website" so users can access the docs easily.

Upon publishing, a `docs` branch is automatically created. To hook this up to GitHub Pages, simply head over to https://github.com/PrefectHQ/prefect-aws/settings/pages, select `docs` under the dropdown menu, keep the default `/root` folder, `Save`, and upon refresh, you should see a prompt stating "Your site is published at https://<username>.github.io/<repository>".
Feel free to [submit your collection](https://orion-docs.prefect.io/collections/overview/#listing-in-the-collections-catalog) to the Prefect [Collections Catalog](https://orion-docs.prefect.io/collections/catalog/)!

## Further guidance

Expand Down
12 changes: 12 additions & 0 deletions MANIFEST.in
Original file line number Diff line number Diff line change
@@ -1,2 +1,14 @@
# Things to always exclude
global-exclude .git*
global-exclude .ipynb_checkpoints
global-exclude *.py[co]
global-exclude __pycache__/**

# Top-level Config
include versioneer.py
include prefect_aws/_version.py
include LICENSE
include MANIFEST.in
include setup.cfg
include requirements.txt
include requirements-dev.txt
9 changes: 9 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,13 @@ Install `prefect-aws`
```bash
pip install prefect-aws
```
Then, register to [view the block](https://orion-docs.prefect.io/ui/blocks/) on Prefect Cloud:

```bash
prefect block register -m prefect_aws.credentials
```

Note, to use the `load` method on Blocks, you must already have a block document [saved through code](https://orion-docs.prefect.io/concepts/blocks/#saving-blocks) or [saved through the UI](https://orion-docs.prefect.io/ui/blocks/).

### AWS Authentication

Expand Down Expand Up @@ -131,6 +138,8 @@ If you encounter and bugs while using `prefect-aws`, feel free to open an issue

If you have any questions or issues while using `prefect-aws`, you can find help in either the [Prefect Discourse forum](https://discourse.prefect.io/) or the [Prefect Slack community](https://prefect.io/slack)

Feel free to ⭐️ or watch [`prefect-aws`](https://github.com/PrefectHQ/prefect-aws) for updates too!

## Development

If you'd like to install a version of `prefect-aws` for development, first clone the repository and then perform an editable install with `pip`:
Expand Down
Binary file added docs/img/prefect-logo-mark-solid-white-500.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
7 changes: 4 additions & 3 deletions mkdocs.yml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
site_name: prefect-aws
site_url: https://prefecthq.github.io/prefect-aws
repo_url: https://github.com/prefecthq/prefect-aws
site_url: https://PrefectHQ.github.io/prefect-aws
repo_url: https://github.com/PrefectHQ/prefect-aws
edit_uri: edit/main/docs/
theme:
name: material
Expand All @@ -10,7 +10,8 @@ theme:
accent: blue
icon:
repo: fontawesome/brands/github
logo: img/prefect-logo-white.png
logo:
img/prefect-logo-mark-solid-white-500.png
font:
text: Inter
code: Source Code Pro
Expand Down
1 change: 1 addition & 0 deletions requirements-dev.txt
Original file line number Diff line number Diff line change
Expand Up @@ -16,3 +16,4 @@ interrogate
coverage
types-boto3 >= 1.0.2
boto3-stubs >= 1.24.39
pillow
1 change: 1 addition & 0 deletions setup.cfg
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ parentdir_prefix =

[tool:interrogate]
ignore-init-module = True
ignore_init_method = True
exclude = prefect_aws/_version.py, tests, setup.py, versioneer.py, docs, site
fail-under = 95
omit-covered-files = True
Expand Down

0 comments on commit 11aedc4

Please sign in to comment.