Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migrate automations to PDM #4625

Merged
merged 4 commits into from
Jul 22, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/ci_cd.yml
Original file line number Diff line number Diff line change
Expand Up @@ -1022,7 +1022,7 @@ jobs:
run: python get_folder_differences.py

- name: Deploy
uses: peaceiris/actions-gh-pages@v3
uses: peaceiris/actions-gh-pages@v4
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This change is unrelated but I think it's ok to include it.

with:
github_token: ${{ secrets.GITHUB_TOKEN }}
publish_dir: /tmp/gh-pages
Expand Down
21 changes: 0 additions & 21 deletions automations/python/Pipfile

This file was deleted.

631 changes: 0 additions & 631 deletions automations/python/Pipfile.lock

This file was deleted.

6 changes: 3 additions & 3 deletions automations/python/justfile
Original file line number Diff line number Diff line change
Expand Up @@ -11,9 +11,9 @@ NO_COLOR := "\\033[0m"


# Install dependencies
install:
pipenv install --dev
install *args:
pdm install {{ args }}

# Run a script
run script *args:
pipenv run {{ script }} {{ args }}
pdm run {{ script }} {{ args }}
584 changes: 584 additions & 0 deletions automations/python/pdm.lock

Large diffs are not rendered by default.

26 changes: 26 additions & 0 deletions automations/python/pyproject.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
[project]
name = "openverse-automations"
version = "0.0.0"
description = "Automations for the Openverse project"
readme = "README.md"
authors = [
{name = "Openverse Contributors", email = "openverse@wordpress.org"},
]

requires-python = "==3.12.*"
dependencies = [
"pygithub >= 2.3.0, < 3",
"pyyaml >= 6.0.1, < 7",
"beautifulsoup4 >= 4.12.3, < 5",
"requests >= 2.32.3, < 3",
"mechanize >= 0.4.10, < 1",
"pyotp >= 2.9.0, < 3",
]

[tool.pdm]
distribution = false

[tool.pdm.dev-dependencies]
dev = [
"ipython",
]
14 changes: 5 additions & 9 deletions documentation/automations/guides/quickstart.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ Follow the [general setup guide](/general/general_setup.md) to set up `ov`.
Node.js dependencies for automations as well as other parts of the repository.
If you wish to install only dependencies for automations, run the following:

- `ov pipenv install` in `automations/python`
- `ov pdm install` in `automations/python`
- `ov pnpm install --filter=automations`

## Running a Script
Expand All @@ -27,7 +27,7 @@ Ex.:
ov just automations/python/run print_labels.py
```

The recipe is an alias for running `ov pipenv run <script>` inside the
The recipe is an alias for running `ov pdm run <script>` inside the
`automations/python` directory. This facilitates correct `PYTHONPATH`
configuration as expected by many scripts.

Expand All @@ -48,10 +48,6 @@ executable scripts available can also be run using the same recipe.
Many Openverse automation scripts expect to run inside a GitHub actions context
and require specific environment variables to operate. Refer to individual
script documentation (or implementation when no module documentation exists) to
know which environment variables are expected.
[Pipenv automatically loads `.env` files](https://pipenv-fork.readthedocs.io/en/latest/advanced.html#automatic-loading-of-env)
so for ease of use, you may place relevant environment variables in
`automations/python/.env`. Unlike Pipenv, however, pnpm does _not_ automatically
load environment variables. If a Node.js script requires environment variables,
you must pass them yourself via the command line or use another method to make
the variables available to the script.
know which environment variables are expected. If a script requires environment
variables, you must pass them yourself via the command line or use another
method to make the variables available to the script.
Comment on lines +51 to +53
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good observation on updating the docs.