-
Notifications
You must be signed in to change notification settings - Fork 29
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Productionize GitHub Action to update dependencies on schedule #188
Comments
I've been using Dependabot and pip-compile for a month now, and it's starting to show some issues. Some of it is fairly specific to Pallets, so I wouldn't expect it to be fixed by pip-compile-multi. Dependabot generating one PR per package is really noisy, especially across 8 repos and counting that all have the same dev dependency pins. So I do like the idea of pip-compile-multi making a single PR for all upgrades, and having it run less often. I don't think I want to add a workflow file to every repository, since I would have to keep any changes synchronized, although that might be the short term solution. In the long run, I'd like a bot that can update or merge an existing PR, it doesn't have to be as complex as Dependabot. I'm especially worried about not getting the notifications, since I will not remember to check every repo. On the pip-compile side, there's two issues. Specific to Pallets, Jinja and Click have dev dependencies (Sphinx and pip-tools) that depend on them in turn, so released versions get pinned, and pip currently installs those over an editable local install. I have issues open with them, but am not confident in a fast resolution. Maybe pip-compile-multi could have some flag to remove a line after building, but that gets messy and it's not pcm's job. The other issue is just that having separate requirements files for separate envs feels both very verbose and not descriptive enough. Three envs requires six files. But we actually have 5 envs (six if you separate maintainer vs contributor tools), Tox runs pip-compile and mypy as separate envs, and it seems silly to have separate All that said, I think the GitHub workflow you came up with is pretty cool, I don't want to discourage you from going forward with it. But I need to think more about what Pallets will do before using it. |
Thanks for the thorough reply! The lack of notifications can be solved by creating a separate GitHub account, I think it can be shared multiple repos, if each repo creates a unique auth token. I need to look more into it. I don't feel like having many For the keeping changes synchronized in all repositories problem, I think it's possible to extract the basic case into a marketplace GitHub Action, so it'll be referenced from each repo and updated in one place. Also, it might make sense to have palletes-specific shared Action, because of the auth token share issue. |
We use I think the part that would be most helpful for us is the ability to have For reference, here is the workflow we use, released here under the MIT and Apache 2.0 licenses: on:
workflow_dispatch: null
schedule:
# Monday 10AM UTC
- cron: "0 10 * * 1"
name: Update Python dependencies
jobs:
pip_compile_multi:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
cache: "pip"
python-version: "3.11"
- run: pip install pip-compile-multi
- run: pip-compile-multi -d <our_directory> --backtracking --autoresolve --header=<our_header.txt>
- uses: peter-evans/create-pull-request@v6
with:
branch: create-pull-request/pip-compile-multi
title: Update python dependencies.
body: Run of `pip-compile-multi` to upgrade all python dependencies.
delete-branch: true
labels: |
Dependencies
commit-message: |
Update python dependencies.
Run of `pip-compile-multi` to upgrade all python dependencies. |
That's a great suggestion! I'd love to incorporate that in the GitHub action. |
One of the friction points in the adoption of pip-compile-multi, is the lack of support by @dependabot-bot 1, which automates regular updates of the lock files.
The requirements for an update system are:
One way of implementing this is by using GitHub Actions.
PoC action definition: https://github.com/peterdemin/pip-compile-multi/blob/master/.github/workflows/pipcompilemulti.yml
Example update PR generated by the GitHub Action: #187
Known problems:
CC @davidism
The text was updated successfully, but these errors were encountered: