Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Set up dependabot configuration for pip #551

Merged
merged 18 commits into from
Jun 5, 2022
Merged

Set up dependabot configuration for pip #551

merged 18 commits into from
Jun 5, 2022

Conversation

weiji14
Copy link
Contributor

@weiji14 weiji14 commented Jun 3, 2022

Enable dependabot to manage python dependencies pins for a more stable CI.

References:

Closes #488

@github-actions github-actions bot added the documentation Improvements or additions to documentation label Jun 4, 2022
@adamjstewart adamjstewart added this to the 0.2.2 milestone Jun 4, 2022
@adamjstewart
Copy link
Collaborator

adamjstewart commented Jun 4, 2022

Pushed a commit so that dependabot actually has something to update (requirements.txt) and we actually use it in CI. Remaining TODOs:

  • Solve version compatibility issues (latest version of numpy doesn't support oldest Python)
  • Make sure docs still render correctly (pytorch_sphinx_theme is very finicky about how it is installed: Missing fonts pytorch/pytorch_sphinx_theme#143)
  • Figure out how to use requirements.txt to control version of deps installed by conda
  • Figure out how to only install a subset of deps for different tests (Install only the needed lint tool in style checks CI #553)
  • Figure out why black is being installed with notebook support, if we can disable that, or fix notebooks

@adamjstewart
Copy link
Collaborator

I think this is ready for review. Some caveats to be aware of:

  1. pytorch-sphinx-theme has no official releases, so we're at the whims of the master branch. It also can't be specified anywhere other than docs/requirements.txt so that's fun: Missing fonts pytorch/pytorch_sphinx_theme#143. We should consider switching to trojanzoo_sphinx_theme, but we can do that in a separate PR.
  2. I haven't figured out an easy way to control the conda-installed packages with dependabot. For now, we can just manually update .github/workflows/tests.yaml every time fiona or rasterio has a new release. We could try to call grep on requirements.txt and use the output to specify the version for conda to install, but honestly it might be easier and more worthwhile to just add wheels to fiona and rasterio.
  3. NEP-29 is fun. If we want to continue to support Python 3.7 (I do), then we need to specify Python version-specific versions for all dependencies that subscribe to NEP-29's philosophy. I have no idea how dependabot will handle these. We can ignore them if we need to.
  4. I don't know how this will interact with Add upper bounds to all dependencies #544. I think dependabot will continue to update versions and ignore setup.cfg, so we have to be careful to update that ourselves whenever a new major version is released.
  5. All tests now install all dependencies. This is unfortunate because I really liked the idea of Install only the needed lint tool in style checks CI #553. There are open issues requesting support for multiple requirements.txt files, but no progress yet. I wonder if poetry would solve any of these issues?

TL;DR: This isn't perfect, but I think it will solve most of our frequent CI issues. We should merge this and see how annoying it is. If it's more trouble than it's worth, we can revert it or see if poetry makes things easier.

@adamjstewart adamjstewart marked this pull request as ready for review June 5, 2022 00:23
@adamjstewart adamjstewart changed the title Setup dependabot configuration for pip Set up dependabot configuration for pip Jun 5, 2022
@calebrob6
Copy link
Member

This all seems fine to me. Why run black on notebooks here?

@adamjstewart
Copy link
Collaborator

For some reason black was being installed with jupyter support by default now. I could figure out how to disable it, but I figured it would be easier to just run it on our notebooks.

@adamjstewart adamjstewart mentioned this pull request Jun 8, 2022
2 tasks
@adamjstewart
Copy link
Collaborator

Hmm, why is dependabot listed as a TorchGeo contributor? How do I remove him before he becomes the primary developer of TorchGeo? 😆

@adamjstewart adamjstewart modified the milestones: 0.2.2, 0.3.0 Jul 2, 2022
@adamjstewart adamjstewart mentioned this pull request Jul 11, 2022
@weiji14 weiji14 deleted the setup-dependabot branch August 19, 2022 23:39
yichiac pushed a commit to yichiac/torchgeo that referenced this pull request Apr 29, 2023
* Setup dependabot configuration for pip

* Add requirements.txt and use it

* Fix numpy for old Python

* segmentation-models-pytorch requires older timm

* Pin timm to older version

* older pyproj requires older python

* Fix documentation build

* Fix rasterio for older Python versions

* Fix sklearn and older Python

* Fix open3d missing wheels

* Fix open3d missing wheels

* Fix pandas for older python

* fix radiant-mlhub for older python

* fix scipy for older python

* flake8 and sphinx require conflicting importlib-metadata versions for older python

* Fix conda installs to match pip

* Fix rasterio on Windows

* Blacken notebooks

Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Dependabot for more stable CI
3 participants