Dependabot making Incompatible Dependency upgrades

[amogkam]

We've recently added Dependabot to our repository (https://github.com/ray-project/ray) and it works great for the most part. We are using pip-tools/pip-compile and have both our manifest and lock file checked in.

However, a problem we've been seeing is that Dependabot is making PRs for upgrades that are incompatible with other dependencies.

For example ray-project/ray#13645, which leads to this error in our CI

ERROR: Cannot install boto3==1.16.59 and botocore==1.19.58 because these package versions have conflicting dependencies.
ERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/user_guide/#fixing-conflicting-dependencies

I would expect Dependabot to automatically resolve these and upgrade other dependencies all in the same PR to maintain compatibility. Calling pip-compile should automatically do this I believe- does Dependabot call this internally?

I'd like to avoid having to manually fix these dependency conflict issues. Thanks!

Package manager/ecosystem

Manifest contents prior to update

Updated dependency

What you expected to see, versus what you actually saw

Images of the diff or a link to the PR, issue or logs

[titouanc]

Hello,

I encountered a similar issue in a private repository. I observed that this problem happens when the requirements are specified in a package's setup.cfg and then compiled into requirements.txt.

I have been able to isolate the issue in this repository: https://github.com/titouanc/demo-incompatible-dependabot-upgrades

[jurre]

Hello,

I encountered a similar issue in a private repository. I observed that this problem happens when the requirements are specified in a package's setup.cfg and then compiled into requirements.txt.

I have been able to isolate the issue in this repository: https://github.com/titouanc/demo-incompatible-dependabot-upgrades

Thanks for putting that together! I'll put this on our backlog to investigate, but that should help a tonne 🙇

[alancleary]

I too am experiencing this issue with a Python repo that uses a requirements.txt file generated by pip-tools/pip-compile. In my case the package that conflicted with Dependabot's PR was already at its most recent version so the PR was impossible to merge.

I've also had the issue in a repo that uses NPM. The repo depends on @ngrx/effects, @ngrx/entity, @ngrx/router-store, and @ngrx/store. The first three packages all depend on @ngrx/store and their versions must be the same. Dependabot opened a separate PR to update each of the first three packages individually. A corresponding update to @ngrx/store was not included in any of those PRs nor did Dependatbot open a separate PR to update @ngrx/store.

[jeffwidman]

👋 Sorry for the slow reply.

I would expect Dependabot to automatically resolve these and upgrade other dependencies all in the same PR to maintain compatibility. Calling pip-compile should automatically do this I believe- does Dependabot call this internally?

For top-level dependencies, Dependabot currently only bumps one dependency at a time in most cases. I say most because there's a handful of exceptions in the npm world. But for python it'll only bump one.

However, it should not be opening a PR to bump a dependency that results in a conflict... Internally Dependabot is calling the native python package manager, in your case pip-compile, so that should throw an error, which eventually bubbles up as an update_not_possible error in the Dependabot logs. Long term, we'd like to eventually add support for grouped updates (#1190 ) but no ETA for that right now.

@titouanc thanks for the fantastic repro example--I broke it out as a new issue in #6593 so that we don't lose track of it, as I'm not convinced it's related to the first report in this issue.

@alancleary I'd recommend you also file a new bug, as tagging onto this as a catch-all when it's really a fairly specific bug report doesn't really make it possible to triage/deal with.

[jeffwidman]

@amogkam it's been two years since your original bug report, and we've shipped a bunch of updates internally, as well as newer versions of pip-compile which include their upstream bug fixes.

Is this still reproducible or should this issue be closed?

[jeffwidman]

I'm going to close as it's too easy for us to lose track of old stale issues otherwise. But if you're still able to reproduce this, please comment and we can re-open.

Again, I broke out one of the reported issues in this thread as a new separate issue since I am not convinced it's related to the original issue:

• setup.cfg + pip-compile > requirements.txt = Dependabot buggy PRs #6593