ResolutionImpossible when rebuilding image with Breeze

[uranusjr]

$ ./breeze build-image
...
#16 30.11 ERROR: Cannot install apache-airflow[devel-ci]==2.2.0.dev0 because these package versions have conflicting dependencies.
#16 30.11
#16 30.11 The conflict is caused by:
#16 30.11     apache-airflow[devel-ci] 2.2.0.dev0 depends on flask-appbuilder<4.0.0 and >=3.3.2
#16 30.11     The user requested (constraint) flask-appbuilder==3.3.1
#16 30.11
#16 30.11 To fix this you could try to:
#16 30.11 1. loosen the range of package versions you've specified
#16 30.11 2. remove package versions to allow pip attempt to solve the dependency conflict
#16 30.11
#16 30.11 ERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/user_guide/#fixing-conflicting-dependencies

I think this is because the constraint file was generated with an older Airflow commit before we upgraded Flask-Appbuilder, so now things are in a deadlock because CI can’t pass to build the constraints, and the currently available constraints can’t be used to run the test suite?

[potiuk]

Yep. I have not pushed latest constraints yesterday. My fault. It should happen automatically after successful merge to main.

But since we are now having the mssql it has to be done manually

For the future reference the procedure to do it is described in this chapter "Manually generating constraint files":

https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#manually-generating-constraint-files

With explanation why we sometimes need to do it (main failure).

(any committer can do it)

[potiuk]

Should be fixed now - I generated them yesterday, but have not pushed the change so this was very quick.

[potiuk]

It is only really needed when you want to force-update some dependency and increase the minimum version (like we did with FAB) - in all other cases it will work just fine and Breeze will automatically upgrade to latest constraints.

I will thin maybe how to avoid this in the future. I think what might actually help is that in this case I could fall-back to non-constraint installation just to get things going. Let me think about all the cases and I will implement something. For sure it could be helpful for PRs and locally building Breeze but should be disabled when you try to build production image (and fail hard in this case as it would indicate a real conflict).

[potiuk]

BTW @uranusjr The new generated constraints are here: f31598e - as you can see from the history - in most cases the constraint are auto-updated but as of 17 days they weren't due to the MSSQL /intermittent failures, so I upgraded them manually from time to time https://github.com/apache/airflow/commits/constraints-main - this has however the risk, that some new dependencies will fail some tests and we have to fix them manually - normally those upgrades only happen when all the tests pass.

[vandonr-amz]

For the future reference the procedure to do it is described in this chapter "Manually generating constraint files":

https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#manually-generating-constraint-files

Note: this content was moved to https://github.com/apache/airflow/blob/main/dev/REFRESHING_CI_CACHE.md#manually-generating-constraint-files