Skip to content

Automatically upgrade various components in canary builds#37113

Merged
potiuk merged 1 commit intoapache:mainfrom
potiuk:upgrade-build-dependencies
Jan 31, 2024
Merged

Automatically upgrade various components in canary builds#37113
potiuk merged 1 commit intoapache:mainfrom
potiuk:upgrade-build-dependencies

Conversation

@potiuk
Copy link
Member

@potiuk potiuk commented Jan 31, 2024

This change attempts to automatically upgrade our dependencies in the canary builds. This means that canary builds will fail occasionally when there is a need to update some dependencies that otherwise are not updated automatically. This gives an opportunity to fix such failing main build as soon as it happns.

It does not impact generation of constraints or building the cache, the only sign that things are wrong is failing canary build, so impact to it should be minimal - we will either see the main build failing after merge, or we will fix it alongside with fixing some other reason for main failing (such us broken dependencies)

With this change, the following upgrades are done in canary build:

  • Upgrading pre-commits
  • Upgrading black version
  • Upgrading build pinned dependencies

^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in newsfragments.

@potiuk potiuk force-pushed the upgrade-build-dependencies branch 4 times, most recently from 6a27259 to 9a2b003 Compare January 31, 2024 08:32
@potiuk
Copy link
Member Author

potiuk commented Jan 31, 2024

OK. Should be ready for review. It won't run in regular PR - only in canary builds, but I commented the if and it nicely works https://github.com/apache/airflow/actions/runs/7723217069/job/21052882896?pr=37113

This way we will know when we should do pre-commit autoupdate , upgrade black, upgrade build dependencies without actually running it for PRs, which might otherwise introduce unrelated changes - only failing job in main will indicate that we need to run upgrade.

@potiuk
Copy link
Member Author

potiuk commented Jan 31, 2024

cc: @Taragolis - that's what I recently explained that we need to have :)

This change attempts to automatically upgrade our dependencies in
the canary builds. This means that canary builds will fail
occasionally when there is a need to update some dependencies that
otherwise are not updated automatically. This gives an opportunity
to fix such failing main build as soon as it happns.

It does not impact generation of constraints or building the
cache, the only sign that things are wrong is failing canary build,
so impact to it should be minimal - we will either see the main
build failing after merge, or we will fix it alongside with fixing
some other reason for main failing (such us broken dependencies)

Currently, the following upgrades are done in canary build:

* Upgrading pre-commits
* Upgrading black version
* Upgrading build pinned dependencies
@potiuk potiuk force-pushed the upgrade-build-dependencies branch from 9a2b003 to 7d879aa Compare January 31, 2024 09:15
@potiuk potiuk merged commit b14dbf2 into apache:main Jan 31, 2024
@potiuk potiuk deleted the upgrade-build-dependencies branch January 31, 2024 09:50
potiuk added a commit that referenced this pull request Feb 7, 2024
This change attempts to automatically upgrade our dependencies in
the canary builds. This means that canary builds will fail
occasionally when there is a need to update some dependencies that
otherwise are not updated automatically. This gives an opportunity
to fix such failing main build as soon as it happns.

It does not impact generation of constraints or building the
cache, the only sign that things are wrong is failing canary build,
so impact to it should be minimal - we will either see the main
build failing after merge, or we will fix it alongside with fixing
some other reason for main failing (such us broken dependencies)

Currently, the following upgrades are done in canary build:

* Upgrading pre-commits
* Upgrading black version
* Upgrading build pinned dependencies

(cherry picked from commit b14dbf2)
@potiuk potiuk added the changelog:skip Changes that should be skipped from the changelog (CI, tests, etc..) label Feb 8, 2024
@potiuk potiuk added this to the Airflow 2.8.2 milestone Feb 8, 2024
ephraimbuddy pushed a commit that referenced this pull request Feb 22, 2024
This change attempts to automatically upgrade our dependencies in
the canary builds. This means that canary builds will fail
occasionally when there is a need to update some dependencies that
otherwise are not updated automatically. This gives an opportunity
to fix such failing main build as soon as it happns.

It does not impact generation of constraints or building the
cache, the only sign that things are wrong is failing canary build,
so impact to it should be minimal - we will either see the main
build failing after merge, or we will fix it alongside with fixing
some other reason for main failing (such us broken dependencies)

Currently, the following upgrades are done in canary build:

* Upgrading pre-commits
* Upgrading black version
* Upgrading build pinned dependencies

(cherry picked from commit b14dbf2)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:dev-tools changelog:skip Changes that should be skipped from the changelog (CI, tests, etc..)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants