Skip to content

Call get_current_version less often in bundle refresh loop#46261

Merged
jedcunningham merged 4 commits intoapache:mainfrom
astronomer:slight_optimization_in_bundle_refresh_loop
Feb 3, 2025
Merged

Call get_current_version less often in bundle refresh loop#46261
jedcunningham merged 4 commits intoapache:mainfrom
astronomer:slight_optimization_in_bundle_refresh_loop

Conversation

@jedcunningham
Copy link
Member

In the bundle refresh loop, we can call get_current_version a lot less often, as 1) we can skip it for bundles that do not support versioning and 2) for those that do, we already know the version from the last time we refreshed!

Since this is a local call, this isn't a huge gain. But every little bit helps.

Trying this again, 3rd times the charm, as #45999 was reverted in #46037, and #46100 was reverted in #46221 🙃

@jedcunningham jedcunningham added full tests needed We need to run full set of tests for this PR to merge AIP-66: DAG Bundle/Manifest labels Jan 29, 2025
@jedcunningham jedcunningham force-pushed the slight_optimization_in_bundle_refresh_loop branch from 9738559 to f907e07 Compare January 29, 2025 22:31
@jedcunningham jedcunningham force-pushed the slight_optimization_in_bundle_refresh_loop branch 2 times, most recently from dd466da to 3f23db3 Compare January 31, 2025 02:26
In the bundle refresh loop, we can call `get_current_version` a lot less
often, as 1) we can skip it for bundles that do not support versioning
and 2) for those that do, we already know the version from the last time
we refreshed!

Since this is a local call, this isn't a huge gain. But every little bit
helps.
@jedcunningham jedcunningham force-pushed the slight_optimization_in_bundle_refresh_loop branch from c9fb76f to b6142c2 Compare February 3, 2025 21:24
@jedcunningham jedcunningham merged commit 930f9d2 into apache:main Feb 3, 2025
91 checks passed
@jedcunningham jedcunningham deleted the slight_optimization_in_bundle_refresh_loop branch February 3, 2025 22:13
niklasr22 pushed a commit to niklasr22/airflow that referenced this pull request Feb 8, 2025
…46261)

In the bundle refresh loop, we can call `get_current_version` a lot less
often, as 1) we can skip it for bundles that do not support versioning
and 2) for those that do, we already know the version from the last time
we refreshed!

Since this is a local call, this isn't a huge gain. But every little bit
helps.
ambika-garg pushed a commit to ambika-garg/airflow that referenced this pull request Feb 17, 2025
…46261)

In the bundle refresh loop, we can call `get_current_version` a lot less
often, as 1) we can skip it for bundles that do not support versioning
and 2) for those that do, we already know the version from the last time
we refreshed!

Since this is a local call, this isn't a huge gain. But every little bit
helps.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

AIP-66: DAG Bundle/Manifest area:DAG-processing full tests needed We need to run full set of tests for this PR to merge

Projects

Development

Successfully merging this pull request may close these issues.

2 participants

Comments