Skip to content

Bump Amazon provider boto3/botocore to 1.43.0, aiobotocore to 3.6.0#69206

Draft
vgkowski wants to merge 1 commit into
apache:mainfrom
vgkowski:amazon-bump-aws-sdk-versions
Draft

Bump Amazon provider boto3/botocore to 1.43.0, aiobotocore to 3.6.0#69206
vgkowski wants to merge 1 commit into
apache:mainfrom
vgkowski:amazon-bump-aws-sdk-versions

Conversation

@vgkowski

@vgkowski vgkowski commented Jul 1, 2026

Copy link
Copy Markdown

The EMR Serverless interactive session APIs (StartSession, GetSession,
GetSessionEndpoint, TerminateSession) are only available in botocore 1.43.0+.
aiobotocore 3.6.0 is the first release whose botocore pin (>=1.42.90,<1.43.1)
admits botocore 1.43.0, so the async/deferrable code paths resolve on the
minimum supported versions.

Split out from the EMR Serverless interactive session feature so the
dependency change can be reviewed on its own.

Was generative AI tooling used to co-author this PR?
  • Yes — Kiro (AI coding assistant)

Generated-by: Kiro, following the Gen-AI guidelines in contributing-docs/05_pull_requests.rst

The EMR Serverless interactive session APIs (StartSession, GetSession, GetSessionEndpoint, TerminateSession) are only available in botocore 1.43.0+. aiobotocore 3.6.0 is the first release whose botocore pin (>=1.42.90,<1.43.1) admits it, so the async/deferrable code paths resolve on the minimum versions.
@boring-cyborg boring-cyborg Bot added area:providers provider:amazon AWS/Amazon - related issues labels Jul 1, 2026
@boring-cyborg

boring-cyborg Bot commented Jul 1, 2026

Copy link
Copy Markdown

Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contributors' Guide
Here are some useful points:

  • Pay attention to the quality of your code (ruff, mypy and type annotations). Our prek-hooks will help you with that.
  • In case of a new feature add useful documentation (in docstrings or in docs/ directory). Adding a new operator? Check this short guide Consider adding an example Dag that shows how users should use it.
  • Consider using Breeze environment for testing locally, it's a heavy docker but it ships with a working Airflow and a lot of integrations.
  • Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
  • Please follow ASF Code of Conduct for all communication including (but not limited to) comments on Pull Requests, Mailing list and Slack.
  • Be sure to read the Airflow Coding style.
  • Always keep your Pull Requests rebased, otherwise your build might fail due to changes not related to your commits.
    Apache Airflow is a community-driven project and together we are making it better 🚀.
    In case of doubts contact the developers at:
    Mailing List: dev@airflow.apache.org
    Slack: https://s.apache.org/airflow-slack

@eladkal

eladkal commented Jul 1, 2026

Copy link
Copy Markdown
Contributor

The EMR Serverless interactive session APIs (StartSession, GetSession,
GetSessionEndpoint, TerminateSession) are only available in botocore 1.43.0+.
aiobotocore 3.6.0 is the first release whose botocore pin (>=1.42.90,<1.43.1)
admits botocore 1.43.0, so the async/deferrable code paths resolve on the
minimum supported versions.

Split out from the EMR Serverless interactive session feature so the
dependency change can be reviewed on its own.

Nothing prevents you from using newer aiobotocore / botocore versions.
Why would you want to limit everyone else? I dare to say that most users of the provider don't use EMR serverless

@ferruzzi

ferruzzi commented Jul 2, 2026

Copy link
Copy Markdown
Contributor

The current min boto is 1.41, almost 8 months old and 2 versions (142 releases) behind. The note in the project file explicitly states we should be updating it regularly. I don't feel like a bump is out of line here, especially when it unlocks features someone is trying to use.

@vgkowski an option for now would be to gate your new features, which is a pain, but would work. Inside each of your new operators you'd need to add if boto version < 1.43: raise some error, which is not a great user experience, but keeps the constraints where they are

@eladkal

eladkal commented Jul 2, 2026

Copy link
Copy Markdown
Contributor

8 months old

that is not very old in terms of production systems.
We normally update when there is a need (pip resolution too deep error or something like that) otherwise we prefer to stay as flexible as possible.
I'd like to remind that at one time we bumped watchtower and then aws relaxed it to support users who run older versions.
I'll let AWS decide how they want to play this. Feel free to proceed if you think it's right.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:providers provider:amazon AWS/Amazon - related issues

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants