Skip to content

Fix broken tests due to pymysql 1.2.0 incompat with aiomysql#67467

Merged
vatsrahul1001 merged 1 commit into
apache:mainfrom
amoghrajesh:worktree-fix-pymysql-aiomysql-compat
May 25, 2026
Merged

Fix broken tests due to pymysql 1.2.0 incompat with aiomysql#67467
vatsrahul1001 merged 1 commit into
apache:mainfrom
amoghrajesh:worktree-fix-pymysql-aiomysql-compat

Conversation

@amoghrajesh
Copy link
Copy Markdown
Contributor

@amoghrajesh amoghrajesh commented May 25, 2026


Was generative AI tooling used to co-author this PR?
  • Yes (please specify the tool below)

Fix for CI like: https://github.com/apache/airflow/actions/runs/26391986239/job/77689459612?pr=67460 and in main too

Current behaviour

pymysql 1.2.0 changed the default of Connection.ping(reconnect) from True to False.
SQLAlchemy's _send_false_to_ping memoized property inspects this signature — when it sees
reconnect=False it switches to calling dbapi_connection.ping() with no arguments.

But SQLAlchemy's AsyncAdapt_aiomysql_connection.ping(self, reconnect: bool) has no default for
reconnect, so the bare call raises a TypeError every time the async connection pool runs a
pre-ping health check (pool_pre_ping=True).

Error looks like this:

_____________________________________________________________________________________ TestSession.test_async_session _____________________________________________________________________________________
airflow-core/tests/unit/utils/test_session.py:66: in test_async_session
    my_special_log_event = await session.scalar(select(Log).where(Log.event == "hihi1234").limit(1))
/usr/python/lib/python3.10/site-packages/sqlalchemy/ext/asyncio/session.py:505: in scalar
    return await greenlet_spawn(
/usr/python/lib/python3.10/site-packages/sqlalchemy/util/_concurrency_py3k.py:203: in greenlet_spawn
    result = context.switch(value)
/usr/python/lib/python3.10/site-packages/sqlalchemy/orm/session.py:2399: in scalar
    return self._execute_internal(
/usr/python/lib/python3.10/site-packages/sqlalchemy/orm/session.py:2239: in _execute_internal
    conn = self._connection_for_bind(bind)
/usr/python/lib/python3.10/site-packages/sqlalchemy/orm/session.py:2108: in _connection_for_bind
    return trans._connection_for_bind(engine, execution_options)
<string>:2: in _connection_for_bind
    ???
/usr/python/lib/python3.10/site-packages/sqlalchemy/orm/state_changes.py:137: in _go
    ret_value = fn(self, *arg, **kw)
/usr/python/lib/python3.10/site-packages/sqlalchemy/orm/session.py:1187: in _connection_for_bind
    conn = bind.connect()
/usr/python/lib/python3.10/site-packages/sqlalchemy/engine/base.py:3293: in connect
    return self._connection_cls(self)
/usr/python/lib/python3.10/site-packages/sqlalchemy/engine/base.py:143: in __init__
    self._dbapi_connection = engine.raw_connection()
/usr/python/lib/python3.10/site-packages/sqlalchemy/engine/base.py:3317: in raw_connection
    return self.pool.connect()
/usr/python/lib/python3.10/site-packages/sqlalchemy/pool/base.py:448: in connect
    return _ConnectionFairy._checkout(self)
/usr/python/lib/python3.10/site-packages/sqlalchemy/pool/base.py:1371: in _checkout
    with util.safe_reraise():
/usr/python/lib/python3.10/site-packages/sqlalchemy/util/langhelpers.py:121: in __exit__
    raise exc_value.with_traceback(exc_tb)
/usr/python/lib/python3.10/site-packages/sqlalchemy/pool/base.py:1309: in _checkout
    result = pool._dialect._do_ping_w_event(
/usr/python/lib/python3.10/site-packages/sqlalchemy/engine/default.py:729: in _do_ping_w_event
    return self.do_ping(dbapi_connection)
/usr/python/lib/python3.10/site-packages/sqlalchemy/dialects/mysql/pymysql.py:123: in do_ping
    dbapi_connection.ping()
E   TypeError: AsyncAdapt_aiomysql_connection.ping() missing 1 required positional argument: 'reconnect'

Solution

Cap pymysql<1.2 in the mysql provider's dependencies until SQLAlchemy's aiomysql adapter is
updated to handle the new signature.


  • Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
  • For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
  • When adding dependency, check compliance with the ASF 3rd Party License Policy.
  • For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

@vatsrahul1001 vatsrahul1001 added the backport-to-v3-2-test Mark PR with this label to backport to v3-2-test branch label May 25, 2026
@amoghrajesh amoghrajesh added the full tests needed We need to run full set of tests for this PR to merge label May 25, 2026
@amoghrajesh
Copy link
Copy Markdown
Contributor Author

Lets run with full tests to be sure

@amoghrajesh amoghrajesh reopened this May 25, 2026
@vatsrahul1001 vatsrahul1001 merged commit 7ea8362 into apache:main May 25, 2026
198 of 203 checks passed
@github-actions
Copy link
Copy Markdown
Contributor

Backport failed to create: v3-2-test. View the failure log Run details

Note: As of Merging PRs targeted for Airflow 3.X
the committer who merges the PR is responsible for backporting the PRs that are bug fixes (generally speaking) to the maintenance branches.

In matter of doubt please ask in #release-management Slack channel.

Status Branch Result
v3-2-test Commit Link

You can attempt to backport this manually by running:

cherry_picker 7ea8362 v3-2-test

This should apply the commit to the v3-2-test branch and leave the commit in conflict state marking
the files that need manual conflict resolution.

After you have resolved the conflicts, you can continue the backport process by running:

cherry_picker --continue

If you don't have cherry-picker installed, see the installation guide.

vatsrahul1001 added a commit that referenced this pull request May 25, 2026
…#67470)

(cherry picked from commit 7ea8362)

Co-authored-by: Amogh Desai <amoghrajesh1999@gmail.com>
potiuk added a commit that referenced this pull request May 25, 2026
The released apache-airflow-providers-mysql on PyPI does not yet carry
the pymysql<1.2 cap from #67467, so PyPI constraint generation picks up
pymysql 1.2.0 and breaks SQLAlchemy's AsyncAdapt_aiomysql_connection.ping().
Mirror the cap in the constraints script's additional_constraints_for_highest_resolution
so PyPI constraints stay installable until the SQLAlchemy fix is released.
Also extend the comment in providers/mysql/pyproject.toml to link the
upstream tracking issue (sqlalchemy/sqlalchemy#13306).
potiuk added a commit that referenced this pull request May 25, 2026
…67492)

The released apache-airflow-providers-mysql on PyPI does not yet carry
the pymysql<1.2 cap from #67467, so PyPI constraint generation picks up
pymysql 1.2.0 and breaks SQLAlchemy's AsyncAdapt_aiomysql_connection.ping().
Mirror the cap in the constraints script's additional_constraints_for_highest_resolution
so PyPI constraints stay installable until the SQLAlchemy fix is released.
Also extend the comment in providers/mysql/pyproject.toml to link the
upstream tracking issue (sqlalchemy/sqlalchemy#13306).
(cherry picked from commit 6e9757b)

Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
vatsrahul1001 added a commit that referenced this pull request May 25, 2026
…#67470)

(cherry picked from commit 7ea8362)

Co-authored-by: Amogh Desai <amoghrajesh1999@gmail.com>
vatsrahul1001 pushed a commit that referenced this pull request May 25, 2026
…67492)

The released apache-airflow-providers-mysql on PyPI does not yet carry
the pymysql<1.2 cap from #67467, so PyPI constraint generation picks up
pymysql 1.2.0 and breaks SQLAlchemy's AsyncAdapt_aiomysql_connection.ping().
Mirror the cap in the constraints script's additional_constraints_for_highest_resolution
so PyPI constraints stay installable until the SQLAlchemy fix is released.
Also extend the comment in providers/mysql/pyproject.toml to link the
upstream tracking issue (sqlalchemy/sqlalchemy#13306).
(cherry picked from commit 6e9757b)

Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
vatsrahul1001 added a commit that referenced this pull request May 25, 2026
…#67470)

(cherry picked from commit 7ea8362)

Co-authored-by: Amogh Desai <amoghrajesh1999@gmail.com>
vatsrahul1001 pushed a commit that referenced this pull request May 25, 2026
…67492)

The released apache-airflow-providers-mysql on PyPI does not yet carry
the pymysql<1.2 cap from #67467, so PyPI constraint generation picks up
pymysql 1.2.0 and breaks SQLAlchemy's AsyncAdapt_aiomysql_connection.ping().
Mirror the cap in the constraints script's additional_constraints_for_highest_resolution
so PyPI constraints stay installable until the SQLAlchemy fix is released.
Also extend the comment in providers/mysql/pyproject.toml to link the
upstream tracking issue (sqlalchemy/sqlalchemy#13306).
(cherry picked from commit 6e9757b)

Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:providers backport-to-v3-2-test Mark PR with this label to backport to v3-2-test branch full tests needed We need to run full set of tests for this PR to merge kind:documentation provider:mysql

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants