Update pandas minimum requirement for Python 3.12

[potiuk]

Pandas minimum requirement for Python 3.12 should be changed to be higher than >=2.1,1 because 2.1.1 version does not have proper exclusions for numpy<2.0.0 and on Python 3.12 it will install incompatible numpy 2.0.0 with Pandas 2.1.1 which fails with

numpy.dtype size changed, may indicate binary incompatibility. Expected 96 from C header, got 88 from PyObject

This is tracked in numpy/numpy#26710 but until (maybe) it is solved in numpy/pandas, we should limit it in Airflow.

---

^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in newsfragments.

[potiuk]

This also started to show-up ins case of "lowest-direct" deps - seems that simplifying pandas requirements to be >=2.1.2,<2.2 for all python versions should solve it better.

[potiuk]

3.8 is not compatible with pandas 2.1.2+ -> so we have to make it a bit more selective.

[johnnyplaydrums]

Hey folks, thanks for fixing this! Help me understand - in general how can I find out if this commit is part of a release yet? And are we able to use this or do we have to wait for a release to be cut?

[potiuk]

You can add your own pandas> if you want - no, it's not yet released it - but this one is merely limiting version of pandas to be bigger than certain versions. So you can do pip install apache-airflow==x.y.z pandas>=.... (look at the change in detail to understand what version to use) and it will be properly installing for you.

[NG-Corp]

so is it fixed yet?

[potiuk]

so is it fixed yet?

What do you think after reading the conversation and seeing two merged PRs? What are your worries? What issue are you thinking is not fixed @NG-Corp ?