Bumping PyArrow to 21.0.0 in awswrangler 3.14.0 breaks installs on al2-v3 SageMaker notebook instances

[ericfzhu]

Describe the bug

al2-v3 notebook instances have CMake version 2.8.12.2, whereas al2023-v1 notebook instances have CMake version 3.22.2. Trying to install awswrangler without specifying a version will now try to install PyArrow 21.0.0, which has a CMake requirement of 3.25 or higher. This means that you will receive an error about failing to build wheel for pyarrow on al2-v3

Should this be included in the docs if the change was intentional?

How to Reproduce

pip install awswrangler #on al2-v3 SageMaker notebook instances

Expected behavior

No response

Your project

No response

Screenshots

No response

OS

Amazon Linux 2

Python version

3.10.19

AWS SDK for pandas version

3.14.0

Additional context

No response

[kukushking]

Hey @ericfzhu thanks for opening this. It is true that pyarrow v21+ stopped building on AL2 due to an older CMake, but you should still be able to use an older version pyarrow (technically >=8,<23) on AL2 if you can't upgrade AL2023.

On AL2, assuming you do not want to upgrade CMake to build pyarrow, try this:

pip install pyarrow==20.0.0 awswrangler