[Python] Crash in tests/test_dataset.py::test_write_dataset_s3

[asfimport]

Fatal Python error: Segmentation fault
1328
1329Thread 0x0000000117170e00 (most recent call first):
1330  File "/usr/local/lib/python3.9/site-packages/pyarrow/dataset.py", line 927 in write_dataset
1331  File "/usr/local/lib/python3.9/site-packages/pyarrow/tests/test_dataset.py", line 4265 in test_write_dataset_s3
1332  File "/usr/local/lib/python3.9/site-packages/_pytest/python.py", line 192 in pytest_pyfunc_call
1333  File "/usr/local/lib/python3.9/site-packages/pluggy/_callers.py", line 39 in _multicall
1334  File "/usr/local/lib/python3.9/site-packages/pluggy/_manager.py", line 80 in _hookexec
1335  File "/usr/local/lib/python3.9/site-packages/pluggy/_hooks.py", line 265 in __call__
1336  File "/usr/local/lib/python3.9/site-packages/_pytest/python.py", line 1761 in runtest
1337  File "/usr/local/lib/python3.9/site-packages/_pytest/runner.py", line 166 in pytest_runtest_call
1338  File "/usr/local/lib/python3.9/site-packages/pluggy/_callers.py", line 39 in _multicall
1339  File "/usr/local/lib/python3.9/site-packages/pluggy/_manager.py", line 80 in _hookexec
1340  File "/usr/local/lib/python3.9/site-packages/pluggy/_hooks.py", line 265 in __call__
1341  File "/usr/local/lib/python3.9/site-packages/_pytest/runner.py", line 259 in <lambda>
1342  File "/usr/local/lib/python3.9/site-packages/_pytest/runner.py", line 338 in from_call
1343  File "/usr/local/lib/python3.9/site-packages/_pytest/runner.py", line 258 in call_runtest_hook
1344  File "/usr/local/lib/python3.9/site-packages/_pytest/runner.py", line 219 in call_and_report
1345  File "/usr/local/lib/python3.9/site-packages/_pytest/runner.py", line 130 in runtestprotocol
1346  File "/usr/local/lib/python3.9/site-packages/_pytest/runner.py", line 111 in pytest_runtest_protocol
1347  File "/usr/local/lib/python3.9/site-packages/pluggy/_callers.py", line 39 in _multicall
1348  File "/usr/local/lib/python3.9/site-packages/pluggy/_manager.py", line 80 in _hookexec
1349  File "/usr/local/lib/python3.9/site-packages/pluggy/_hooks.py", line 265 in __call__
1350  File "/usr/local/lib/python3.9/site-packages/_pytest/main.py", line 347 in pytest_runtestloop
1351  File "/usr/local/lib/python3.9/site-packages/pluggy/_callers.py", line 39 in _multicall
1352  File "/usr/local/lib/python3.9/site-packages/pluggy/_manager.py", line 80 in _hookexec
1353  File "/usr/local/lib/python3.9/site-packages/pluggy/_hooks.py", line 265 in __call__
1354  File "/usr/local/lib/python3.9/site-packages/_pytest/main.py", line 322 in _main
1355  File "/usr/local/lib/python3.9/site-packages/_pytest/main.py", line 268 in wrap_session
1356  File "/usr/local/lib/python3.9/site-packages/_pytest/main.py", line 315 in pytest_cmdline_main
1357  File "/usr/local/lib/python3.9/site-packages/pluggy/_callers.py", line 39 in _multicall
1358  File "/usr/local/lib/python3.9/site-packages/pluggy/_manager.py", line 80 in _hookexec
1359  File "/usr/local/lib/python3.9/site-packages/pluggy/_hooks.py", line 265 in __call__
1360  File "/usr/local/lib/python3.9/site-packages/_pytest/config/__init__.py", line 164 in main
1361  File "/usr/local/lib/python3.9/site-packages/_pytest/config/__init__.py", line 187 in console_main
1362  File "/usr/local/bin/pytest", line 8 in <module>
1363ci/scripts/python_test.sh: line 55: 20279 Segmentation fault: 11  pytest -r s -v ${PYTEST_ARGS} --pyargs pyarrow
1364tests/test_dataset.py::test_write_dataset_s3  

Reporter: Alessandro Molina / @amol-

Related issues:

• [C++] S3-related crash on macOS (duplicates)

_{Note: This issue was originally created as ARROW-16139. Please see the migration documentation for further details.}

[asfimport]

David Li / @lidavidm:
Is this the same as ARROW-16043 possibly?

[asfimport]

Joris Van den Bossche / @jorisvandenbossche:
Yes, I assume that is the case (wanted to mention the same issue). The Python CI is already failing with that crash for a while.