shuffle_transfer failed during shuffle in test_fused_blockwise_dataframe_merge

[j-bennet]

Describe the issue:

Distributed test started failing in dask/dask CI with distributed main.

The error looks like this:

FAILED dask/tests/test_distributed.py::test_fused_blockwise_dataframe_merge[True] - RuntimeError: shuffle_barrier failed during shuffle 1d8d714896c0fa093a11a0ddfed35635
FAILED dask/tests/test_distributed.py::test_fused_blockwise_dataframe_merge[False] - RuntimeError: shuffle_transfer failed during shuffle 1d8d714896c0fa093a11a0ddfed35635

Example workflow:

https://github.com/dask/dask/actions/runs/5555724247/jobs/10147277695

Anything else we need to know?:

Environment:

• Dask version: 2023.7.0
• Python version:
• Operating System:
• Install method (conda, pip, source):

[j-bennet]

cc @hendrikmakait @fjetter @crusaderky

[hendrikmakait]

Thanks for flagging this, @j-bennet. I can reproduce this locally when installing pyarrow 13.0.0.dev517. If using pyarrow=12, the test succeeds.

[hendrikmakait]

I've created a dedicated issue for this: #8007