`split_whitespace` sporadically produces different results on MacOS

[agoose77]

Describe the bug, including details regarding any error messages, version, and platform.

We wrap PyArrow's string functions in Awkward Array, and it's been observed that one test fails sporadically on MacOSX for the split_whitespace function.

We don't always see this, which makes me think it's a default-allocation problem, which we tend to see on Mac.

https://github.com/scikit-hep/awkward/actions/runs/6203141156/job/16843262485?pr=2712

Component(s)

Python

[danepitkin]

Hey @agoose77, are you able to repro using just pyarrow? There is a lot of wrapper code in Awkward Array between the test case and the arrow call. I'd be curious to see the call stack and what the inputs are to the pyarrow compute call.

[agoose77]

@danepitkin ah, that's an oversight on my part; I intended to do so!

Let me figure it out.

[danepitkin]

No problem! Here's a small PyArrow-only example that does not reproduce the issue. I ran it locally on an M1 Mac. I don't think it recreates the exact inputs that Awkward Array is using though. Note that the empty array does fail, but I believe Awkward Array handles this internally before calling PyArrow.

arrow_37757.py

import pyarrow as pa
import pyarrow.compute as pc


string_padded = [
    ["      αβγ      ", "               "],
    [],
    ["     →δε←      ", "     ζz zζ     ", "      abc      "],
]
expected = [
    [["", "αβγ", "", ""], ["", "", ""]],
    [],
    [["", "→δε←", "", ""], ["", "ζz", "zζ", "", ""], ["", "abc", "", ""]],
]

for i, item in enumerate(string_padded):
    arr = pa.array(item)
    print(f"Testing: {arr}")
    try:
        assert pc.utf8_split_whitespace(item, max_splits=None).to_pylist() == expected[i]
    except Exception as err:
        print(f"{err=}")
        print("Test FAILED")
    else:
        print("Test PASSED")
    print()

Output:

$ python arrow_37757.py
Testing: [
  "      αβγ      ",
  "               "
]
Test PASSED

Testing: 0 nulls
err=ArrowNotImplementedError("Function 'utf8_split_whitespace' has no kernel matching input types (null)")
Test FAILED

Testing: [
  "     →δε←      ",
  "     ζz zζ     ",
  "      abc      "
]
Test PASSED

Version

$ python -c "import pyarrow as pa; print(pa.__version__);"
13.0.0

[agoose77]

Hmm, this one is going to be flaky to debug; we only seem to see it in the 3.11 runners (IIRC), and only sometimes on Mac. I'll keep an eye on it, but for now it's not something I can figure out an easy way to repro.