Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Python] Interchange - test failing on MacOS builds #35490

Closed
AlenkaF opened this issue May 8, 2023 · 1 comment · Fixed by #35504
Closed

[Python] Interchange - test failing on MacOS builds #35490

AlenkaF opened this issue May 8, 2023 · 1 comment · Fixed by #35504

Comments

@AlenkaF
Copy link
Member

AlenkaF commented May 8, 2023

Describe the bug, including details regarding any error messages, version, and platform.

Noticed tests started failing on MacOS CI builds, example:

=================================== FAILURES ===================================
___________________ test_pandas_assertion_error_large_string ___________________

    @pytest.mark.large_memory
    @pytest.mark.pandas
    def test_pandas_assertion_error_large_string():
        # Test AssertionError as pandas does not support "U" type strings
        if Version(pd.__version__) < Version("1.5.0"):
            pytest.skip("__dataframe__ added to pandas in 1.5.0")
    
        data = np.array([b'x'*1024]*(3*1024**2), dtype='object')  # 3GB bytes data
        arr = pa.array(data, type=pa.large_string())
        table = pa.table([arr], names=["large_string"])
    
        from pandas.api.interchange import (
            from_dataframe as pandas_from_dataframe
        )
    
>       with pytest.raises(AssertionError):
E       Failed: DID NOT RAISE <class 'AssertionError'>

/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/pyarrow/tests/interchange/test_conversion.py:294: Failed

from: https://github.com/apache/arrow/actions/runs/4917476625/jobs/8782656971

Component(s)

Python

@AlenkaF
Copy link
Member Author

AlenkaF commented May 9, 2023

The AssertionError should start to fail with pandas 2.0.1 as the support for large strings was added to the interchange protocol implementation in pandas-dev/pandas#52795.

I do find it confusing that it errors only on MacOS. Plus, on my local machine (MacOS) it doesn't error (with 2.1.0 dev pandas version).
Edit: they are skipped due to the large_memory mark.

@jorisvandenbossche jorisvandenbossche added this to the 13.0.0 milestone May 11, 2023
jorisvandenbossche pushed a commit that referenced this issue May 11, 2023
…large_string (#35504)

### Rationale for this change

In pandas version 2.0.1 the interchange protocol implementation has support for large strings. The tests on our side need to be updated accordingly.

### What changes are included in this PR?
Changes in tests:
- `test_pandas_assertion_error_large_string` removed
- `test_roundtrip_pandas_string ` updated
* Closes: #35490

Authored-by: Alenka Frim <frim.alenka@gmail.com>
Signed-off-by: Joris Van den Bossche <jorisvandenbossche@gmail.com>
ArgusLi pushed a commit to Bit-Quill/arrow that referenced this issue May 15, 2023
…g and large_string (apache#35504)

### Rationale for this change

In pandas version 2.0.1 the interchange protocol implementation has support for large strings. The tests on our side need to be updated accordingly.

### What changes are included in this PR?
Changes in tests:
- `test_pandas_assertion_error_large_string` removed
- `test_roundtrip_pandas_string ` updated
* Closes: apache#35490

Authored-by: Alenka Frim <frim.alenka@gmail.com>
Signed-off-by: Joris Van den Bossche <jorisvandenbossche@gmail.com>
rtpsw pushed a commit to rtpsw/arrow that referenced this issue May 16, 2023
…g and large_string (apache#35504)

### Rationale for this change

In pandas version 2.0.1 the interchange protocol implementation has support for large strings. The tests on our side need to be updated accordingly.

### What changes are included in this PR?
Changes in tests:
- `test_pandas_assertion_error_large_string` removed
- `test_roundtrip_pandas_string ` updated
* Closes: apache#35490

Authored-by: Alenka Frim <frim.alenka@gmail.com>
Signed-off-by: Joris Van den Bossche <jorisvandenbossche@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants