Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CI][Python] Dask integration nightly jobs fail on dask.dataframe.io.tests.test_parquet::test_pandas_timestamp_overflow_pyarrow #36629

Closed
raulcd opened this issue Jul 12, 2023 · 0 comments · Fixed by #36630

Comments

@raulcd
Copy link
Member

raulcd commented Jul 12, 2023

Describe the bug, including details regarding any error messages, version, and platform.

Dask nightlies have been failing since: #33321 was merged.

=================================== FAILURES ===================================
____________________ test_pandas_timestamp_overflow_pyarrow ____________________

tmpdir = local('/tmp/pytest-of-root/pytest-1/test_pandas_timestamp_overflow0')

    @PYARROW_MARK
    def test_pandas_timestamp_overflow_pyarrow(tmpdir):
        info = np.iinfo(np.dtype("int64"))
        # In `numpy=1.24.0` NumPy warns when an overflow is encountered when casting from float to int
        # https://numpy.org/doc/stable/release/1.24.0-notes.html#numpy-now-gives-floating-point-errors-in-casts
        if _numpy_124:
            ctx = pytest.warns(RuntimeWarning, match="invalid value encountered in cast")
        else:
            ctx = contextlib.nullcontext()
        with ctx:
            arr_numeric = np.linspace(
                start=info.min + 2, stop=info.max, num=1024, dtype="int64"
            )
        arr_dates = arr_numeric.astype("datetime64[ms]")
    
        table = pa.Table.from_arrays([pa.array(arr_dates)], names=["ts"])
        pa.parquet.write_table(
            table, f"{tmpdir}/file.parquet", use_deprecated_int96_timestamps=False
        )
    
        # This will raise by default due to overflow
>       with pytest.raises(pa.lib.ArrowInvalid) as e:
E       Failed: DID NOT RAISE <class 'pyarrow.lib.ArrowInvalid'>

Component(s)

Continuous Integration, Python

raulcd added a commit to raulcd/arrow that referenced this issue Jul 12, 2023
@jorisvandenbossche jorisvandenbossche added this to the 13.0.0 milestone Jul 13, 2023
jorisvandenbossche pushed a commit that referenced this issue Jul 13, 2023
…ges in arrow->pandas conversion (#36630)

### Rationale for this change

Due to the changes on #33321 a dask test started failing.

### What changes are included in this PR?

Skip the test in the meantime

### Are these changes tested?

Yes, with crossbow

### Are there any user-facing changes?

No
* Closes: #36629

Authored-by: Raúl Cumplido <raulcumplido@gmail.com>
Signed-off-by: Joris Van den Bossche <jorisvandenbossche@gmail.com>
raulcd added a commit that referenced this issue Jul 13, 2023
…ges in arrow->pandas conversion (#36630)

### Rationale for this change

Due to the changes on #33321 a dask test started failing.

### What changes are included in this PR?

Skip the test in the meantime

### Are these changes tested?

Yes, with crossbow

### Are there any user-facing changes?

No
* Closes: #36629

Authored-by: Raúl Cumplido <raulcumplido@gmail.com>
Signed-off-by: Joris Van den Bossche <jorisvandenbossche@gmail.com>
chelseajonesr pushed a commit to chelseajonesr/arrow that referenced this issue Jul 20, 2023
…d changes in arrow->pandas conversion (apache#36630)

### Rationale for this change

Due to the changes on apache#33321 a dask test started failing.

### What changes are included in this PR?

Skip the test in the meantime

### Are these changes tested?

Yes, with crossbow

### Are there any user-facing changes?

No
* Closes: apache#36629

Authored-by: Raúl Cumplido <raulcumplido@gmail.com>
Signed-off-by: Joris Van den Bossche <jorisvandenbossche@gmail.com>
R-JunmingChen pushed a commit to R-JunmingChen/arrow that referenced this issue Aug 20, 2023
…d changes in arrow->pandas conversion (apache#36630)

### Rationale for this change

Due to the changes on apache#33321 a dask test started failing.

### What changes are included in this PR?

Skip the test in the meantime

### Are these changes tested?

Yes, with crossbow

### Are there any user-facing changes?

No
* Closes: apache#36629

Authored-by: Raúl Cumplido <raulcumplido@gmail.com>
Signed-off-by: Joris Van den Bossche <jorisvandenbossche@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment