ARROW-14381: [CI][Python] Fix Spark integration failures #11465
Conversation
|
@github-actions crossbow submit -spark- |
|
|
|
@amol- I don't have a concise reproducer, but this has resolved the spark tests for me locally. |
|
Revision: bbd6804 Submitted crossbow builds: ursacomputing/crossbow @ actions-1009
|
|
@github-actions crossbow submit -spark- |
|
Revision: 2e9d790 Submitted crossbow builds: ursacomputing/crossbow @ actions-1010
|
|
@github-actions crossbow submit -spark- |
|
Revision: 06fb996 Submitted crossbow builds: ursacomputing/crossbow @ actions-1012
|
| "when converting numpy arrays") | ||
| if mask is not None: | ||
| if _is_array_like(mask): | ||
| mask = get_values(mask, &is_pandas_object) |
There was a problem hiding this comment.
The spark integrations tests verify that this resolves the python error, but we should cover this with unittests.
Deferred to https://issues.apache.org/jira/browse/ARROW-14388
There was a problem hiding this comment.
Passing a Series or a pandas extension array here should trigger it (I suppose that's what @amol- is testing)?
|
@BryanCutler updated the spark tasks to build against specific spark releases to maintain compatibility. |
kszucs
changed the title
kszucs
changed the title
|
Benchmark runs are scheduled for baseline = 0960fa6 and contender = bc223c6. bc223c6 is a master commit associated with this PR. Results will be available as each benchmark for each run completes. |
|
Thanks @kszucs |
I don't have a small reproducer, but either a pandas series or a dataframe gets passed as mask to `pa.array()` Closes apache#11465 from kszucs/ARROW-14381 Authored-by: Krisztián Szűcs <szucs.krisztian@gmail.com> Signed-off-by: Krisztián Szűcs <szucs.krisztian@gmail.com>
I don't have a small reproducer, but either a pandas series or a dataframe gets passed as mask to
pa.array()