Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[CI] Use latest RAPIDS; Pandas 2.0 compatibility fix #10175

Merged
merged 6 commits into from
Apr 15, 2024
Merged

[CI] Use latest RAPIDS; Pandas 2.0 compatibility fix #10175

Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
8721c8b
[CI] Update RAPIDS to latest stable
hcho3 Apr 12, 2024
fbc8511
[CI] Use rapidsai stable channel; fix syntax errors in Dockerfile.gpu
hcho3 Apr 12, 2024
acf3482
Don't combine astype() with loc()
hcho3 Apr 12, 2024
2674b67
Work around https://github.com/dmlc/xgboost/issues/10181
hcho3 Apr 13, 2024
7e1d1a9
Fix formatting
hcho3 Apr 13, 2024
fc12745
Fix test
hcho3 Apr 13, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 8 additions & 1 deletion python-package/xgboost/data.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -909,9 +909,16 @@ def _transform_cudf_df(
enable_categorical: bool,
) -> Tuple[ctypes.c_void_p, list, Optional[FeatureNames], Optional[FeatureTypes]]:
try:
from cudf.api.types import is_categorical_dtype
from cudf.api.types import is_categorical_dtype, is_bool_dtype
except ImportError:
from cudf.utils.dtypes import is_categorical_dtype
from pandas.api.types import is_bool_dtype

# Work around https://github.com/dmlc/xgboost/issues/10181
if _is_cudf_ser(data) and is_bool_dtype(data.dtype):
data = data.astype(np.uint8)
else:
data = data.astype({col: np.uint8 for col in data.select_dtypes(include="bool")})
Copy link
Collaborator

@hcho3 hcho3 Apr 13, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note. This workaround is only needed for cuDF. The handler for Pandas already converts all Boolean columns into float type. See #10181


if _is_cudf_ser(data):
dtypes = [data.dtype]
Expand Down
4 changes: 2 additions & 2 deletions python-package/xgboost/testing/__init__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -429,8 +429,8 @@ def make_categorical(
categories = np.arange(0, n_categories)
for col in df.columns:
if rng.binomial(1, cat_ratio, size=1)[0] == 1:
df.loc[:, col] = df[col].astype("category")
df.loc[:, col] = df[col].cat.set_categories(categories)
df[col] = df[col].astype("category")
df[col] = df[col].cat.set_categories(categories)
Comment on lines +432 to +433
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Per https://pandas.pydata.org/docs/user_guide/basics.html#astype, we should not mix astype() with loc() accessor, as loc() tries to fit in what we are assigning to the current dtypes. So df.loc[:, col] = df[col].astype("category") would have no effect.


if sparsity > 0.0:
for i in range(n_features):
Expand Down
2 changes: 1 addition & 1 deletion tests/buildkite/conftest.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ set -x

CUDA_VERSION=11.8.0
NCCL_VERSION=2.16.5-1
RAPIDS_VERSION=24.02
RAPIDS_VERSION=24.04
SPARK_VERSION=3.4.0
JDK_VERSION=8
R_VERSION=4.3.2
Expand Down
8 changes: 4 additions & 4 deletions tests/ci_build/Dockerfile.gpu
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,14 +21,14 @@ ENV PATH=/opt/mambaforge/bin:$PATH

# Create new Conda environment with cuDF, Dask, and cuPy
RUN \
conda install -c conda-forge mamba && \
mamba create -n gpu_test -c rapidsai-nightly -c rapidsai -c nvidia -c conda-forge -c defaults \
export NCCL_SHORT_VER=$(echo "$NCCL_VERSION_ARG" | cut -d "-" -f 1) && \
mamba create -y -n gpu_test -c rapidsai -c nvidia -c conda-forge \
python=3.10 cudf=$RAPIDS_VERSION_ARG* rmm=$RAPIDS_VERSION_ARG* cudatoolkit=$CUDA_VERSION_ARG \
nccl>=$(cut -d "-" -f 1 << $NCCL_VERSION_ARG) \
"nccl>=${NCCL_SHORT_VER}" \
dask=2024.1.1 \
dask-cuda=$RAPIDS_VERSION_ARG* dask-cudf=$RAPIDS_VERSION_ARG* cupy \
numpy pytest pytest-timeout scipy scikit-learn pandas matplotlib wheel python-kubernetes urllib3 graphviz hypothesis \
pyspark>=3.4.0 cloudpickle cuda-python && \
"pyspark>=3.4.0" cloudpickle cuda-python && \
mamba clean --all && \
conda run --no-capture-output -n gpu_test pip install buildkite-test-collector

Expand Down
Loading