Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[CI] Use latest RAPIDS; Pandas 2.0 compatibility fix #10175

Merged
merged 6 commits into from
Apr 15, 2024
Merged

[CI] Use latest RAPIDS; Pandas 2.0 compatibility fix #10175

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
8721c8b
[CI] Update RAPIDS to latest stable
hcho3 Apr 12, 2024
fbc8511
[CI] Use rapidsai stable channel; fix syntax errors in Dockerfile.gpu
hcho3 Apr 12, 2024
acf3482
Don't combine astype() with loc()
hcho3 Apr 12, 2024
2674b67
Work around https://github.com/dmlc/xgboost/issues/10181
hcho3 Apr 13, 2024
7e1d1a9
Fix formatting
hcho3 Apr 13, 2024
fc12745
Fix test
hcho3 Apr 13, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 11 additions & 1 deletion python-package/xgboost/data.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -909,9 +909,19 @@ def _transform_cudf_df(
enable_categorical: bool,
) -> Tuple[ctypes.c_void_p, list, Optional[FeatureNames], Optional[FeatureTypes]]:
try:
from cudf.api.types import is_categorical_dtype
from cudf.api.types import is_bool_dtype, is_categorical_dtype
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we have a problem with pandas? We started using the array interface with it recently.

Copy link
Collaborator

@hcho3 hcho3 Apr 15, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought we convert boolean columns to float?

xgboost/python-package/xgboost/data.py

Lines 513 to 524 in 6e5c335

def oth_type(ser: pd.Series) -> np.ndarray:
# The dtypes module is added in 1.25.
npdtypes = np.lib.NumpyVersion(np.__version__) > np.lib.NumpyVersion("1.25.0")
npdtypes = npdtypes and isinstance(
ser.dtype,
(
# pylint: disable=no-member
np.dtypes.Float32DType, # type: ignore
# pylint: disable=no-member
np.dtypes.Float64DType, # type: ignore
),
)

except ImportError:
from cudf.utils.dtypes import is_categorical_dtype
from pandas.api.types import is_bool_dtype

# Work around https://github.com/dmlc/xgboost/issues/10181
if _is_cudf_ser(data):
if is_bool_dtype(data.dtype):
data = data.astype(np.uint8)
else:
data = data.astype(
{col: np.uint8 for col in data.select_dtypes(include="bool")}
)

if _is_cudf_ser(data):
dtypes = [data.dtype]
Expand Down
4 changes: 2 additions & 2 deletions python-package/xgboost/testing/__init__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -429,8 +429,8 @@ def make_categorical(
categories = np.arange(0, n_categories)
for col in df.columns:
if rng.binomial(1, cat_ratio, size=1)[0] == 1:
df.loc[:, col] = df[col].astype("category")
df.loc[:, col] = df[col].cat.set_categories(categories)
df[col] = df[col].astype("category")
df[col] = df[col].cat.set_categories(categories)
Comment on lines +432 to +433
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Per https://pandas.pydata.org/docs/user_guide/basics.html#astype, we should not mix astype() with loc() accessor, as loc() tries to fit in what we are assigning to the current dtypes. So df.loc[:, col] = df[col].astype("category") would have no effect.


if sparsity > 0.0:
for i in range(n_features):
Expand Down
2 changes: 1 addition & 1 deletion tests/buildkite/conftest.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ set -x

CUDA_VERSION=11.8.0
NCCL_VERSION=2.16.5-1
RAPIDS_VERSION=24.02
RAPIDS_VERSION=24.04
SPARK_VERSION=3.4.0
JDK_VERSION=8
R_VERSION=4.3.2
Expand Down
8 changes: 4 additions & 4 deletions tests/ci_build/Dockerfile.gpu
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,14 +21,14 @@ ENV PATH=/opt/mambaforge/bin:$PATH

# Create new Conda environment with cuDF, Dask, and cuPy
RUN \
conda install -c conda-forge mamba && \
mamba create -n gpu_test -c rapidsai-nightly -c rapidsai -c nvidia -c conda-forge -c defaults \
export NCCL_SHORT_VER=$(echo "$NCCL_VERSION_ARG" | cut -d "-" -f 1) && \
mamba create -y -n gpu_test -c rapidsai -c nvidia -c conda-forge \
python=3.10 cudf=$RAPIDS_VERSION_ARG* rmm=$RAPIDS_VERSION_ARG* cudatoolkit=$CUDA_VERSION_ARG \
nccl>=$(cut -d "-" -f 1 << $NCCL_VERSION_ARG) \
"nccl>=${NCCL_SHORT_VER}" \
dask=2024.1.1 \
dask-cuda=$RAPIDS_VERSION_ARG* dask-cudf=$RAPIDS_VERSION_ARG* cupy \
numpy pytest pytest-timeout scipy scikit-learn pandas matplotlib wheel python-kubernetes urllib3 graphviz hypothesis \
pyspark>=3.4.0 cloudpickle cuda-python && \
"pyspark>=3.4.0" cloudpickle cuda-python && \
mamba clean --all && \
conda run --no-capture-output -n gpu_test pip install buildkite-test-collector

Expand Down
9 changes: 0 additions & 9 deletions tests/python-gpu/test_from_cudf.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -71,15 +71,6 @@ def _test_from_cudf(DMatrixT):
assert dtrain.num_col() == 1
assert dtrain.num_row() == 5

# Boolean is not supported.
X_boolean = cudf.DataFrame({"x": cudf.Series([True, False])})
with pytest.raises(Exception):
dtrain = DMatrixT(X_boolean)

y_boolean = cudf.DataFrame({"x": cudf.Series([True, False, True, True, True])})
with pytest.raises(Exception):
dtrain = DMatrixT(X_boolean, label=y_boolean)


def _test_cudf_training(DMatrixT):
import pandas as pd
Expand Down
Loading