[BUG] Merging on categorical variables with mismatched ordering is ambiguous

[RitikParmar]

Describe the bug
Grouping query fails with "Merging on categorical variables with mismatched ordering is ambiguous" error. This error is the same as #8200 but now with the most updated version of cudf and rapids to date. The source for this code is from https://github.com/h2oai/db-benchmark.

Steps/Code to reproduce bug
Follow this guide http://matthewrocklin.com/blog/work/2018/02/28/minimal-bug-reports to craft a minimal bug report. This helps us reproduce the issue you're having and resolve the issue more quickly.

• generate data

wget https://raw.githubusercontent.com/h2oai/db-benchmark/master/_data/groupby-datagen.R

install data.table if needed, then generate data using script

Rscript groupby-datagen.R 1e7 1e2 0 0
-run code

#!/usr/bin/env python
print("# groupby-cudf.py", flush=True)
import os
import gc
import timeit
import cudf as cu
import dask_cudf as dc

exec(open("./_helpers/helpers.py").read())

ver = cu.version.split("+", 1)[0]
git = ""
task = "groupby"
solution = "cudf"
fun = ".groupby"
cache = "TRUE"

data_name = os.environ['SRC_DATANAME']
src_grp = os.path.join("data", data_name+".csv")
print("loading dataset %s" % data_name, flush=True)

na_flag = int(data_name.split("_")[3])
if na_flag > 0:
print("skip due to na_flag>0: #221", flush=True, file=sys.stderr)
exit(0) # not yet implemented #221

on_vmem = data_name.split("_")[1] == "1e7" # spilling vmem to mem
on_disk = not(on_vmem) # no really disk, just variable name used to log in script below
print("using video and main memory data storage" if on_disk else "using only video memory data storage", flush=True)
if on_disk:
cu.set_allocator("managed")

x = dc.read_csv(src_grp, header=0, dtype=['str','str','str','int32','int32','int32','int32','int32','float64'])
x['id1'] = x['id1'].astype('category')
x['id2'] = x['id2'].astype('category')
x['id3'] = x['id3'].astype('category')
x = x.persist()
in_rows = len(x.index)
print(in_rows, flush=True)
1,1 Top
question = "sum v1 by id1" # q1
gc.collect()
t_start = timeit.default_timer()
ans = x.groupby('id1', as_index=False, dropna=False).agg({'v1':'sum'}).compute()
print(ans.shape, flush=True)
t = timeit.default_timer() - t_start
m = memory_usage()
t_start = timeit.default_timer()
chk = [ans['v1'].sum()]
chkt = timeit.default_timer() - t_start
write_log(task=task, data=data_name, in_rows=in_rows, question=question, out_rows=ans.shape[0], out_cols=ans.shape[1], solution=solution, version=ver, git=git, fun=fun, run=1, time_sec=t, mem_gb=m, cache=cache, chk=make_chk(chk), chk_time_sec=chkt, on_disk=on_disk)
del ans
gc.collect()
t_start = timeit.default_timer()
ans = x.groupby('id1', as_index=False, dropna=False).agg({'v1':'sum'}).compute()
print(ans.shape, flush=True)
t = timeit.default_timer() - t_start
m = memory_usage()
t_start = timeit.default_timer()
chk = [ans['v1'].sum()]
chkt = timeit.default_timer() - t_start
write_log(task=task, data=data_name, in_rows=in_rows, question=question, out_rows=ans.shape[0], out_cols=ans.shape[1], solution=solution, version=ver, git=git, fun=fun, run=2, time_sec=t, mem_gb=m, cache=cache, chk=make_chk(chk), chk_time_sec=chkt, on_disk=on_disk)
print(ans.head(3), flush=True)
print(ans.tail(3), flush=True)
del ans

Expected behavior
Query should complete successfully

Environment overview (please complete the following information)

• Environment location: Bare-metal
• Method of cuDF install: conda

Environment details

OS Information
NAME="Ubuntu"
VERSION="20.04.3 LTS (Focal Fossa)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 20.04.3 LTS"
VERSION_ID="20.04"

CPU
Architecture: x86_64

packages in environment at /home/.../rapids-22.06:

Name Version Build Channel

_libgcc_mutex 0.1 conda_forge conda-forge
_openmp_mutex 4.5 2_gnu conda-forge
abseil-cpp 20211102.0 hd4dd3e8_0
aiohttp 3.8.1 py39h7f8727e_1
aiosignal 1.2.0 pyhd3eb1b0_0
anyio 3.5.0 py39h06a4308_0
appdirs 1.4.4 pyhd3eb1b0_0
argon2-cffi 21.3.0 pyhd3eb1b0_0
argon2-cffi-bindings 21.2.0 py39h7f8727e_0
arrow-cpp 7.0.0 py39h0f417f0_8_cuda conda-forge
arrow-cpp-proc 3.0.0 cuda conda-forge
asttokens 2.0.5 pyhd3eb1b0_0
async-timeout 4.0.1 pyhd3eb1b0_0
attrs 21.4.0 pyhd3eb1b0_0
aws-c-cal 0.5.11 h95a6274_0 conda-forge
aws-c-common 0.6.2 h7f98852_0 conda-forge
aws-c-event-stream 0.2.7 h3541f99_13 conda-forge
aws-c-io 0.10.5 hfb6a706_0 conda-forge
aws-checksums 0.1.11 ha31a3da_7 conda-forge
aws-sdk-cpp 1.8.186 hb4091e7_3 conda-forge
backcall 0.2.0 pyhd3eb1b0_0
beautifulsoup4 4.11.1 py39h06a4308_0
bleach 4.1.0 pyhd3eb1b0_0
blosc 1.21.0 h4ff587b_1
bokeh 2.4.2 py39h06a4308_1
boost 1.74.0 py39h5472131_5 conda-forge
boost-cpp 1.74.0 h312852a_4 conda-forge
brotli 1.0.9 he6710b0_2
brotlipy 0.7.0 py39h27cfd23_1003
brunsli 0.1 h2531618_0
bzip2 1.0.8 h7b6447c_0
c-ares 1.18.1 h7f8727e_0
c-blosc2 2.2.0 h7a311fb_0 conda-forge
ca-certificates 2022.4.26 h06a4308_0
cachetools 4.2.2 pyhd3eb1b0_0
cairo 1.16.0 h6cf1ce9_1008 conda-forge
certifi 2022.6.15 py39h06a4308_0
cffi 1.15.0 py39hd667e15_1
cfitsio 3.470 h5893167_7
charls 2.2.0 h2531618_0
charset-normalizer 2.0.4 pyhd3eb1b0_0
click 8.0.4 py39h06a4308_0
click-plugins 1.1.1 pyhd3eb1b0_0
cligj 0.7.2 py39h06a4308_0
cloudpickle 2.0.0 pyhd3eb1b0_0
colorcet 3.0.0 py39h06a4308_0
cryptography 37.0.1 py39h9ce1e76_0
cucim 22.06.00 cuda_11_py39_g575240a_0 rapidsai
cuda-python 11.7.0 py39h3fd9d12_0 nvidia
cudatoolkit 11.4.1 h8ab8bb3_9 nvidia
cudf 22.06.01 cuda_11_py39_g0dab0f8305_0 rapidsai
cudf_kafka 22.06.01 py39_g0dab0f8305_0 rapidsai
cugraph 22.06.00 cuda11_py39_gfb99d2d7_0 rapidsai
cuml 22.06.01 cuda11_py39_ge38b5bdd5_0 rapidsai
cupy 9.6.0 py39ha91d172_0 conda-forge
curl 7.83.1 h7bff187_0 conda-forge
cusignal 22.06.00 py39_g7660efa_0 rapidsai
cuspatial 22.06.00 py39_g96613ba_0 rapidsai
custreamz 22.06.01 py39_g0dab0f8305_0 rapidsai
cuxfilter 22.06.00 py39_gda2f3f6_0 rapidsai
cycler 0.11.0 pyhd3eb1b0_0
cyrus-sasl 2.1.27 h9c0eb46_9
cytoolz 0.11.0 py39h27cfd23_0
dask 2022.5.2 pyhd8ed1ab_0 conda-forge
dask-core 2022.5.2 pyhd8ed1ab_0 conda-forge
dask-cuda 22.06.00 py39_0 rapidsai
dask-cudf 22.06.01 cuda_11_py39_g0dab0f8305_0 rapidsai
datashader 0.13.1a py_0 rapidsai
datashape 0.5.4 py39h06a4308_1
debugpy 1.5.1 py39h295c915_0
decorator 5.1.1 pyhd3eb1b0_0
defusedxml 0.7.1 pyhd3eb1b0_0
distributed 2022.5.2 pyhd8ed1ab_0 conda-forge
dlpack 0.5 h9c3ff4c_0 conda-forge
entrypoints 0.4 py39h06a4308_0
executing 0.8.3 pyhd3eb1b0_0
expat 2.4.8 h27087fc_0 conda-forge
faiss-proc 1.0.0 cuda rapidsai
fastavro 1.4.7 py39h7f8727e_0
fastrlock 0.6 py39h2531618_0
fiona 1.8.20 py39h427c1bf_2 conda-forge
font-ttf-dejavu-sans-mono 2.37 hd3eb1b0_0
font-ttf-inconsolata 2.001 hcb22688_0
font-ttf-source-code-pro 2.030 hd3eb1b0_0
font-ttf-ubuntu 0.83 h8b1ccd4_0
fontconfig 2.14.0 h8e229c2_0 conda-forge
fonts-anaconda 1 h8fa9717_0
fonts-conda-ecosystem 1 hd3eb1b0_0
fonttools 4.25.0 pyhd3eb1b0_0
freetype 2.11.0 h70c0345_0
freexl 1.0.6 h27cfd23_0
frozenlist 1.2.0 py39h7f8727e_0
fsspec 2022.3.0 py39h06a4308_0
gdal 3.3.2 py39h218ed2d_3 conda-forge
geopandas 0.9.0 py_1
geopandas-base 0.9.0 py_1
geos 3.9.1 h295c915_0
geotiff 1.7.0 h08e826d_2 conda-forge
gettext 0.21.0 hf68c758_0
gflags 2.2.2 he6710b0_0
giflib 5.2.1 h7b6447c_0
glog 0.6.0 h6f12383_0 conda-forge
grpc-cpp 1.46.3 hc275302_1 conda-forge
hdf4 4.2.15 h10796ff_3 conda-forge
hdf5 1.12.1 nompi_h2386368_104 conda-forge
heapdict 1.0.1 pyhd3eb1b0_0
holoviews 1.14.6 pyhd8ed1ab_0 conda-forge
icu 68.1 h2531618_0
idna 3.3 pyhd3eb1b0_0
imagecodecs 2021.8.26 py39h44211f0_1 conda-forge
imageio 2.9.0 pyhd3eb1b0_0
importlib-metadata 4.11.3 py39h06a4308_0
ipykernel 6.9.1 py39h06a4308_0
ipython 8.4.0 py39h06a4308_0
ipython_genutils 0.2.0 pyhd3eb1b0_1
ipywidgets 7.6.5 pyhd3eb1b0_1
jbig 2.1 hdba287a_0
jedi 0.18.1 py39h06a4308_1
jinja2 3.0.3 pyhd3eb1b0_0
joblib 1.1.0 pyhd3eb1b0_0
jpeg 9e h7f8727e_0
json-c 0.15 h98cffda_0 conda-forge
jsonschema 4.4.0 py39h06a4308_0
jupyter-server-proxy 3.2.1 pyhd8ed1ab_0 conda-forge
jupyter_client 7.2.2 py39h06a4308_0
jupyter_core 4.10.0 py39h06a4308_0
jupyter_server 1.17.1 py39h06a4308_0
jupyterlab_pygments 0.1.2 py_0
jupyterlab_widgets 1.0.0 pyhd3eb1b0_1
jxrlib 1.1 h7b6447c_2
kealib 1.4.15 hfe1a663_0 conda-forge
keyutils 1.6.1 h166bdaf_0 conda-forge
kiwisolver 1.4.2 py39h295c915_0
krb5 1.19.3 h3790be6_0 conda-forge
lcms2 2.12 h3be6417_0
ld_impl_linux-64 2.38 h1181459_1
lerc 3.0 h295c915_0
libaec 1.0.6 h9c3ff4c_0 conda-forge
libblas 3.9.0 15_linux64_openblas conda-forge
libbrotlicommon 1.0.9 h166bdaf_7 conda-forge
libbrotlidec 1.0.9 h166bdaf_7 conda-forge
libbrotlienc 1.0.9 h166bdaf_7 conda-forge
libcblas 3.9.0 15_linux64_openblas conda-forge
libcrc32c 1.1.2 h9c3ff4c_0 conda-forge
libcucim 22.06.00 cuda11_g575240a_0 rapidsai
libcudf 22.06.01 cuda11_g0dab0f8305_0 rapidsai
libcudf_kafka 22.06.01 g0dab0f8305_0 rapidsai
libcugraph 22.06.00 cuda11_gfb99d2d7_0 rapidsai
libcugraph_etl 22.06.00 cuda11_gfb99d2d7_0 rapidsai
libcugraphops 22.06.00 cuda11_gb4e1cbb_0 nvidia
libcuml 22.06.01 cuda11_ge38b5bdd5_0 rapidsai
libcumlprims 22.06.00 cuda11_g8c6e2c0_0 nvidia
libcurl 7.83.1 h7bff187_0 conda-forge
libcusolver 11.3.5.50 hcab339c_0 nvidia
libcuspatial 22.06.00 cuda11_g96613ba_0 rapidsai
libdap4 3.20.6 hd7c4107_2 conda-forge
libdeflate 1.8 h7f8727e_5
libedit 3.1.20210910 h7f8727e_0
libev 4.33 h7f8727e_1
libevent 2.1.10 h9b69904_4 conda-forge
libfaiss 1.7.0 cuda114hda32b58_8_cuda rapidsai
libffi 3.4.2 h295c915_4
libgcc-ng 12.1.0 h8d9b700_16 conda-forge
libgcrypt 1.9.3 h27cfd23_0
libgdal 3.3.2 h6acdded_3 conda-forge
libgfortran-ng 11.2.0 h00389a5_1
libgfortran5 11.2.0 h1234567_1
libglib 2.72.1 h2d90d5f_0 conda-forge
libgomp 12.1.0 h8d9b700_16 conda-forge
libgoogle-cloud 1.40.2 hefc27d0_0 conda-forge
libgpg-error 1.42 h2531618_0
libgsasl 1.10.0 h7f8727e_0
libiconv 1.16 h7f8727e_2
libkml 1.3.0 h238a007_1014 conda-forge
liblapack 3.9.0 15_linux64_openblas conda-forge
libllvm11 11.1.0 h3826bc1_1
libnetcdf 4.8.1 nompi_h329d8a1_102 conda-forge
libnghttp2 1.47.0 h727a467_0 conda-forge
libnsl 2.0.0 h7f98852_0 conda-forge
libntlm 1.6 h7f8727e_0
libopenblas 0.3.20 h043d6bf_1
libpng 1.6.37 hbc83047_0
libpq 13.5 hd57d9b9_1 conda-forge
libprotobuf 3.20.1 h4ff587b_0
libraft-distance 22.06.00 cuda11_ged2c529_0 rapidsai
libraft-headers 22.06.00 cuda11_ged2c529_0 rapidsai
libraft-nn 22.06.00 cuda11_ged2c529_0 rapidsai
librdkafka 1.7.0 hc49e61c_1 conda-forge
librmm 22.06.01 cuda11_g1c7ff61a_0 rapidsai
librttopo 1.1.0 h1185371_6 conda-forge
libsodium 1.0.18 h7b6447c_0
libspatialindex 1.9.3 h2531618_0
libspatialite 5.0.1 h5cf074c_8 conda-forge
libssh2 1.10.0 h8f2d780_0
libstdcxx-ng 12.1.0 ha89aaad_16 conda-forge
libthrift 0.16.0 h519c5ea_1 conda-forge
libtiff 4.3.0 h6f004c6_2 conda-forge
libutf8proc 2.7.0 h7f98852_0 conda-forge
libuuid 2.32.1 h7f98852_1000 conda-forge
libuv 1.40.0 h7b6447c_0
libwebp 1.2.2 h55f646e_0
libwebp-base 1.2.2 h7f8727e_0
libxcb 1.15 h7f8727e_0
libxgboost 1.6.0dev.rapidsai22.06 cuda_11_0 rapidsai
libxml2 2.9.12 h72842e0_0 conda-forge
libzip 1.8.0 h5cef20c_0
libzlib 1.2.12 h166bdaf_1 conda-forge
libzopfli 1.0.3 he6710b0_0
llvmlite 0.38.1 py39h7d9a04d_0 conda-forge
locket 1.0.0 py39h06a4308_0
lz4 3.1.3 py39h27cfd23_0
lz4-c 1.9.3 h295c915_1
mapclassify 2.4.3 pyhd3eb1b0_0
markdown 3.3.4 py39h06a4308_0
markupsafe 2.1.1 py39h7f8727e_0
matplotlib-base 3.5.1 py39ha18d171_1
matplotlib-inline 0.1.2 pyhd3eb1b0_2
mistune 0.8.4 py39h27cfd23_1000
msgpack-python 1.0.3 py39hd09550d_0
multidict 5.2.0 py39h7f8727e_2
multipledispatch 0.6.0 py39h06a4308_0
munch 2.5.0 pyhd3eb1b0_0
munkres 1.1.4 py_0
nbclient 0.5.13 py39h06a4308_0
nbconvert 6.4.4 py39h06a4308_0
nbformat 5.3.0 py39h06a4308_0
nccl 2.12.12.1 h0800d71_0 conda-forge
ncurses 6.3 h5eee18b_3
nest-asyncio 1.5.5 py39h06a4308_0
networkx 2.6.3 pyhd8ed1ab_1 conda-forge
nodejs 14.8.0 hda19d22_0
notebook 6.4.11 py39h06a4308_0
nspr 4.33 h295c915_0
nss 3.74 h0370c37_0
numba 0.55.2 py39h66db6d7_0 conda-forge
numpy 1.22.4 py39hc58783e_0 conda-forge
nvtx 0.2.3 py39h3811e60_1 conda-forge
openjpeg 2.4.0 h3ad879b_0
openssl 1.1.1q h7f8727e_0
orc 1.7.5 h6c59b99_0 conda-forge
packaging 21.3 pyhd3eb1b0_0
pandas 1.4.3 py39h1832856_0 conda-forge
pandocfilters 1.5.0 pyhd3eb1b0_0
panel 0.12.7 pyhd3eb1b0_0
param 1.12.0 pyhd3eb1b0_0
parquet-cpp 1.5.1 h34088ae_4
parso 0.8.3 pyhd3eb1b0_0
partd 1.2.0 pyhd3eb1b0_1
pcre 8.45 h295c915_0
pexpect 4.8.0 pyhd3eb1b0_3
pickleshare 0.7.5 pyhd3eb1b0_1003
pillow 9.2.0 py39hace64e9_0
pip 22.1.2 py39h06a4308_0
pixman 0.40.0 h7f8727e_1
poppler 21.09.0 ha39eefc_3 conda-forge
poppler-data 0.4.11 h06a4308_0
postgresql 13.5 h2510834_1 conda-forge
proj 8.1.0 h277dcde_1 conda-forge
prometheus_client 0.13.1 pyhd3eb1b0_0
prompt-toolkit 3.0.20 pyhd3eb1b0_0
protobuf 3.20.1 py39h295c915_0
psutil 5.9.0 py39h5eee18b_0
ptxcompiler 0.2.0 py39h107f55c_0 rapidsai
ptyprocess 0.7.0 pyhd3eb1b0_2
pure_eval 0.2.2 pyhd3eb1b0_0
py-xgboost 1.6.0dev.rapidsai22.06 cuda_11_py39_0 rapidsai
pyarrow 7.0.0 py39h1ed2e5d_8_cuda conda-forge
pycparser 2.21 pyhd3eb1b0_0
pyct 0.4.6 py39h06a4308_0
pydeck 0.5.0 pyh9f0ad1d_0 conda-forge
pyee 8.1.0 pyhd8ed1ab_0 conda-forge
pygments 2.11.2 pyhd3eb1b0_0
pylibcugraph 22.06.00 cuda11_py39_gfb99d2d7_0 rapidsai
pynvml 11.4.1 pyhd8ed1ab_0 conda-forge
pyopenssl 22.0.0 pyhd3eb1b0_0
pyparsing 3.0.4 pyhd3eb1b0_0
pyppeteer 1.0.2 pyhd8ed1ab_0 conda-forge
pyproj 3.1.0 py39h1fd097d_4 conda-forge
pyraft 22.06.00 cuda11_py39_ged2c529_0 rapidsai
pyrsistent 0.18.0 py39heee7806_0
pysocks 1.7.1 py39h06a4308_0
python 3.9.13 h9a8a25e_0_cpython conda-forge
python-confluent-kafka 1.7.0 py39h3811e60_2 conda-forge
python-dateutil 2.8.2 pyhd3eb1b0_0
python-fastjsonschema 2.15.1 pyhd3eb1b0_0
python_abi 3.9 2_cp39 conda-forge
pytz 2022.1 py39h06a4308_0
pyviz_comms 2.0.2 pyhd3eb1b0_0
pywavelets 1.3.0 py39h7f8727e_0
pyyaml 6.0 py39h7f8727e_1
pyzmq 23.2.0 py39h6a678d5_0
rapids 22.06.00 cuda11_py39_ged167d1_108 rapidsai
rapids-xgboost 22.06.00 cuda11_py39_ged167d1_108 rapidsai
re2 2022.04.01 h295c915_0
readline 8.1.2 h7f8727e_1
requests 2.28.1 py39h06a4308_0
rmm 22.06.01 cuda11_py39_g1c7ff61a_0 rapidsai
rtree 0.9.7 py39h06a4308_1
s2n 1.0.10 h9b69904_0 conda-forge
scikit-image 0.19.2 py39h51133e4_0
scikit-learn 1.0.2 py39h51133e4_1
scipy 1.8.1 py39he49c0e8_0 conda-forge
send2trash 1.8.0 pyhd3eb1b0_1
setuptools 60.10.0 py39hf3d152e_0 conda-forge
shapely 1.8.0 py39ha61afbd_0 conda-forge
simpervisor 0.4 pyhd8ed1ab_0 conda-forge
six 1.16.0 pyhd3eb1b0_1
snappy 1.1.9 h295c915_0
sniffio 1.2.0 py39h06a4308_1
sortedcontainers 2.4.0 pyhd3eb1b0_0
soupsieve 2.3.1 pyhd3eb1b0_0
spdlog 1.8.5 h4bd325d_1 conda-forge
sqlite 3.38.5 hc218d9a_0
stack_data 0.2.0 pyhd3eb1b0_0
streamz 0.6.3 pyhd3eb1b0_0
tblib 1.7.0 pyhd3eb1b0_0
terminado 0.13.1 py39h06a4308_0
testpath 0.6.0 py39h06a4308_0
threadpoolctl 2.2.0 pyh0d69192_0
tifffile 2021.7.2 pyhd3eb1b0_2
tiledb 2.3.4 he87e0bf_0 conda-forge
tk 8.6.12 h1ccaba5_0
toolz 0.11.2 pyhd3eb1b0_0
tornado 6.1 py39h27cfd23_0
tqdm 4.64.0 py39h06a4308_0
traitlets 5.1.1 pyhd3eb1b0_0
treelite 2.4.0 py39heaea588_0 conda-forge
treelite-runtime 2.4.0 pypi_0 pypi
typing-extensions 4.1.1 hd3eb1b0_0
typing_extensions 4.1.1 pyh06a4308_0
tzcode 2022a h166bdaf_0 conda-forge
tzdata 2022a hda174b7_0
ucx 1.12.1 h7a399c7_1 conda-forge
ucx-proc 1.0.0 gpu rapidsai
ucx-py 0.26.00 py39_g85a4428_0 rapidsai
urllib3 1.26.9 py39h06a4308_0
wcwidth 0.2.5 pyhd3eb1b0_0
webencodings 0.5.1 py39h06a4308_1
websocket-client 0.58.0 py39h06a4308_4
websockets 10.3 py39hb9d737c_0 conda-forge
wheel 0.37.1 pyhd3eb1b0_0
widgetsnbextension 3.5.2 py39h06a4308_0
xarray 0.20.1 pyhd3eb1b0_1
xerces-c 3.2.3 h9d8b166_3 conda-forge
xgboost 1.6.0dev.rapidsai22.06 cuda_11_py39_0 rapidsai
xorg-kbproto 1.0.7 h7f98852_1002 conda-forge
xorg-libice 1.0.10 h7f98852_0 conda-forge
xorg-libsm 1.2.3 hd9c2040_1000 conda-forge
xorg-libx11 1.7.2 h7f98852_0 conda-forge
xorg-libxext 1.3.4 h7f98852_1 conda-forge
xorg-libxrender 0.9.10 h7f98852_1003 conda-forge
xorg-renderproto 0.11.1 h7f98852_1002 conda-forge
xorg-xextproto 7.3.0 h7f98852_1002 conda-forge
xorg-xproto 7.0.31 h27cfd23_1007
xz 5.2.5 h7f8727e_1
yaml 0.2.5 h7b6447c_0
yarl 1.6.3 py39h27cfd23_0
zeromq 4.3.4 h2531618_0
zfp 0.5.5 h295c915_6
zict 2.1.0 py39h06a4308_0
zipp 3.8.0 py39h06a4308_0
zlib 1.2.12 h166bdaf_1 conda-forge
zlib-ng 2.0.6 h166bdaf_0 conda-forge
zstd 1.5.2 ha4553b6_0

Additional context
#8332 added support for this issue, but this was on a much older version of cudf, so I am not sure if this support still exists.

[shwina]

Hi @RitikParmar - thanks for reporting. Are you able to provide the stack trace for the failure? It seems like somewhere along the way you end up with two categorical columns, one ordered and one unordered, and cuDF is unable to merge on them. You may have to cast them both to be ordered (or unordered).

[RitikParmar]

File "/home/ritik/anaconda3/envs/rapids-22.06/lib/python3.9/site-packages/dask/core.py", line 149, in get
result = _execute_task(task, cache)
File "/home/ritik/anaconda3/envs/rapids-22.06/lib/python3.9/site-packages/dask/core.py", line 119, in _execute_task
return func(*(_execute_task(a, cache) for a in args))
File "/home/ritik/anaconda3/envs/rapids-22.06/lib/python3.9/site-packages/dask/utils.py", line 41, in apply
return func(*args, **kwargs)
File "/home/ritik/anaconda3/envs/rapids-22.06/lib/python3.9/site-packages/dask/dataframe/groupby.py", line 958, in _groupby_apply_funcs
return df.class(result)
File "/home/ritik/anaconda3/envs/rapids-22.06/lib/python3.9/contextlib.py", line 79, in inner
return func(*args, **kwds)
File "/home/ritik/anaconda3/envs/rapids-22.06/lib/python3.9/site-packages/cudf/core/dataframe.py", line 730, in init
self._init_from_dict_like(
File "/home/ritik/anaconda3/envs/rapids-22.06/lib/python3.9/contextlib.py", line 79, in inner
return func(*args, **kwds)
File "/home/ritik/anaconda3/envs/rapids-22.06/lib/python3.9/site-packages/cudf/core/dataframe.py", line 913, in _init_from_dict_like
data, index = self._align_input_series_indices(data, index=index)
File "/home/ritik/anaconda3/envs/rapids-22.06/lib/python3.9/contextlib.py", line 79, in inner
return func(*args, **kwds)
File "/home/ritik/anaconda3/envs/rapids-22.06/lib/python3.9/site-packages/cudf/core/dataframe.py", line 977, in _align_input_series_indices
aligned_input_series = cudf.core.series._align_indices(
File "/home/ritik/anaconda3/envs/rapids-22.06/lib/python3.9/contextlib.py", line 79, in inner
return func(*args, **kwds)
File "/home/ritik/anaconda3/envs/rapids-22.06/lib/python3.9/site-packages/cudf/core/series.py", line 4608, in _align_indices
result = [
File "/home/ritik/anaconda3/envs/rapids-22.06/lib/python3.9/site-packages/cudf/core/series.py", line 4609, in
sr._align_to_index(
File "/home/ritik/anaconda3/envs/rapids-22.06/lib/python3.9/site-packages/cudf/core/indexed_frame.py", line 1277, in _align_to_index
result = lhs.join(rhs, how=how, sort=sort)
File "/home/ritik/anaconda3/envs/rapids-22.06/lib/python3.9/contextlib.py", line 79, in inner
return func(*args, **kwds)
File "/home/ritik/anaconda3/envs/rapids-22.06/lib/python3.9/site-packages/cudf/core/dataframe.py", line 3525, in join
df = self.merge(
File "/home/ritik/anaconda3/envs/rapids-22.06/lib/python3.9/contextlib.py", line 79, in inner
return func(*args, **kwds)
File "/home/ritik/anaconda3/envs/rapids-22.06/lib/python3.9/site-packages/cudf/core/dataframe.py", line 3474, in merge
gdf_result = super()._merge(
File "/home/ritik/anaconda3/envs/rapids-22.06/lib/python3.9/contextlib.py", line 79, in inner
return func(*args, **kwds)
File "/home/ritik/anaconda3/envs/rapids-22.06/lib/python3.9/site-packages/cudf/core/frame.py", line 2278, in _merge
return merge_cls(
File "/home/ritik/anaconda3/envs/rapids-22.06/lib/python3.9/site-packages/cudf/core/join/join.py", line 186, in perform_merge
lcol_casted, rcol_casted = _match_join_keys(lcol, rcol, self.how)
File "/home/ritik/anaconda3/envs/rapids-22.06/lib/python3.9/site-packages/cudf/core/join/_join_helpers.py", line 74, in _match_join_keys
return _match_categorical_dtypes_both(
File "/home/ritik/anaconda3/envs/rapids-22.06/lib/python3.9/site-packages/cudf/core/join/_join_helpers.py", line 132, in _match_categorical_dtypes_both
raise TypeError(
TypeError: Merging on categorical variables with mismatched ordering is ambiguous
on_disk question run time_sec
1 False sum v1 by id1 1 0.106
2 False sum v1 by id1 2 0.034
3 False sum v1 by id1:id2 1 0.074
4 False sum v1 by id1:id2 2 0.073
5 False sum v1 mean v3 by id3 1 0.098
6 False sum v1 mean v3 by id3 2 0.085
7 False mean v1:v3 by id4 1 0.037
8 False mean v1:v3 by id4 2 0.038
9 False sum v1:v3 by id6 1 0.065
10 False sum v1:v3 by id6 2 0.065

[RitikParmar]

I updated my environment to rapids-22.08 which solved this issue.