[BUG] Writing ORC files with KvikIO is 5x slower #12780
Labels
bug
Something isn't working
cuIO
cuIO issue
libcudf
Affects libcudf (C++/CUDA) code.
Performance
Performance related issue
Describe the bug
When running the
orc_write_io_compression
libcudf benchmarks, usingLIBCUDF_CUFILE_POLICY=KVIKIO
causes the benchmarks to run 5x slower.When set to
OFF
orGDS
the runtimes are 500-700 ms, however when set toKVIKIO
the runtimes are >2.5 s.Steps/Code to reproduce bug
Build from source (
./build.sh libcudf benchmarks
) and run:Expected behavior
I expect KVIKIO to be similar or better performance than the libcudf default data sink.
Environment overview (please complete the following information)
I collected these numbers with
3c39be5a9
and docker image0022659d9d65
from rapidsai-dev-nightlyAdditional context
The extra time is spent in the write step. The encoding kernels are not impacted.
KVIKIO
OFF
I also tried using the tuning parameters
KVIKIO_TASK_SIZE
,KVIKIO_NTHREADS
andKVIKIO_COMPAT_MODE
but could not recover performance completely. Increasing the number of threads showed a positive impact.The text was updated successfully, but these errors were encountered: