[BUG] High overhead

[lmeyerov]

Describe the bug
Seeing high overhead (3ms+) on most calls that should take < 1ms. For software that chains together many RAPIDS expressions or makes concurrent RAPIDS calls, this prevents subsecond execution on both small & big data. Both are common in visual software: complex UI logic, or many concurrent UI components. Ex: One of our histogram helpers can already easily cover 100 expressions, and we'd like to run it 30-300 times for any UI state.

Steps/Code to reproduce bug
See Issue 1 of https://colab.research.google.com/drive/1S5UhplicI4gpEIJz0jQdhGd3gyBAYbee#scrollTo=HASMl-ZxWH4H

Expected behavior
All of the calls should have taken < 1ms.

Environment details (please complete the following information):

• !conda install -q -y --prefix /usr/local -c conda-forge
  -c rapidsai-nightly/label/cuda10.0 -c nvidia/label/cuda10.0
  cudf cuml => 0.7.0.dev0
• Environment location: Colab
• Method of cuDF install: Conda
• Please run and attach the output of the cudf/print_env.sh script to gather relevant environment details
  git*
  Not inside a git repository

OS Information
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=18.04
DISTRIB_CODENAME=bionic
DISTRIB_DESCRIPTION="Ubuntu 18.04.2 LTS"
NAME="Ubuntu"
VERSION="18.04.2 LTS (Bionic Beaver)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 18.04.2 LTS"
VERSION_ID="18.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=bionic
UBUNTU_CODENAME=bionic
Linux fe4dd8671d4c 4.14.79+ #1 SMP Wed Dec 19 21:19:13 PST 2018 x86_64 x86_64 x86_64 GNU/Linux

GPU Information
Wed May 8 00:28:11 2019
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 418.56 Driver Version: 410.79 CUDA Version: 10.0 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
|===============================+======================+======================|
| 0 Tesla T4 Off | 00000000:00:04.0 Off | 0 |
| N/A 73C P0 31W / 70W | 489MiB / 15079MiB | 0% Default |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes: GPU Memory |
| GPU PID Type Process name Usage |
|=============================================================================|
+-----------------------------------------------------------------------------+

CPU
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Byte Order: Little Endian
CPU(s): 2
On-line CPU(s) list: 0,1
Thread(s) per core: 2
Core(s) per socket: 1
Socket(s): 1
NUMA node(s): 1
Vendor ID: GenuineIntel
CPU family: 6
Model: 63
Model name: Intel(R) Xeon(R) CPU @ 2.30GHz
Stepping: 0
CPU MHz: 2300.000
BogoMIPS: 4600.00
Hypervisor vendor: KVM
Virtualization type: full
L1d cache: 32K
L1i cache: 32K
L2 cache: 256K
L3 cache: 46080K
NUMA node0 CPU(s): 0,1
Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc rep_good nopl xtopology nonstop_tsc cpuid tsc_known_freq pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt aes xsave avx f16c rdrand hypervisor lahf_lm abm invpcid_single pti ssbd ibrs ibpb stibp fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid xsaveopt arat arch_capabilities

CMake
/usr/local/bin/cmake
cmake version 3.12.0

CMake suite maintained and supported by Kitware (kitware.com/cmake).

g++
/usr/bin/g++
g++ (Ubuntu 7.3.0-27ubuntu1~18.04) 7.3.0
Copyright (C) 2017 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

nvcc
/usr/local/cuda/bin/nvcc
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2018 NVIDIA Corporation
Built on Sat_Aug_25_21:08:01_CDT_2018
Cuda compilation tools, release 10.0, V10.0.130

Python
/usr/local/bin/python
Python 3.6.7

Environment Variables
PATH : /usr/local/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/tools/node/bin:/tools/google-cloud-sdk/bin:/opt/bin
LD_LIBRARY_PATH : /usr/lib64-nvidia
NUMBAPRO_NVVM : /usr/local/cuda/nvvm/lib64/libnvvm.so
NUMBAPRO_LIBDEVICE : /usr/local/cuda/nvvm/libdevice/
CONDA_PREFIX :
PYTHON_PATH :

conda packages
/usr/local/bin/conda

packages in environment at /usr/local:

Name Version Build Channel

arrow-cpp 0.12.1 py36h0e61e49_0 conda-forge
asn1crypto 0.24.0 py36_0
boost-cpp 1.68.0 h11c811c_1000 conda-forge
bzip2 1.0.6 h14c3975_1002 conda-forge
ca-certificates 2019.3.9 hecc5488_0 conda-forge
certifi 2019.3.9 py36_0 conda-forge
cffi 1.11.5 py36h9745a5d_0
chardet 3.0.4 py36h0f667ec_1
conda 4.6.14 py36_0 conda-forge
conda-env 2.6.0 h36134e3_1
cryptography 2.6.1 py36h72c5cf5_0 conda-forge
cudatoolkit 10.0.130 0
cudf 0.7.0.dev0 py36_1516 rapidsai-nightly/label/cuda10.0
cuml 0.8.0a cuda10.0_py36_539 rapidsai-nightly/label/cuda10.0
icu 58.2 hf484d3e_1000 conda-forge
idna 2.6 py36h82fb2a8_1
libblas 3.8.0 8_openblas conda-forge
libcblas 3.8.0 8_openblas conda-forge
libcudf 0.7.0.dev0 cuda10.0_1514 rapidsai-nightly/label/cuda10.0
libcuml 0.8.0a cuda10.0_539 rapidsai-nightly/label/cuda10.0
libcumlmg 0.0.0.dev0 cuda10.0_373 nvidia/label/cuda10.0
libedit 3.1.20170329 h6b74fdf_2
libffi 3.2.1 hd88cf55_4
libgcc-ng 8.2.0 hdf63c60_1
libgfortran-ng 7.3.0 hdf63c60_0
liblapack 3.8.0 8_openblas conda-forge
libnvstrings 0.7.0.dev0 cuda10.0_130 rapidsai-nightly/label/cuda10.0
libprotobuf 3.6.1 hdbcaa40_1001 conda-forge
librmm 0.7.0.dev0 cuda10.0_48 rapidsai-nightly/label/cuda10.0
libstdcxx-ng 8.2.0 hdf63c60_1
llvmlite 0.28.0 py36hdbcaa40_0 conda-forge
ncurses 6.1 hf484d3e_0
numba 0.43.1 py36hf2d7682_0 conda-forge
numpy 1.16.3 py36he5ce36f_0 conda-forge
nvstrings 0.7.0.dev0 py36_130 rapidsai-nightly/label/cuda10.0
openblas 0.3.6 h6e990d7_1 conda-forge
openssl 1.1.1b h14c3975_1 conda-forge
pandas 0.24.2 py36hf484d3e_0 conda-forge
parquet-cpp 1.5.1 4 conda-forge
pip 10.0.1 py36_0
pyarrow 0.12.1 py36hbbcf98d_0 conda-forge
pycosat 0.6.3 py36h0a5515d_0
pycparser 2.18 py36hf9f622e_1
pyopenssl 18.0.0 py36_0
pysocks 1.6.8 py36_0
python 3.6.7 h381d211_1004 conda-forge
python-dateutil 2.8.0 py_0 conda-forge
pytz 2019.1 py_0 conda-forge
readline 7.0 ha6073c6_4
requests 2.18.4 py36he2e5f8d_1
rmm 0.7.0.dev0 py36_48 rapidsai-nightly/label/cuda10.0
ruamel_yaml 0.15.37 py36h14c3975_2
setuptools 39.2.0 py36_0
six 1.11.0 py36h372c433_1
sqlite 3.26.0 h67949de_1001 conda-forge
thrift-cpp 0.12.0 h0a07b25_1002 conda-forge
tk 8.6.9 h84994c4_1001 conda-forge
urllib3 1.22 py36hbe7ace6_0
wheel 0.31.1 py36_0
xz 5.2.4 h14c3975_4
yaml 0.1.7 had09818_2
zlib 1.2.11 ha838bed_2

[kkraus14]

This was resolved in 0.7 hotfixes.