cuDF - GPU DataFrames

📢 cuDF can now be used as a no-code-change accelerator for pandas! To learn more, see here!

cuDF is a GPU DataFrame library for loading joining, aggregating, filtering, and otherwise manipulating data. cuDF leverages libcudf, a blazing-fast C++/CUDA dataframe library and the Apache Arrow columnar format to provide a GPU-accelerated pandas API.

You can import cudf directly and use it like pandas:

import cudf
import requests
from io import StringIO

url = "https://github.com/plotly/datasets/raw/master/tips.csv"
content = requests.get(url).content.decode("utf-8")

tips_df = cudf.read_csv(StringIO(content))
tips_df["tip_percentage"] = tips_df["tip"] / tips_df["total_bill"] * 100

# display average tip by dining party size
print(tips_df.groupby("size").tip_percentage.mean())

Or, you can use cuDF as a no-code-change accelerator for pandas, using cudf.pandas. cudf.pandas supports 100% of the pandas API, utilizing cuDF for supported operations and falling back to pandas when needed:

%load_ext cudf.pandas  # pandas operations now use the GPU!

import pandas as pd
import requests
from io import StringIO

url = "https://github.com/plotly/datasets/raw/master/tips.csv"
content = requests.get(url).content.decode("utf-8")

tips_df = pd.read_csv(StringIO(content))
tips_df["tip_percentage"] = tips_df["tip"] / tips_df["total_bill"] * 100

# display average tip by dining party size
print(tips_df.groupby("size").tip_percentage.mean())

Resources

• Try cudf.pandas now: Explore cudf.pandas on a free GPU enabled instance on Google Colab!
• Install: Instructions for installing cuDF and other RAPIDS libraries.
• cudf (Python) documentation
• libcudf (C++/CUDA) documentation
• RAPIDS Community: Get help, contribute, and collaborate.

Installation

CUDA/GPU requirements

• CUDA 11.2+
• NVIDIA driver 450.80.02+
• Pascal architecture or better (Compute Capability >=6.0)

Conda

cuDF can be installed with conda (via miniconda or the full Anaconda distribution) from the rapidsai channel:

conda install -c rapidsai -c conda-forge -c nvidia \
    cudf=23.12 python=3.10 cuda-version=11.8

We also provide nightly Conda packages built from the HEAD of our latest development branch.

Note: cuDF is supported only on Linux, and with Python versions 3.9 and later.

See the Get RAPIDS version picker for more OS and version info.

Build/Install from Source

See build instructions.

Contributing

Please see our guide for contributing to cuDF.

Contact

Find out more details on the RAPIDS site

Open GPU Data Science

The RAPIDS suite of open source software libraries aim to enable execution of end-to-end data science and analytics pipelines entirely on GPUs. It relies on NVIDIA® CUDA® primitives for low-level compute optimization, but exposing that GPU parallelism and high-bandwidth memory speed through user-friendly Python interfaces.

Apache Arrow on GPU

The GPU version of Apache Arrow is a common API that enables efficient interchange of tabular data between processes running on the GPU. End-to-end computation on the GPU avoids unnecessary copying and converting of data off the GPU, reducing compute time and cost for high-performance analytics common in artificial intelligence workloads. As the name implies, cuDF uses the Apache Arrow columnar data format on the GPU. Currently, a subset of the features in Apache Arrow are supported.