Find file
Fetching contributors…
Cannot retrieve contributors at this time
162 lines (129 sloc) 5.04 KB
# Gdev - Managing GPUs as First-Class Computing Resources
# Copyright (C) Shinpei Kato
# University of California, Santa Cruz
# Systems Research Lab.
# In collaboration with PathScale Inc.
# All Rights Reserved.
Gdev is a runtime-unified operating system module that manages GPUs
as first-class computing resources.
Currently it supports only NVIDIA's Fermi GPUs, but the concept of
Gdev is also applicable to generic "compute devices".
Gdev coordinates with a DRM-based GPU device driver (pscnv/nouveau)
in the operating system, providing APIs for application programs.
Gdev API is a low-level primitive that allows programmers to control
the details of GPU resource parameters, while Gdev also supports a
high-level API, such as CUDA.
Gdev is available for GPGPU and graphics applications. It is self-
contained for GPGPU, though graphics applications require additional
packages, such as OpenGL, LIBDRM, and DDX.
Gdev is open-source. We believe that this open-source implementation
facilitates further research and development of GPU technology.
The following instruction will tell you how to install Gdev. Some of
the installation stages may require you to install additional tools
and packages.
$(TOPDIR) will represent your top working directory, henceforth.
1. Download
cd $(TOPDIR)
git clone git://
git clone git://
2. envytools
envytools is a rich set of open-source tools to compile or decompile
NVIDIA GPU program code, firmware code, macro code, and so on. It is
also used to generate C header files with GPU command definitions.
In addition, envytools document the NVIDIA GPU architecture details,
while are not disclosed to the public so far. If you are interested
in GPU system software development, this is what you should read!
Please follow the instruction below to install envytools.
cd $(TOPDIR)/envytools
mkdir build
cd build
cmake .. # may require some packages on your distro
sudo make install # will install tools to /usr/local/{bin,lib}
3. Device Driver
Recent versions of Gdev disgregate from the device driver. You need
to install some GPU device driver underlying Gdev. Currently there
are two open-source drivers available in Linux: pscnv and nouveau.
The following shows how to install pscnv.
NOTE1: pscnv is PathScale's open-source driver, but we have applied
several patches, making it available for Gdev.
NOTE2: pscnv is available with Linux 2.6.33 or later. It may also
have issues with some Fermi chipsets - nvc0 is best tested.
See for chipset IDs.
3.1. When you use pscnv
cd $(TOPDIR)/gdev/driver/pscnv
sudo make modules_install
sudo shutdown -r now # will reboot your machine
3.2. When you use nouveau
cd $(TOPDIR)/gdev/driver/nouveau
sudo make install NOUVEAUROOTDIR=.
cd drivers/gpu/drm/nouveau
sudo sh
sudo shutdown -r now # will reboot your machine
If you are not sure if Nouveau is loaded successfully, do:
modprobe -r nouveau; modprobe nouveau modeset=1 noaccel=0
4. Gdev Module
Gdev is double-edge, i.e., it provides a runtime-unified operating
system approach as well as a typical user-space runtime approach.
The following installation is required only for the former.
cd $(TOPDIR)/gdev/driver
mkdir build
cd build
sudo insmod gdev.ko
sudo sh
5. Gdev Library
Gdev's user-space library provides Gdev API. This API can be used
by either user programs directly or another high-level API library.
For instance, third party's CUDA library can use Gdev API.
If you have taken Step 4, i.e., chosen the runtime-unified operating
system approach, this library is just a set of wrapper functions
that call Gdev module's functions via ioctl.
cd $(TOPDIR)/gdev/lib
mkdir build
cd build
../configure # if you skipped Step 4, you must specify --target=user
sudo make install
export LD_LIBRARY_PATH="/usr/local/gdev/lib64:$LD_LIBRARY_PATH"
export PATH="/usr/local/gdev/bin:$PATH"
6. CUDA Driver API
Gdev currently supports a limited set of CUDA Driver API. We plan to
support a full set of CUDA Driver API in future work. If you need
CUDA Runtime API, you should use some compiler framework, such as
Ocelot, which can translate CUDA Drier API to Runtime API.
cd $(TOPDIR)/gdev/cuda
mkdir build
cd build
sudo make install
Gdev also supports CUDA in the operating system. You are required to
install "kcuda" module to use this functionality.
cd $(TOPDIR)/gdev/cuda
mkdir kbuild
../configure --target=kcuda
sudo make install
7. CUDA Driver API test (user-space programs)
cd $(TOPDIR)/test/cuda/user/madd
./user_test 256 # a[256] x b[256] = c[256]
8. CUDA Driver API test (OS-space programs)
cd $(TOPDIR)/test/cuda/kernel/memcpy
sudo insmod ./kernel_test.ko size=10000 # copy 0x10000 size
NOTE: Please be careful when doing this test as it runs a program
in module_init(). If you run a very long program as it is, you may
crash your system. If you want to run a very long program, you must
provide a proper module implementation, e.g., using kernel threads.