Skip to content
HugeCTR is a high efficiency GPU framework designed for Click-Through-Rate (CTR) estimating training
C++ Cuda CMake Other
Branch: master
Clone or download
Latest commit a02c027 Dec 12, 2019

README.md

HugeCTR

HugeCTR is a high-efficiency GPU framework designed for Click-Through-Rate (CTR) estimation training.

Design Goals:

  • Optimized for recommender system
  • Easy to be customized

Please find more introductions in our HugeCTR User Guide and doxygen files in directory docs\

Requirements

  • cuBLAS >= 9.1
  • Compute Capability >= 60 (P100)
  • CMake >= 3.8
  • cuDNN >= 7.5
  • NCCL >= 2.0
  • Clang-Format 3.8
  • OpenMPI >= 4.0 (optional, if require multi-nodes training)

Build

Init Git

$ git submodule init
$ git submodule update

Build with Release

Compute Capability can be specified by -DSM=XX, which is SM=60 by default. Only one Compute Capability is avaliable to be set.

$ mkdir -p build
$ cd build
$ cmake -DCMAKE_BUILD_TYPE=Release -DSM=XX ..
$ make

Build with Debug

Compute Capability can be specified by -DSM=XX, which is SM=60 by default. Only one Compute Capability is avaliable to be set.

$ mkdir -p build
$ cd build
$ cmake -DCMAKE_BUILD_TYPE=Debug -DSM=XX ..
$ make

Build with Mixed Precision (WMMA) Support

To use mixed precision training, enable USE_WMMA and set SCALER to 128/256/512/1024 by:

$ mkdir -p build
$ cd build
$ cmake -DSM=XX -DUSE_WMMA=ON -DSCALER=YYY ..

Run

Please refer to samples/*

Coding Style and Refactor

Default coding style follows Google C++ coding style (link). This project also uses Clang-Format(link) to help developers to fix style issue, such as indent, number of spaces to tab, etc. The Clang-Format is a tool that can auto-refactor source code. Use following instructions to install and enable Clang-Format:

Install

$ sudo apt-get install clang-format

Run

# First, configure Cmake as usual 
$ mkdir -p build
$ cd build
$ cmake -DCLANGFORMAT=ON ..
# Second, run Clang-Format
$ cmake --build . --target clangformat
# Third, check what Clang-Format did modify
$ git status
# or
$ git diff

Document Generation

Doxygen is supported in HugeCTR and by default on-line documentation browser (in HTML) and an off-line reference manual (in LaTeX) can be generated within docs/.

Install

Download doxygen

Generation

Within project home directory

$ doxygen

File Format

Totally three kinds of files will be used as input of HugeCTR Training: configuration file (.json), model file, data set.

Configuration File

Configuration file should be a json format file e.g. simple_sparse_embedding.json

There are four sessions in a configuration file: "solver", "optimizer", "data", "layers". The sequence of these sessions is not restricted.

  • You can specify the device (or devices), batchsize, model_file.. in solver session;
  • and the optimizer that will be used in every layer.
  • File list and data set related configurations will be specified under data session.
  • Finally, layers should be listed under layers. Note that embedders should always be the first layer.

Model File

Model file is a binary file that will be loaded for weight initilization. In model file weight will be stored in the order of layers in configuration file.

Data Set

A data set includes a ASCII format file list and a set of data in binary format.

A file list starts with a number which indicate the number of files in the file list, then comes with the path of each data file.

$ cat simple_sparse_embedding_file_list.txt
10
./simple_sparse_embedding/simple_sparse_embedding0.data
./simple_sparse_embedding/simple_sparse_embedding1.data
./simple_sparse_embedding/simple_sparse_embedding2.data
./simple_sparse_embedding/simple_sparse_embedding3.data
./simple_sparse_embedding/simple_sparse_embedding4.data
./simple_sparse_embedding/simple_sparse_embedding5.data
./simple_sparse_embedding/simple_sparse_embedding6.data
./simple_sparse_embedding/simple_sparse_embedding7.data
./simple_sparse_embedding/simple_sparse_embedding8.data
./simple_sparse_embedding/simple_sparse_embedding9.data

A data file (binary) contains a header and data (many samples).

Header Definition:

typedef struct DataSetHeader_{
  long long number_of_records; //the number of samples in this data file
  long long label_dim; //dimension of label
  long long slot_num; //the number of slots in each sample 
  long long reserved; //reserved for future use
} DataSetHeader;

Data Definition (each sample):

typedef struct Data_{
  int label[label_dim];
  Slot slots[slot_num];
} Data;

typedef struct Slot_{
  int nnz;
  T*  keys; //long long or uint
} Slot;
You can’t perform that action at this time.