tensorflow prediction using c++ api
Switch branches/tags
Clone or download
Latest commit 34d10c5 May 5, 2018
Permalink
Failed to load latest commit information.
demo fix Apr 24, 2018
include fix Oct 13, 2017
python fix Apr 24, 2018
src Update CMakeLists.txt May 5, 2018
.gitignore Initial commit Sep 22, 2017
CMakeLists.txt linux build cmake Oct 31, 2017
README.md readme Apr 24, 2018

README.md

tensorflow-predictor-cpp

TensorFlow prediction using its C++ API. Having this repo, you will not need TensorFlow-Serving. This project has been tested on OSX and Linux.

Contains two examples:

  • simple model c = a * b
  • an industrial deep model for large scale click through rate prediction

Covered knowledge points:

  • save model and checkpoint
  • freeze model with checkpoint
  • replace part of nodes in the model for prediction
  • transform libfm data into tfrecord
  • use lookup table for sparse embedding
  • load model in C++
  • construct SparseTensor in C++
  • prediction in C++

Build on OSX

1) Build TensorFlow

Follow the instruction build tensorflow from source

git clone --recursive https://github.com/tensorflow/tensorflow.git
cd tensorflow
sh tensorflow/contrib/makefile/build_all_linux.sh (works for linux and osx)
cd ..

2) Build this repo

Keep this repo in the same directory with tensorflow.

git clone https://github.com/formath/tensorflow-predictor-cpp.git
cd tensorflow-predictor-cpp
mkdir build && cd build
cmake ..
make

Simple Model

This demo used c = a * b to show how to save the model and load it using C++ for prediction.

  • Save model
  • Load model
  • Prediction

More detail in Chinese: tensorflow_c++_api_prediction

cd demo/simple_model
# train
sh train.sh
# predict
sh predict.sh

If works right, you will see this

Session created successfully
Load graph protobuf successfully
Add graph to session successfully
Run session successfully
Tensor<type: float shape: [] values: 6>
output value: 6

Deep CTR Model

This demo show a real-world deep model usage in click through rate prediction.

  • Transform LibFM data into TFRecord
  • Usage of lookup table
  • Save model and checkpoint
  • Replace parts of model and freeze graph with checkpoint
  • Load model and checkpoint
  • Prediction

More detail in Chinese: tensorflow_c++_api_prediction

1) Transform LibFM data into TFRecord

  • LibFM format: label fieldId:featureId:value ...
cd demo/deep_model
sh trans_data_to_tfrecord.sh

2) Train model

sh train.sh

3) Freeze model

sh freeze_graph.sh

4) Predict using C++

sh predict.sh

If works right, you will see this

Session created successfully
Load graph protobuf successfully
Add graph to session successfully
Init lookup table successfully
Run session successfully
Tensor<type: float shape: [1,2] values: [0.85252136 0.14747864]>
output value: 0.147479

The output value may be different.

Build on Linux

The procedure is similar with that of OSX.

Issues

  • Error: This file was generated by an older version of protoc. Solution: Please use the matching version of protobuf with tensorflow. You will find it in tensorflow/tensorflow/contrib/makefile/gen/protobuf/bin/protoc