Skip to content
Branch: master
Find file History
Type Name Latest commit message Commit time
Failed to load latest commit information.
Makefile Minor Markdown format revision May 17, 2018 Use default serving signature key Oct 13, 2017

Inference with the TensorFlow C++ API

This example shows how to translate a batch of sentences using the TensorFlow C++ API.


  • TensorFlow compiled as a library (see below)
  • an exported translation model

Note: for this example to work, the model should be a SequenceToSequence model with WordEmbedder as the source inputter.

Compile TensorFlow as a shared library

1. Prepare the environment for compiling TensorFlow.

2. Download the TensorFlow repository:

git clone ~/tensorflow
cd ~/tensorflow
git checkout r1.6

3. Add a custom compilation target at the end of tensorflow/BUILD:

    name = "",
    linkopts = select({
        "//tensorflow:darwin": [
            "-Wl,-exported_symbols_list",  # This line must be directly followed by the file
        "//tensorflow:windows": [],
        "//tensorflow:windows_msvc": [],
        "//conditions:default": [
            "-z defs",
            "-Wl,--version-script",  #  This line must be directly followed by the file
    deps = [

4. Configure and compile:

bazel build --config=opt //

Note: this compilation approach is presented for the sole purpose of this example. It may not be the recommended way to work with the TensorFlow C++ API.


export LD_LIBRARY_PATH=$HOME/tensorflow/bazel-bin/tensorflow
./main --export_dir=$HOME/OpenNMT-tf/models/enfr/1507109306/

where $HOME/OpenNMT-tf/models/enfr/1507109306/ is a directory containing the exported model.

The output of the last command should look like this:

2017-10-04 18:20:22.190490: I tensorflow/cc/saved_model/] Loading SavedModel from: $HOME/OpenNMT-tf/models/enfr/1507109306/
2017-10-04 18:20:22.228595: I tensorflow/cc/saved_model/] Restoring SavedModel bundle.
2017-10-04 18:20:22.373715: I tensorflow/cc/saved_model/] Running LegacyInitOp on SavedModel bundle.
2017-10-04 18:20:22.407657: I tensorflow/cc/saved_model/] Loading SavedModel: success. Took 217175 microseconds.
 Hello world !
 My name is John .
 I live on the West coast .
 Bonjour le monde !
 Mon nom est John .
 Je vis sur la côte Ouest .
You can’t perform that action at this time.