Intel(R) nGraph(TM) Compiler and runtime for TensorFlow*
This repository contains the code needed to enable Intel(R) nGraph(TM) Compiler and runtime engine for TensorFlow. Use it to speed up your TensorFlow training and inference workloads. The nGraph Library and runtime suite can also be used to customize and deploy Deep Learning inference models that will "just work" with a variety of nGraph-enabled backends: CPU, GPU, and custom silicon like the Intel(R) Nervana(TM) NNP.
Option 1: Use a pre-built nGraph-TensorFlow bridge
You need to instantiate a specific kind of
virtualenvto be able to proceed with the
ngraph-tfbridge installation. For systems with Python 3.n or Python 2.7, these commands are
virtualenv --system-site-packages -p python3 your_virtualenv virtualenv --system-site-packages -p /usr/bin/python2 your_virtualenv source your_virtualenv/bin/activate # bash, sh, ksh, or zsh
Install TensorFlow v1.12.0:
pip install -U tensorflow
Install nGraph-TensorFlow bridge:
pip install -U ngraph-tensorflow-bridge
Test the installation by running the following command:
python -c "import tensorflow as tf; print('TensorFlow version: r',tf.__version__);import ngraph_bridge; print(ngraph_bridge.__version__)"
This will produce something like this:
TensorFlow version: r 1.12.0 TensorFlow version installed: 1.12.0 (v1.12.0-0-ga6d8ffae09) nGraph bridge built with: 1.12.0 (v1.12.0-0-ga6d8ffae09) b'0.8.0'
Next you can try out the TensorFlow models by adding one line to your existing TensorFlow model scripts and running them the usual way:
Note: The version of the ngraph-tensorflow-bridge is not going to be exactly the same as when you build from source. This is due to delay in the source release and publishing the corresponding Python wheel.
Option 2: Build nGraph bridge from source using TensorFlow source
To use the latest version, or to run unit tests, or if you are planning to contribute, install the nGraph bridge using the TensorFlow source tree as follows:
Prepare the build environment
The installation prerequisites are the same as described in the TensorFlow prepare environment for linux.
TensorFlow uses a build system called "bazel". These instructions were tested with bazel version 0.16.0.
wget https://github.com/bazelbuild/bazel/releases/download/0.16.0/bazel-0.16.0-installer-linux-x86_64.sh chmod +x bazel-0.16.0-installer-linux-x86_64.sh ./bazel-0.16.0-installer-linux-x86_64.sh --user
Add and source the
binpath to your
~/.bashrcfile in order to be able to call bazel from the user's installation we set up:
export PATH=$PATH:~/bin source ~/.bashrc
Additionally, you need to install
cmakeversion 3.1 or higher and gcc 4.8 or higher.
Once TensorFlow's dependencies are installed, clone
git clone https://github.com/NervanaSystems/ngraph-tf.git cd ngraph-tf git checkout v0.9.0
Next run the following Python script to build TensorFlow, nGraph and the bridge:
Once the build finishes, a new virtualenv directory is created in the
build/venv-tf-py3. The build artifacts i.e., the
ngraph_tensorflow_bridge-<VERSION>-py2.py3-none-manylinux1_x86_64.whl is created in the
Test the installation by running the following command:
This command will run all the C++ and python unit tests from the ngraph-tf source tree. Additionally this will also run various TensorFlow python tests using nGraph.
To use the ngraph-tensorflow bridge, activate this virtual environment to start using nGraph with TensorFlow.
Once the build and installation steps are complete, you can start using TensorFlow with nGraph backends.
Please add the following line to enable nGraph:
Option 3: Using the upstreamed version
nGraph is updated in the TensorFlow source tree using pull requests periodically.
In order to build that version of nGraph, follow the steps below, which involves building TensorFlow from source with certain settings.
Install bazel using the same instructions outlined in option 2, step 1 above.
Create a virtual environment using the instructions outlined in option 1, step 1 above.
Get tensorflow v1.12.0
git clone https://github.com/tensorflow/tensorflow.git cd tensorflow git checkout v1.12.0
Note: To get the latest version of nGraph, use the tip of
master branch of TensorFlow. The exact version of
bazel changes for a specific version of TensorFlow. Please consult the build instructions from TensorFlow web site for specific bazel requirements.
nofor the following when prompted to build TensorFlow.
Do you wish to build TensorFlow with XLA JIT support? [Y/n]: n No XLA JIT support will be enabled for TensorFlow.
Do you wish to build TensorFlow with CUDA support? [y/N]: N No CUDA support will be enabled for TensorFlow.
Note that if you are running TensorFlow on a Skylake family processor then select
-march=broadwellwhen prompted to specify the optimization flags:
Please specify optimization flags to use during compilation when bazel option "--config=opt" is specified [Default is -march=native]: -march=broadwell
This is due to an issue in TensorFlow tracked in this issue: https://github.com/tensorflow/tensorflow/issues/17273
Prepare the pip package
bazel build --config=opt --config=ngraph //tensorflow/tools/pip_package:build_pip_package bazel-bin/tensorflow/tools/pip_package/build_pip_package ./
Note: The specific questions for the
configure step and the build command mentioned above changes for different versions of TensorFlow.
Once the pip package is built, install using
pip install -U ./tensorflow-1.*whl
For this final option, there is no need to separately build
ngraph-tf or to
pip to install the nGraph module. With this configuration, your TensorFlow model scripts will work without any changes, ie, you do not need to add
import ngraph_bridge, like option 1 and 2.
Note: The version that is available in the upstreamed version of TensorFlow usually
lags the features and bug fixes available in the
master branch of this repository.
You can run a few of your own DL models to validate the end-to-end
functionality. Also, you can use the
ngraph-tf/examples directory and try to
run the following model:
cd examples python3 keras_sample.py
Using OS X
The build and installation instructions are idential for Ubuntu 16.04 and OS X.
See the instructions provided in the diagnostics directory.
Please submit your questions, feature requests and bug reports via GitHub issues.
How to Contribute
We welcome community contributions to nGraph. If you have an idea for how to improve it:
- Share your proposal via GitHub issues.
- Ensure you can build the product and run all the examples with your patch.
- In the case of a larger feature, create a test.
- Submit a pull request.
- We will review your contribution and, if any additional fixes or modifications are necessary, may provide feedback to guide you. When accepted, your pull request will be merged to the repository.
About Intel(R) nGraph(TM)
See the full documentation here: http://ngraph.nervanasys.com/docs/latest