Tiny, no-nonsense, self contained, Tensorflow and ONNX inference
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
.travis Cleanup (#47) Jan 16, 2019
cli post-release 0.2.5-pre Jan 22, 2019
core post-release 0.2.5-pre Jan 22, 2019
linalg post-release 0.2.5-pre Jan 22, 2019
onnx post-release 0.2.5-pre Jan 22, 2019
protos bump tf protos Nov 28, 2018
tensorflow post-release 0.2.5-pre Jan 22, 2019
tests Split TF, introduce Onnx (#27) Sep 21, 2018
.gitignore Tf cleanup (#36) Nov 2, 2018
.travis.yml fix cache dir Jan 22, 2019
Cargo.toml move conv optim to sep crate Jan 8, 2019
LICENSE licensing info Aug 7, 2017
LICENSE-APACHE licensing info Aug 7, 2017
LICENSE-MIT licensing info Aug 7, 2017
README.md formating in readme Dec 5, 2018
post-release.sh Cleanup (#47) Jan 16, 2019
release.sh release core/0.2.0 Jan 16, 2019

README.md

Tract

Build Status Doc

A tiny TensorFlow and ONNX inference engine.

This project used to be called tfdeploy, or Tensorflow-deploy-rust.

What ?

Tract is a tensorflow-compatible inference library. It loads a Tensorflow or ONNX frozen model from the regular protobuf format, and flows data through it.

Status and compatibility

ONNX

As of today (nov 2018), Tract passes successfully about 85% of ONNX backends tests. sqeezenet, densenet121, resnet50, inception_v2 and vgg19 tests are passing.

Tensorflow

Even if Tract is very far from supporting any arbitrary model, it can run Google Inception v3, Snips wakeword models, missing operators are easy to add. The lack of easy to reuse test suite, and the wide diversity of operators in Tensorflow make it difficult to target a full support.

Streaming capability ("pulse")

This is a semi-experimental support for real-time application like voice processing. It is similar in purpose to the way kaldi decodes voice: decoding must happens "as you go", you can not wait for the end of the incoming audio to start decoding.

Our current approach to the implementation is a bit different though. We convert a regular network to a pulse network that will act on small time slices (the "pulses"). This conversion only makes sense for a subset of operators, so not all networks can be converted to a pulse network.

Roadmap

One important guiding cross-concern: this library must cross-compile as easily as practical to small-ish devices (think 20$ boards).

License

Note: files in the protos/tensorflow directory are copied from the TensorFlow project and are not covered by the following licence statement.

Note: files in the protos/onnx directory are copied from the ONNX project and are not covered by the following licence statement.

Apache 2.0/MIT

All original work licensed under either of

Contribution

Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in the work by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.