Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Packaged TensorFlow C++ library for bazel-independent use #2412

Closed
vladfi1 opened this issue May 18, 2016 · 73 comments

Comments

@vladfi1
Copy link
Contributor

commented May 18, 2016

Currently, building a C++ application in tensorflow requires creating a project in the tensorflow source tree and compiling with bazel. In my case I would like to use tensorflow in a (fairly large) existing application with an existing build system that would be difficult to port to bazel. The solution to me seems to be exposing tensorflow as a library that can be linked with.

@mrry

This comment has been minimized.

Copy link
Member

commented May 18, 2016

The "//tensorflow:libtensorflow.so" bazel target enables you to build a standalone TensorFlow library that you can dynamically link into your other project. Would that be sufficient?

@vladfi1

This comment has been minimized.

Copy link
Contributor Author

commented May 19, 2016

Yes, that is what I need. It would be nice if this was documented somewhere too!

@cgao3

This comment has been minimized.

Copy link

commented Aug 18, 2016

could you give a more specific example?

suppose I have this file, and I want to compile it with tensorflow with g++ directly (no bazel, not in tensorflow's repo), how should I do this??

loader.cc.txt

@MisayaZ

This comment has been minimized.

Copy link

commented Jan 16, 2017

@vladfi1 @mrry what is the headers should I include while linking libtensorflow.so?

@mrry

This comment has been minimized.

Copy link
Member

commented Jan 17, 2017

@MisayaZ That depends on what you're building. The most reliable interface is the C API, which you can include with #include "tensorflow/c/c_api.h".

@gunan

This comment has been minimized.

Copy link
Member

commented Jan 29, 2017

@asimshankar I think we can call this resolved once c distributable is ready, right?

@asimshankar

This comment has been minimized.

Copy link
Contributor

commented Jan 29, 2017

Yes and no :)
I'm hoping to have the C API distributable soon, but C++ will be a little while longer since we're still working on the right API surface.

FWIW, for using the C API you need just c_api.h and libtensorflow.so, which we will package into a tarball.

@inflation

This comment has been minimized.

Copy link
Contributor

commented Feb 5, 2017

Hi, I'm trying to play around with the C API. After running bazel build //tensorflow:libtensorflow.so
and copy the shared library into another folder, I successfully compiled and linked a small example. But when I tried to run it, it showed something like this:

dyld: Library not loaded: bazel-out/local-py3-opt/bin/tensorflow/libtensorflow.so
  Referenced from: /Users/inflation/workspace/link_test/./a.out
  Reason: image not found

I must put the library in the same folder structure as it said to run it without error. So what is wrong here?

@cgao3

This comment has been minimized.

Copy link

commented Feb 5, 2017

@asimshankar

This comment has been minimized.

Copy link
Contributor

commented Feb 5, 2017

Seems like you're using OS X.
For that your choices are to either:

  • Copy libtensorflow.so into /usr/local/lib
  • Or use DYLD_LIBRARY_PATH (e.g., export DYLD_LIBRARY_PATH=/path/to/dir/containing/libtensorflow.so)

Hope that helps

@inflation

This comment has been minimized.

Copy link
Contributor

commented Feb 5, 2017

@asimshankar Thanks a lot!

@acomarce

This comment has been minimized.

Copy link

commented Feb 21, 2017

@mrry bit off topic, what would be needed to build it as a lib for windows and is there a plan for this do you know?

@mrry

This comment has been minimized.

Copy link
Member

commented Feb 21, 2017

@acomarce Two things, really:

  1. We'd need to set up the __declspec(dllexport)/__declspec(dllimport) annotations to define the set of symbols that would be exported by/imported from such a DLL.
  2. We'd need some additional build rules for creating the DLL in either the CMake or Bazel builds.

Part 1 will probably be easier if we focus on the C API, which has been designed as a stable API for consumption by many different projects, and has stronger versioning guarantees. The C++ API is more sprawling, and it would require invasive changes to set up the appropriate annotations. (I was just chatting with @guschmue, who observed that CMake has a newish feature for generating these definitions without code modification, but the resulting DLL ends up with more than the maximum 65536 symbols exported....)

I'm not aware of anyone who is working on this right now. Contributions would be welcome!

@guschmue

This comment has been minimized.

Copy link
Contributor

commented Feb 21, 2017

The cmake support for this is called CMAKE_WINDOWS_EXPORT_ALL_SYMBOLS. In theory this would be perfect but it adds everything public to the .def which the linker is unhappy about (just too much). Doing the c-api with dllexport is fine (I have a version of this) but I think even to do an user_op to be loaded with tf.load_library() you need some of the c++ api and the doc shows a lot of classes for the c++ api: https://www.tensorflow.org/api_docs/cc/.
Still looking for some better way to do this.
Same ask here: #7258

@guschmue

This comment has been minimized.

Copy link
Contributor

commented Feb 21, 2017

Maybe let me double check: is the c++ api one needs to support this
https://www.tensorflow.org/api_docs/cc/
or
https://github.com/tensorflow/tensorflow/tree/master/tensorflow/g3doc/api_docs/cc
The later is manageable. Until 1.0 it was the same.

@acomarce

This comment has been minimized.

Copy link

commented Feb 21, 2017

@guschmue I think it should be the later.

@guschmue

This comment has been minimized.

Copy link
Contributor

commented Feb 22, 2017

i made the changes in my local tree for https://github.com/tensorflow/tensorflow/tree/master/tensorflow/g3doc/api_docs/cc - around 130 methods/classes. With that I got the c# bindings to work. Going to try if I can get tensorflow/user_ops to work with that.

@ed-alertedh

This comment has been minimized.

Copy link

commented Mar 20, 2017

@guschmue Do you think the project would ever consider explicitly exposing just the public API for all platforms? Would it be fair to say this is currently implicit based on the doc generation process? Looks like GCC 4 supports something similar to __declspec(dllexport) - http://gcc.gnu.org/wiki/Visibility. Although, the caveats on that page regarding exceptions sound messy, so perhaps this has already been discussed and decided against...

@guschmue

This comment has been minimized.

Copy link
Contributor

commented Mar 20, 2017

Yes - for windows this would be perfect. But its a lot of classes/files to be touched and its hard for me as external contributor to decide what is long term public and what is private on the tensorflow api; I think a tensorflow team member would need to go over the entire api and make that call.
In the meantime I have fixed the tf.load_library() #8217

@ed-alertedh

This comment has been minimized.

Copy link

commented Mar 21, 2017

@guschmue Apologies, I wrongly assumed you were an internal contributor! Would be interested to see your changes to get the C# bindings to work though.

Perhaps my question would have been better directed at @mrry

@guschmue

This comment has been minimized.

Copy link
Contributor

commented Mar 21, 2017

For the c# binding: I actually got https://github.com/migueldeicaza/tensorflowsharp to work with my changes. The 'tensorflow.dll' is in the python wheel, only that it is called _pywrap_tensorflow.pyd.
If you copy _pywrap_tensorflow.pyd to tensorflow.dll and than point the tensorflowsharp project to that dll it should work. I think the only other thing I had to do was to tell the project that tensorflow.dll is a x64 native dll.
Would be cool if we could make this a nuget package ... thinking about that.

@mrry

This comment has been minimized.

Copy link
Member

commented Mar 22, 2017

For the public client API, it should be relatively easy to make a DLL that exports just the symbols in c_api.h and I think we'd welcome contributions to the CMake or Bazel/Windows builds that make this possible.

The _pywrap_tensorflow.pyd library exports far more symbols in order to support the implicity-defined public framework API which custom op kernels use. Making tf.load_library() work on Windows required heroic efforts from @guschmue to scrape all of the necessary symbols :) from the library (and then subset them to avoid hitting symbol table limits). Part of the issue is that some of the public framework API includes generated code (e.g. protobuf headers), and threading the appropriate definitions through to the linker is somewhat challenging....

All of which is a long-winded way of saying (in my opinion) the de facto public API for the framework (i.e. to support custom ops) is too unwieldy, and I don't think we should exhaustively trace down every symbol that needs to be exported. It would be better to spend time defining a clean, self-contained, and possibly versioned API for creating extension modules, probably exporting C bindings like the client API, and then we could have more confidence about modules working across different builds of TensorFlow. This would be a big undertaking though....

@guschmue

This comment has been minimized.

Copy link
Contributor

commented Mar 22, 2017

Totally agree. _pywrap_tensorflow.pyd includes a ton more than the public c api and exporting the c api only would be easy (+ maybe tensorflow::ops?). We added the TF_EXPORT define so we could just annotate the c api with TF_EXPORT.
BUT if your model requires a custom op that needs to be loaded (say a gru) I think you'd need to have my large .def file again. All a bit painful.
I'm actively trying to use a tensorflow.dll so I can get rid of the static linking and still have a few issues with it: tensorflow/tools/benchmark/ works, tensorflow/examples/label_image/ has a few unresolved symbols, mostly in the namespace tensorflow::ops ... still looking at that.

@PatWie

This comment has been minimized.

Copy link

commented Apr 8, 2017

This is still an open issue. Even when creating the libfile-file by:

bazel build //tensorflow:libtensorflow.so

I cannot build the file

#include <tensorflow/core/framework/tensor.pb.h>

int main(int argc, char const *argv[])
{
    tensorflow::TensorProto tensor;
    return 0;
}

because it misses some important parts:

/tmp/ccHrvxj1.o: In function `main':
standalone.cpp:(.text.startup+0x1f): undefined reference to `tensorflow::TensorProto::TensorProto()'
standalone.cpp:(.text.startup+0x27): undefined reference to `tensorflow::TensorProto::~TensorProto()'
collect2: error: ld returned 1 exit status
g++ standalone.cpp -Wall -Wextra  -std=c++11 -O3 -march=native -fPIC -I /home/myusername/.local/lib/python2.7/site-packages/tensorflow/include -L/path/to/tensorflow/bazel-bin/tensorflow -ltensorflow

This basically prevents to build a c++ project with tensorflow as dependencies without messing files around in the official repo.

@asimshankar

This comment has been minimized.

Copy link
Contributor

commented Apr 8, 2017

@PatWie : libtensorflow.so only export symbols from the C API (tensorflow/c/c_api.h). It does not export any protocol buffer symbols or other C++ ones. This issue is still open since there is not C++ API distribution yet.

@nikolai3d

This comment has been minimized.

Copy link

commented Apr 20, 2017

@mrry @asimshankar : What would be the steps to build and link to TF as a static C++ library then? Or would that not be a feasible task outside of bazel build environment?

@asimshankar

This comment has been minimized.

Copy link
Contributor

commented Apr 20, 2017

@skye is looking into a distribution for the C++ library (i.e., library + all required header files).
However, you can build from source for now yourself with the //tensorflow:libtensorflow_cc.so target.

If you need a static library, we're currently blocked on bazelbuild/bazel#1920

@memo

This comment has been minimized.

Copy link
Contributor

commented Nov 4, 2017

Not official, but I provide C++ libraries and headers for use outside bazel, and build instructions too (mainly to be used with openFrameworks, but not limited to).
https://github.com/memo/ofxMSATensorFlow
https://github.com/memo/ofxMSATensorFlow/wiki/

@vade

This comment has been minimized.

Copy link
Contributor

commented Nov 24, 2017

Hey @memo ;) Out of curiosity, have you had reports of linker issues with ofxMSATensorFlow on OS X with TF 1.3 and TF 1.4? Ive been getting the following linker issue in my Synopsis project which is basically your instructions the wiki you linked almost verbatim:

#12539

Trying to isolate - but I cant quite deduce where my issues lies.

@memo

This comment has been minimized.

Copy link
Contributor

commented Nov 25, 2017

Hey @vade, I haven't built on OSX since r1.0 I'm afraid. Linux only now.

Did you update the headers? This is a bit of a dirty solution, but right now (I think because the C++ API is changing), you need to change headers for each lib (not always, but sometimes). So it might be that if you're using the 1.2 headers, it's not finding the implementations in the 1.3 lib?

This script updates the headers
https://github.com/memo/ofxMSATensorFlow/blob/master/scripts/ubuntu/copy_headers.sh

Otherwise I'm not sure what it might be.

@vade

This comment has been minimized.

Copy link
Contributor

commented Nov 25, 2017

Memo- thanks. I had done that before, but had a different compiler error (Nsync issue as noted here:

#12482

But the new fixes in that thread resolved the issue. Much appreciated!

@kikirizki

This comment has been minimized.

Copy link

commented Nov 25, 2017

@memo sir does the source code https://github.com/memo/ofxMSATensorFlow.git has GPU support ?
thx

@tjacobs

This comment has been minimized.

Copy link

commented Nov 28, 2017

I've written a small guide for C++ CMake TensorFlow on Mac: https://medium.com/@TomPJacobs/c-tensorflow-a-journey-bdecbbdd0f65

@memo

This comment has been minimized.

Copy link
Contributor

commented Nov 28, 2017

@vade
glad to hear that that problem is sorted. Interestingly I just tried building 1.4 (I'd only gone as far as 1.3), and I too get the nsync error! I'm currently looking for a solution which doesn't involved modifying the src, but will resort to that if I have to :)

@kikirizki
the source code itself is agnostic to GPU or CPU or platform. For tf1.3 I provide precompiled libraries for linux only with GPU support (here). If you wanted without GPU support, or for another platform (i.e. OSX) you can build the library yourself (following instructions here - or @tjacobs's guide also looks great!). Or if someone else builds the lib, I can include it in my repo for download.

@alextp

This comment has been minimized.

Copy link
Member

commented Feb 7, 2018

Closing the issue. There's a lot of discussion here, but we already release shared objects which are usable without bazel on many platforms. Please file new specific issues if this is not enough.

@alextp alextp closed this Feb 7, 2018

@prclibo

This comment has been minimized.

Copy link

commented Mar 18, 2018

Hi @alextp Maybe a naive question: where do we get the released shared objects you mentioned? I can get libtensorflow_framework.so from my pip-installed tensorflow, but lack tensorflow_cc.

@jimaldon

This comment has been minimized.

Copy link

commented Mar 19, 2018

@prclibo You have to build tensorflow from source for that.
Clone the repo, run ./configure and then bazel build //tensorflow:libtensorflow_cc.so

@FloopCZ

This comment has been minimized.

Copy link
Contributor

commented Apr 17, 2018

For those who like Docker, the tensorflow_cc project provides images with prebuilt TensorFlow C++ API.

@public1024

This comment has been minimized.

Copy link

commented Jun 7, 2018

I have success compile the libtensorflow_cc.so, and managed to complie my release version of classification so. However, if I tried to compile my demo to use the so file in release mode, it failed with many undefined link errrors. If I tried in the default debug mode, I succeed. The log of errors is:

/home/eli/tensorflow/tensorflow-master/tensorflow/cc/pointnet/build/libclassification.so:对‘tensorflow::Tensor::CheckTypeAndIsAligned(tensorflow::DataType) const’未定义的引用 /home/eli/tensorflow/tensorflow-master/tensorflow/cc/pointnet/build/libclassification.so:对‘tensorflow::TensorShapeRep::DestructorOutOfLine()’未定义的引用 /home/eli/tensorflow/tensorflow-master/tensorflow/cc/pointnet/build/libclassification.so:对‘tensorflow::operator<<(std::ostream&, tensorflow::Status const&)’未定义的引用 /home/eli/tensorflow/tensorflow-master/tensorflow/cc/pointnet/build/libclassification.so:对‘tensorflow::ReadBinaryProto(tensorflow::Env*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, google::protobuf::MessageLite*)’未定义的引用 /home/eli/tensorflow/tensorflow-master/tensorflow/cc/pointnet/build/libclassification.so:对‘tensorflow::TensorShapeBase<tensorflow::TensorShape>::TensorShapeBase()’未定义的引用 /home/eli/tensorflow/tensorflow-master/tensorflow/cc/pointnet/build/libclassification.so:对‘tensorflow::internal::CheckOpMessageBuilder::NewString[abi:cxx11]()’未定义的引用 /home/eli/tensorflow/tensorflow-master/tensorflow/cc/pointnet/build/libclassification.so:对‘tensorflow::internal::LogMessage::LogMessage(char const*, int, int)’未定义的引用 /home/eli/tensorflow/tensorflow-master/tensorflow/cc/pointnet/build/libclassification.so:对‘tensorflow::strings::StrCat[abi:cxx11](tensorflow::strings::AlphaNum const&, tensorflow::strings::AlphaNum const&, tensorflow::strings::AlphaNum const&)’未定义的引用 /home/eli/tensorflow/tensorflow-master/tensorflow/cc/pointnet/build/libclassification.so:对‘tensorflow::TensorShapeRep::SlowCopyFrom(tensorflow::TensorShapeRep const&)’未定义的引用 /home/eli/tensorflow/tensorflow-master/tensorflow/cc/pointnet/build/libclassification.so:对‘tensorflow::ConfigProto::~ConfigProto()’未定义的引用 /home/eli/tensorflow/tensorflow-master/tensorflow/cc/pointnet/build/libclassification.so:对‘tensorflow::NewSession(tensorflow::SessionOptions const&)’未定义的引用 /home/eli/tensorflow/tensorflow-master/tensorflow/cc/pointnet/build/libclassification.so:对‘tensorflow::Env::Default()’未定义的引用 /home/eli/tensorflow/tensorflow-master/tensorflow/cc/pointnet/build/libclassification.so:对‘tensorflow::internal::LogMessageFatal::~LogMessageFatal()’未定义的引用 /home/eli/tensorflow/tensorflow-master/tensorflow/cc/pointnet/build/libclassification.so:对‘tensorflow::internal::CheckOpMessageBuilder::CheckOpMessageBuilder(char const*)’未定义的引用 /home/eli/tensorflow/tensorflow-master/tensorflow/cc/pointnet/build/libclassification.so:对‘tensorflow::internal::CheckOpMessageBuilder::~CheckOpMessageBuilder()’未定义的引用 /home/eli/tensorflow/tensorflow-master/tensorflow/cc/pointnet/build/libclassification.so:对‘tensorflow::Tensor::CheckIsAlignedAndSingleElement() const’未定义的引用 /home/eli/tensorflow/tensorflow-master/tensorflow/cc/pointnet/build/libclassification.so:对‘tensorflow::SessionOptions::SessionOptions()’未定义的引用 /home/eli/tensorflow/tensorflow-master/tensorflow/cc/pointnet/build/libclassification.so:对‘tensorflow::Tensor::~Tensor()’未定义的引用 /home/eli/tensorflow/tensorflow-master/tensorflow/cc/pointnet/build/libclassification.so:对‘tensorflow::TensorShapeBase<tensorflow::TensorShape>::TensorShapeBase(std::initializer_list<long long>)’未定义的引用 /home/eli/tensorflow/tensorflow-master/tensorflow/cc/pointnet/build/libclassification.so:对‘tensorflow::internal::LogMessage::~LogMessage()’未定义的引用 /home/eli/tensorflow/tensorflow-master/tensorflow/cc/pointnet/build/libclassification.so:对‘tensorflow::Tensor::Tensor(tensorflow::DataType, tensorflow::TensorShape const&)’未定义的引用 /home/eli/tensorflow/tensorflow-master/tensorflow/cc/pointnet/build/libclassification.so:对‘tensorflow::GraphDef::~GraphDef()’未定义的引用 /home/eli/tensorflow/tensorflow-master/tensorflow/cc/pointnet/build/libclassification.so:对‘tensorflow::GraphDef::GraphDef()’未定义的引用 /home/eli/tensorflow/tensorflow-master/tensorflow/cc/pointnet/build/libclassification.so:对‘tensorflow::port::InitMain(char const*, int*, char***)’未定义的引用 /home/eli/tensorflow/tensorflow-master/tensorflow/cc/pointnet/build/libclassification.so:对‘tensorflow::internal::LogMessageFatal::LogMessageFatal(char const*, int)’未定义的引用 /home/eli/tensorflow/tensorflow-master/tensorflow/cc/pointnet/build/libclassification.so:对‘tensorflow::internal::CheckOpMessageBuilder::ForVar2()’未定义的引用 /home/eli/tensorflow/tensorflow-master/tensorflow/cc/pointnet/build/libclassification.so:对‘tensorflow::Status::Status(tensorflow::error::Code, tensorflow::StringPiece)’未定义的引用
Any help~

@kecsap

This comment has been minimized.

Copy link

commented Aug 9, 2018

I can also recommend FloopCZ's project, especially if you want to use Tensorflow inference with GPU. Those who wants a packaged version, I packaged both the C and C++ API in my project and I release Debian packages time to time. The disadvantage of my project that it is based on contrib/makefiles therefore it is CPU-only.

@FloopCZ FloopCZ referenced this issue Aug 15, 2018
@zpge

This comment has been minimized.

Copy link

commented Aug 22, 2018

@kikirizki
Can you kindly write a short tutorial to show how to use your code?

@HackersSpirit

This comment has been minimized.

Copy link

commented Nov 27, 2018

Guys i tried to build libtensorflow.so using bazel and tried to import it in my qt-based project but getting the following error.
eferenced in function "public: __cdecl tensorflow::SessionOptions::~SessionOptions(void)" (??1SessionOptions@tensorflow@@QEAA@XZ)
main.obj : error LNK2019: unresolved external symbol "public: __cdecl tensorflow::SessionOptions::SessionOptions(void)" (??0SessionOptions@tensorflow@@QEAA@XZ) referenced in function main
main.obj : error LNK2019: unresolved external symbol "class tensorflow::Status __cdecl tensorflow::NewSession(struct tensorflow::SessionOptions const &,class tensorflow::Session * *)" (?NewSession@tensorflow@@ya?AVStatus@1@AEBUSessionOptions@1@PEAPEAVSession@1@@z) referenced in function main
debug\TensorFlowExp1.exe : fatal error LNK1120: 3 unresolved externals.
Beside i also tried to run the pre-built windows binary and end up getting the same error.

@guikarist

This comment has been minimized.

Copy link

commented Dec 17, 2018

If you guys have same demand on Windows, I have asked a question which may help you to some extent.

@HackersSpirit

This comment has been minimized.

Copy link

commented Dec 17, 2018

@guikarist : Ok :)

@hluu11

This comment has been minimized.

Copy link

commented Dec 25, 2018

I've just published my old code and some prebuilt stuff here: https://github.com/hluu11/SimpleTF-CPP
Didn't know people need it

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
You can’t perform that action at this time.