Tensorflow lite dll build failed on Windows

[twkx]

System information

• OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Windows 10 Home
• TensorFlow installed from (source or binary): source (master)
• TensorFlow version: 2.0 (master)
• Python version: 3.8.0
• Installed using virtualenv? pip? conda?: VirtualBoxVM
• Bazel version (if compiling from source): 0.26.1
• GCC/Compiler version (if compiling from source): Visual Studio Build Tools 201

Describe the problem

I have a code base successfully running on Linux/MacOS/Android & iOS.

Now I need to run tflite model inference under Windows system.
For this, I need to compile a tensorflowlite dll for Windows (I've not found any precompiled version)

Unfortunately, the build fails at the link step... (see logs below).

Apparently, even if the build fails, a libtensorflowlite.so file is stil generated (and is indeed a dll), but it contains no exported symbols, so I cannot use it.
After checking at the tensorflow/lite/BUILD file it appears that no .def file is generated for the tflite Windows link step, as it is done for tensorflow.dll (seen in tensorflow/BUILD), which explains why no symbols are exported.

Provide the exact sequence of commands / steps that you executed before running into the problem

In Windows command prompt (Run as Administrator):

python configure.py
bazel build -s tensorflow/lite:libtensorflowlite.so

Any other info / logs

C:\Users\soule\Downloads\tensorflow> bazel build -s tensorflow/lite:libtensorflowlite.so
[...]
C:/Program Files (x86)/Microsoft Visual Studio/2019/BuildTools/VC/Tools/MSVC/14.23.28105/bin/HostX64/x64/link.exe /nologo /DLL /SUBSYSTEM:CONSOLE -s -ignore:4221 -ignore:4221 -ignore:4221 -ignore:4221 /MACHINE:X64 @bazel-out/x64_windows-opt/bin/tensorflow/lite/libtensorflowlite.so-2.params /OPT:ICF /OPT:REF
INFO: From Linking tensorflow/lite/libtensorflowlite.so:
LINK : warning LNK4044: unrecognized option '/s'; ignored
ERROR: C:/users/soule/downloads/tensorflow/tensorflow/lite/BUILD:442:1: output 'tensorflow/lite/libtensorflowlite.so.if.lib' was not created
ERROR: C:/users/soule/downloads/tensorflow/tensorflow/lite/BUILD:442:1: not all outputs were created or valid
Target //tensorflow/lite:libtensorflowlite.so failed to build
INFO: Elapsed time: 920.482s, Critical Path: 84.10s
INFO: 235 processes: 235 local.
FAILED: Build did NOT complete successfully

C:\Users\soule\Downloads\tensorflow> dumpbin /exports bazel-out\x64_windows-opt\bin\tensorflow\lite\libtensorflowlite.so
Microsoft (R) COFF/PE Dumper Version 14.23.28106.4
Copyright (C) Microsoft Corporation. All rights reserved.

Dump of file bazel-out\x64_windows-opt\bin\tensorflow\lite\libtensorflowlite.so

File Type: DLL

Summary

    1000 .data
    1000 .pdata
    1000 .rdata
    1000 .reloc
    2000 .text
    1000 _RDATA

[twkx]

I confirm the issue is caused by the fact that no .def file is provided in the case of tflite build for Windows.

I successfully build a tflite dll for Windows by calling link.exe by myself and providing a custom .def file I created.

A block similar to the following one (found in tensorflow/BUILD) should be added to tensorflow/lite:libtensorflowlite.so target:

# add win_def_file for tensorflow_cc
win_def_file = select({
    # We need this DEF file to properly export symbols on Windows
    "//tensorflow:windows": ":tensorflow_filtered_def_file",
    "//conditions:default": None,
}),

[jdduke]

Good catch! Feel free to propose a PR, I'd be happy to approve.

[ValYouW]

@jdduke Can you please elaborate on the required change? when I add what @twkx mentioned in tensorflow/lite/BUILD I get the error:

unexpected keyword 'win_def_file' in call to tflite_cc_shared_object

What am I doing wrong?
Thx.

[twkx]

@ValYouW This is normal that you have this error.

In order to make it fully work you must copy all the tensorflow BUILD parts that generate the .def file, and customize it for tensorflow lite dll. I'm not an expert in Bazel, so I could not make it myself, this is why I didn't push a proper PR.

What I did is to generate manually the .def file based on the missing symbols I had in my own project, and used link.exe to build the dll from command line.

By the way, @jdduke the dll I obtained is not SSE optimized, so it is very slow. I have started to check why NEON2SSE.h is not used in windows build (as in MacOS build) but not yet found. I move on other tasks right now, but I have planned to make it work some day.

[jdduke]

By the way, @jdduke the dll I obtained is not SSE optimized, so it is very slow. I have started to check why NEON2SSE.h is not used in windows build (as in MacOS build) but not yet found. I move on other tasks right now, but I have planned to make it work some day.

OK, thanks for checking, there may be some tweaks we need to make in our build config to get this working (sadly we don't have exhaustive internal tests for Windows internally).

[ValYouW]

@twkx Thanks. Too complicated, I'll skip windows for now... :)

[jdduke]

I think this might be resolved with bazelbuild/bazel#9976. I'm curious if adding features = ["windows_export_all_symbols"], to this build rule resolves the problem temporarily?

[twkx]

@jdduke I confirm adding this fixes the issue and creates a working dll (just rename the .so as .dll)! Thanks a lot.

I'll check for neon asap.

[jdduke]

Btw, I'm landing a change shortly that should automatically generate the appropriate lib name based on the target platform (e.g., tensorflowlite.dll, libtensorflowlite.so, libtensorflowlite.dylib). I'll also go ahead and land the features = ["windows_export_all_symbols"] for now. Thanks for your patience!

[ChrisAGBlake]

Am I understanding correctly that if I build bazel from master to get that latest fix, then add features = ["windows_export_all_symbols"], to tensorflow/lite/build_def.bzl like:

def tflite_cc_shared_object(
name,
copts = tflite_copts(),
linkopts = [],
linkstatic = 1,
deps = [],
visibility = None,
per_os_targets = False,
tags = None):
"""Builds a shared object for TFLite."""
tf_cc_shared_object(
name = name,
copts = copts,
linkstatic = linkstatic,
linkopts = linkopts + tflite_jni_linkopts(),
framework_so = [],
deps = deps,
visibility = visibility,
tags = tags,
per_os_targets = per_os_targets,
features = ["windows_export_all_symbols"],
)

I should then be able to compile tflite on windows using the bazel version built from source with: bazel build //tensorflow/lite:libtensorflowlite.so

Is this correct? Or will I need to do other steps to get it working?
Thanks

[ChrisAGBlake]

For anyone who is interested I got it built on windows using the 2.1.0 release of bazel, adding in the features = ["windows_export_all_symbols"], to tensorflow/lite/build_def.bzl as before and compiling with bazel-2.1.0.exe build //tensorflow/lite:libtensorflowlite.so

[jdduke]

From a master checkout, could you try doing:

bazel build -c opt //tensorflow/lite:tensorflowlite

That should produce a proper .dll, though I want to make sure it's properly symbolized. Thanks!

[ChrisAGBlake]

I pulled from master on tensorflow. Then tried:
bazel build -c opt //tensorflow/lite:tensorflowlite

This built the .lib and .dll, however the files were tiny ~2KB and when I tried to use them I got error with unresolved symbols. I then added the following after line 168 in build_def.bzl and rebuilt in the same way:
features = ["windows_export_all_symbols"],

This produced the .lib that I could link against. However my code failed at runtime. (I'm not sure why yet).

[ChrisAGBlake]

The same code worked fine when using an older commit on tensorflow:master ~15 Jan, and adding the extra line to build_def.bzl then compiling with bazel build //tensorflow/lite:libtensorflowlite.so (2.1.0 release of bazel). However I did have to keep the generated library named as .so.

[brookman]

Is it possible to build TFLite for Windows and ARM(32/64)? I am asking because it would be useful to run it on the HoloLens 2 which is based on Universal Windows Platform (UWP) and has an ARM chip set.

[iwatake2222]

@ChrisAGBlake

I built tensorflowlite.dll.if.lib and tensorflowlite.dll from master branch with modified build_def.bzl.
The generated library files work in my project on Visual Studio 2017.
Do you happen to forget to copy tensorflowlite.dll?

I just arranged the way to generate tensorflowlite.dll which actually works at the moment.
https://gist.github.com/iwatake2222/3017d9ac3112b27cc9f5011ece993009

[ChrisAGBlake]

I'm fairly sure I copied the .dll across into the runtime folder correctly. However I could have made some other mistake. I'll give it another go later when I get a chance.

[Lotte1990]

I think I might be experiencing the same problem as @ChrisAGBlake

• Pull nightly (TF 2.2.0-dev20200308)
• Add features = ["windows_export_all_symbols"], to tensorflow/lite/build_def.bzl
• bazel build -c opt //tensorflow/lite:tensorflowlite
• This gives me tensorflowlite.dll.if.lib (56.4 MB) and tensorflowlite.dll (13.6 MB)
• I can successfully link the .dll using the .dll.if.lib in CMake as proposed by @iwatake2222
• No errors (not during build, not during runtime)
• Code crashes at runtime. More specifically, at float* input_tensor = interpreter->typed_input_tensor<float>(0);
  (the FlatBufferModel is able to load properly before this)

The same code works perfectly on Ubuntu 18.04 using libtensorflowlite.so compiled on a Ubuntu system. Any updates on this issue?

[jdduke]

In general, if you're using the C++ API, you need to be extremely careful that your client library (referencing TFLite code) is built with identical build/link options as the TFLite library itself.

In general, prefer using the tensorflow/lite/c shared library to avoid ABI and stdlib incompatibilities.

[twkx]

@Lotte1990 double check you use the exact same .h files in your project that the ones used to compile the library itself (extract those .h from the cloned tflite repository to be sure).

Also, as @jdduke said, be carefull to use the same stdlib implementation in your project than the one used in the tensorflowlite library.

[Lotte1990]

Thanks for the quick reply. I'm using the same git checkout for (1) compiling tensorflowlite.dll (2) the header files (3) exporting the .tflite file. I'm no expert regarding the library stuff. I tried adding different lines in different combinations to CMakeLists.txt, such as set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -stdlib=libc++") and set(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} -stdlib=libc++ -lc++abi"), but without any success.
I'm not getting any error, the program just crashes, which makes it quite hard to figure out what the problem is. It would be great if you could point me in the right direction.
As I mentioned before, the same code works perfectly on Ubuntu 18.04 using libtensorflowlite.so compiled on a Ubuntu system, so I have a very strong feeling that this is indeed a library issue...

[iwatake2222]

@Lotte1990
Do you use MSVC and run your program with Debug configuration?
If so, you can try Release configuration. I actually experienced unexpected crash with Debug configuration on Visual Studio 2017. With Release configuration, my program worked well.

[Lotte1990]

I use CMake to generate a .sln file that I open in MSVC. Then I press right-click --> build on the solution. Nothing else. I tried set(CMAKE_BUILD_TYPE "Debug") and set(CMAKE_BUILD_TYPE "Release") in CMakeLists.txt, but the result is the same. Any other suggestions?

[iwatake2222]

You can select that configuration using UI in Visual Studio (I'm not sure CMAKE_BUILD_TYPE makes an effect to configuration for Visual Studio). Please find a drop box just under a menu bar in Visual Studio, and select Release. Also make sure you use x64 architecture (not x86). You can also find a drop box to select architecture next to the drop box to select configuration. It's also located under the menu bar in Visual Studio.

[Lotte1990]

Debug config was indeed the problem. When I compile everything with release config, it works great. As you suspected, Visual Studio completely ignores CMAKE_BUILD_TYPE.

[PharrellWANG]

You can select that configuration using UI in Visual Studio (I'm not sure CMAKE_BUILD_TYPE makes an effect to configuration for Visual Studio). Please find a drop box just under a menu bar in Visual Studio, and select Release. Also make sure you use x64 architecture (not x86). You can also find a drop box to select architecture next to the drop box to select configuration. It's also located under the menu bar in Visual Studio.

Is there any way to build tensorflowlite.dll for windows X86 support?

[jdduke]

I don't think 32-bit (x86) Windows build are supported directly. There is some additional information here.

[brookman]

@Lotte1990 How exactly did you use CMAKE to create an .sln file?

[Lotte1990]

I used CMake to create an .sln file for my own project that uses TensorFlow Lite, not for TensorFlow Lite itself. For TensorFlow Lite, unfortunately, you still need to use Bazel.

[brookman]

I see. Thank you.

[ValYouW]

@jdduke I see that build_def.bzl on master still don't have features = ["windows_export_all_symbols"] is there any problem with this "fix"? it seems to be working. Also any idea why the dll is quite big (~15MB) while so for Android are fairly small? thx

[jdduke]

Thanks for flagging, I'll go ahead and land this for the C++ .dll target.

I'm curious if you're able to use the C API/dll in lite/c:tensorflowlite_c? That should have finer grained symbol export, has a simpler header export story and also avoids many of the C++ ABI issues with compatibility.

[ValYouW]

@jdduke Thanks, I see this fix hasn't made it to v2.2.0 :(
Yes, I was able to build the C dll and use it in my object detector, conversion was pretty much straight forward and the dll size is only ~1.4MB.
Thx!

[jdduke]

@jdduke Thanks, I see this fix hasn't made it to v2.2.0 :(

Yeah, unfortunately the 2.2 branch was only accepting P0/P1 issues at that point. Should be in the next release.

[OmarJay1]

Hello, I got the latest version of TF and tensorflow/lite/build_def.bzl seems to be slightly different than listed above. I try to build and it says there are multiple values for argument features.

ERROR: C:/temp/tensorflow/tensorflow/lite/build_def.bzl:157:24: Traceback (most recent call last):
File "C:/temp/tensorflow/tensorflow/lite/BUILD", line 610
tflite_cc_shared_object(<5 more arguments>)
File "C:/temp/tensorflow/tensorflow/lite/build_def.bzl", line 157, in tflite_cc_shared_object
tf_cc_shared_object(name = name, <7 more arguments>)
tf_cc_shared_object() got multiple values for keyword argument 'features'

Anything obviously wrong? Thanks.

def tflite_cc_shared_object(
name,
copts = tflite_copts(),
linkopts = [],
linkstatic = 1,
per_os_targets = False,
**kwargs):
"""Builds a shared object for TFLite."""
tf_cc_shared_object(
name = name,
copts = copts,
linkstatic = linkstatic,
linkopts = linkopts + tflite_jni_linkopts(),
framework_so = [],
per_os_targets = per_os_targets,
features = ["windows_export_all_symbols"],
**kwargs,

)

[ValYouW]

@OmarJay1 Check commit fa32891 to see the correct files, it should work on "master".
Regardless, as jdduke recommended above, you can try and use the C library, it worked pretty well for me (made a youtube video showing how to build and use)

[OmarJay1]

Thanks ValYouW, I watched your video and I was doing something really dumb. I wasn't running python configure.py.