Verify setup & build on Windows #2

danomatika · 2021-02-10T20:39:00Z

This addon was initially developed on Linux and macOS, but has not been tested on Windows. The layout, scripting, and project support files are set up, more or less, but we haven't been able to verify yet.

If you are a Windows developer, please let us know if this addon is working as expected in:

Visual Studio, 64-bit
Msys2, 64-bit

paul-ferragut · 2021-02-13T16:23:53Z

I didn't manage to compile the examples with Visual Studio 64bits
First I was missing files from https://storage.googleapis.com/tensorflow/libtensorflow/libtensorflow-cpu-windows-x86_64-2.4.0.zip ; tensorflow/c/c_api_macros.h, tensorflow/c/tf_tstring.h, tensorflow/core/platform/ctstring.h, tensorflow/core/platform/ctstring_internal.h
After adding those files, I get errors in cppflow/model.h "op_idx", "op_name" undeclared identifier.
Then, I tried to comment the part with the undeclared identifier and got cannot include file "sys/errno.h" in ofxTensorFlow2Utils.cpp

danomatika · 2021-02-13T16:28:01Z

Did you install libtensorflow with the download script? If not, the required layout is basically:

libs/tensorflow/
libs/tensorflow//LICENSE
libs/tensorflow//include
libs/tensorflow//include/tensorflow
libs/tensorflow//include/tensorflow/core
libs/tensorflow//include/tensorflow/core/platform
libs/tensorflow//include/tensorflow/core/platform/ctstring.h
libs/tensorflow//include/tensorflow/core/platform/ctstring_internal.h
libs/tensorflow//include/tensorflow/c
libs/tensorflow//include/tensorflow/c/c_api_experimental.h
libs/tensorflow//include/tensorflow/c/c_api.h
libs/tensorflow//include/tensorflow/c/tf_status.h
libs/tensorflow//include/tensorflow/c/tf_attrtype.h
libs/tensorflow//include/tensorflow/c/tf_datatype.h
libs/tensorflow//include/tensorflow/c/tf_file_statistics.h
libs/tensorflow//include/tensorflow/c/c_api_macros.h
libs/tensorflow//include/tensorflow/c/tf_tensor.h
libs/tensorflow//include/tensorflow/c/tensor_interface.h
libs/tensorflow//include/tensorflow/c/tf_tstring.h
libs/tensorflow//include/tensorflow/c/eager
libs/tensorflow//include/tensorflow/c/eager/c_api_experimental.h
libs/tensorflow//include/tensorflow/c/eager/dlpack.h
libs/tensorflow//include/tensorflow/c/eager/c_api.h
libs/tensorflow//THIRD_PARTY_TF_C_LICENSES
libs/tensorflow//.gitkeep
libs/tensorflow//lib

with the libraries in lib going into:

libs/tensorflow//lib/vs

As for sys/errno.h, yes comment that out and also comment the setLogLevel stuff in ofxTensorFlow2Utils.h & ofxTensorFlow2Utils.cpp.

danomatika · 2021-02-13T16:29:30Z

After adding those files, I get errors in cppflow/model.h "op_idx", "op_name" undeclared identifier.

Also, can you confirm the cppflow sources are in libs/cppflow? If the folder is empty, you need to init and update the submodule.

paul-ferragut · 2021-02-13T21:32:39Z

Did you install libtensorflow with the download script? If not, the required layout is basically:

libs/tensorflow/
libs/tensorflow//LICENSE
libs/tensorflow//include
libs/tensorflow//include/tensorflow
libs/tensorflow//include/tensorflow/core
libs/tensorflow//include/tensorflow/core/platform
libs/tensorflow//include/tensorflow/core/platform/ctstring.h
libs/tensorflow//include/tensorflow/core/platform/ctstring_internal.h
libs/tensorflow//include/tensorflow/c
libs/tensorflow//include/tensorflow/c/c_api_experimental.h
libs/tensorflow//include/tensorflow/c/c_api.h
libs/tensorflow//include/tensorflow/c/tf_status.h
libs/tensorflow//include/tensorflow/c/tf_attrtype.h
libs/tensorflow//include/tensorflow/c/tf_datatype.h
libs/tensorflow//include/tensorflow/c/tf_file_statistics.h
libs/tensorflow//include/tensorflow/c/c_api_macros.h
libs/tensorflow//include/tensorflow/c/tf_tensor.h
libs/tensorflow//include/tensorflow/c/tensor_interface.h
libs/tensorflow//include/tensorflow/c/tf_tstring.h
libs/tensorflow//include/tensorflow/c/eager
libs/tensorflow//include/tensorflow/c/eager/c_api_experimental.h
libs/tensorflow//include/tensorflow/c/eager/dlpack.h
libs/tensorflow//include/tensorflow/c/eager/c_api.h
libs/tensorflow//THIRD_PARTY_TF_C_LICENSES
libs/tensorflow//.gitkeep
libs/tensorflow//lib

with the libraries in lib going into:

libs/tensorflow//lib/vs

As for sys/errno.h, yes comment that out and also comment the setLogLevel stuff in ofxTensorFlow2Utils.h & ofxTensorFlow2Utils.cpp.

I tried to use the download scripts but it wasn't working, I don't know if it something wrong with my configuration. So I have added the files missing manually with the layout. Now the errors in cppflow/model.h "op_idx", "op_name" undeclared identifier disappeared :)
sys/errno.h commented out worked without other complications
I had an issue with

#ifdef TARGET_WIN32
/// Windows doesn't provide setenv(), use this _putenv_s() wrapper from:
/// https://stackoverflow.com/a/23616164/2146055
static setenv(const char *name, const char *value, int overwrite) {

changed it to: int setenv(const char *name, const char *value, int overwrite) {
and it worked.

I have the library in the folder libs/tensorflow//lib/vs as shown in your layout
Now it seems I have one last issue;

Severity Code Description Project File Line Suppression State
Error LNK2001 unresolved external symbol "public: class std::vector<class cppflow::tensor,class std::allocator > __cdecl cppflow::model::operator()(class std::vector<class std::tuple<class std::basic_string<char,struct std::char_traits,class std::allocator >,class cppflow::tensor>,class std::allocator<class std::tuple<class std::basic_string<char,struct std::char_traits,class std::allocator >,class cppflow::tensor> > >,class std::vector<class std::basic_string<char,struct std::char_traits,class std::allocator >,class std::allocator<class std::basic_string<char,struct std::char_traits,class std::allocator > > >)" (??Rmodel@cppflow@@qeaa?AV?$vector@Vtensor@cppflow@@v?$allocator@Vtensor@cppflow@@@std@@@std@@v?$vector@V?$tuple@V?$basic_string@DU?$char_traits@D@std@@v?$allocator@D@2@@std@@Vtensor@cppflow@@@std@@v?$allocator@V?$tuple@V?$basic_string@DU?$char_traits@D@std@@v?$allocator@D@2@@std@@Vtensor@cppflow@@@std@@@2@@3@V?$vector@V?$basic_string@DU?$char_traits@D@std@@v?$allocator@D@2@@std@@v?$allocator@V?$basic_string@DU?$char_traits@D@std@@v?$allocator@D@2@@std@@@2@@3@@z) example_basics C:\Users\Paul\Documents\Coding\of_v0.11.0_vs2017_release\of_v0.11.0_vs2017_release\addons\ofxTensorFlow2\example_basics\ofxTF2Model.obj 1

danomatika · 2021-02-13T21:45:54Z

I tried to use the download scripts but it wasn't working, I don't know if it something wrong with my configuration.

They require a Unix shell, so they would work in Msys as long as curl and unzip are installed. I'm not sure, but maybe they would work in Powershell?

If you have any time later, send any script errors our way too. :)

Now it seems I have one last issue;

That looks like some sort of issue with cppflow templating. Since cppflow is built into the project, then maybe something that was declared in one of the ofxTensorFlow2 src files but the implementation is not found. I would think that we would have seen this on other platforms too, though...

UPDATE: Another option is a bug with cppflow. Since I assume you checked out the submodule, it's the correct commit form cppflow upstream. This is important right now as the TF2 support for cppflow is in a branch right now.

danomatika · 2021-02-13T21:56:13Z

The sys/errno.h and setenv errors should be fixed in the latest commit.

paul-ferragut · 2021-02-13T23:05:00Z

the error with the script is
"tar.exe: Error opening archive: Unrecognized archive format"
if changed .tar.gz to .zip in the sh script it downloads but with a few missing files

I tried to start over and get cppflow using git submodule update --init --recursive
getting again errors undeclared identifiers 'op_name', 'op_idx' in model.h

danomatika · 2021-02-14T00:58:47Z

if changed .tar.gz to .zip in the sh script it downloads but with a few missing files

Ah ok, that makes sense. They give the Windows download as a zip as most users wouldn't have tar available by default.

Hrmmm strange. As you noted, the Windows zips have fewer headers. Maybe they are not up to date? @bytosaur what do you think?

libtensorflow-cpu-windows-x86_64-2.4.0
libtensorflow-cpu-windows-x86_64-2.4.0/LICENSE
libtensorflow-cpu-windows-x86_64-2.4.0/include
libtensorflow-cpu-windows-x86_64-2.4.0/include/tensorflow
libtensorflow-cpu-windows-x86_64-2.4.0/include/tensorflow/c
libtensorflow-cpu-windows-x86_64-2.4.0/include/tensorflow/c/c_api.h
libtensorflow-cpu-windows-x86_64-2.4.0/include/tensorflow/c/tf_status.h
libtensorflow-cpu-windows-x86_64-2.4.0/include/tensorflow/c/tf_attrtype.h
libtensorflow-cpu-windows-x86_64-2.4.0/include/tensorflow/c/tf_datatype.h
libtensorflow-cpu-windows-x86_64-2.4.0/include/tensorflow/c/tf_tensor.h
libtensorflow-cpu-windows-x86_64-2.4.0/include/tensorflow/c/eager
libtensorflow-cpu-windows-x86_64-2.4.0/include/tensorflow/c/eager/c_api.h
libtensorflow-cpu-windows-x86_64-2.4.0/THIRD_PARTY_TF_C_LICENSES
libtensorflow-cpu-windows-x86_64-2.4.0/lib
libtensorflow-cpu-windows-x86_64-2.4.0/lib/tensorflow.dll
libtensorflow-cpu-windows-x86_64-2.4.0/lib/tensorflow.lib

danomatika · 2021-02-14T01:42:40Z

Latest commit has fix for script zip file handling.

Also, I noticed that the Nightly Builds mentioned on the TF C download page don't have Windows builds.

paul-ferragut · 2021-02-14T11:59:03Z

The tensorflow download script is working now.
Currently I tried to copy the missing headers from the linux tensorflow library and commented out this part in cppflow/model.h

		/*
    for (int i=0; i<inputs.size(); i++) {

            // Operations
            const auto[op_name, op_idx] = parse_name(std::get<0>(inputs[i]));
            inp_ops[i].oper = TF_GraphOperationByName(this->graph.get(), op_name.c_str());
            inp_ops[i].index = op_idx;

            if (!inp_ops[i].oper)
                throw std::runtime_error("No operation named \"" + op_name + "\" exists");

            // Values
            inp_val[i] = std::get<1>(inputs[i]).get_tensor().get();
        }
		*/
		
        std::vector<TF_Output> out_ops(outputs.size());
        auto out_val = std::make_unique<TF_Tensor*[]>(outputs.size());
		/*
        for (int i=0; i<outputs.size(); i++) {

            const auto[op_name, op_idx] = parse_name(outputs[i]);
            out_ops[i].oper = TF_GraphOperationByName(this->graph.get(), op_name.c_str());
            out_ops[i].index = op_idx;

            if (!out_ops[i].oper)
                throw std::runtime_error("No operation named \"" + op_name + "\" exists");

        }
		*/

I am not sure what this part of code is doing?

This time the example compiled until the end but throwing an exception on launch 'tensorflow.pdb not loaded"
In the console:

2021-02-14 12:41:56.466284: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2021-02-14 12:41:56.489900: I tensorflow/cc/saved_model/reader.cc:32] Reading SavedModel from: data\model
2021-02-14 12:41:56.491334: I tensorflow/cc/saved_model/reader.cc:55] Reading meta graph with tags { serve }
2021-02-14 12:41:56.491412: I tensorflow/cc/saved_model/reader.cc:93] Reading SavedModel debug info (if present) from: data\model
2021-02-14 12:41:56.504371: I tensorflow/cc/saved_model/loader.cc:206] Restoring SavedModel bundle.
2021-02-14 12:41:56.521362: I tensorflow/cc/saved_model/loader.cc:190] Running initialization op on SavedModel bundle at path: data\model
2021-02-14 12:41:56.524460: I tensorflow/cc/saved_model/loader.cc:277] SavedModel load for tags { serve }; Status: success: OK. Took 34542 microseconds.

C:\Users\Paul\Documents\Coding\of_v0.11.0_vs2017_release\of_v0.11.0_vs2017_release\addons\ofxTensorFlow2\example_basics\bin\example_basics.exe (process 30340) exited with code -1.
To automatically close the console when debugging stops, enable Tools->Options->Debugging->Automatically close the console when debugging stops.
Press any key to close this window . . .

danomatika · 2021-02-15T16:19:16Z

This time the example compiled until the end but throwing an exception on launch 'tensorflow.pdb not loaded"

I think that's an optional file used for debugging, judging from this. I imagine libtensorflow is built for release and doesn't include this.

paul-ferragut · 2021-02-16T16:57:11Z

Latest commit has fix for script zip file handling.

Also, I noticed that the Nightly Builds mentioned on the TF C download page don't have Windows builds.

tensorflow/tensorflow#46538 hopefully this should be fixed soon

liquidzym · 2021-10-16T12:34:13Z

i'm just did complied the lastest dev branch, vs2017& win 10 ,with a bit of change

bytosaur · 2021-10-16T12:58:16Z

hey @liquidzym,

that sounds wonderful! Let me know if you have a repo up which I can review and pull from :)

liquidzym · 2021-10-16T13:12:11Z

sorry,not yet, but for the ref you can have a look at this

download the windows GPU version https://www.tensorflow.org/install/lang_c
i'm using the default master branch which is https://github.com/serizba/cppflow/
i've have to changed some code since OF not fully support c++ 17 yet, according to this pull requests Use std::get instead of cpp17 tuple unpack serizba/cppflow#108
download the lastest cundd 8.2.4 not working with 8.04
that's all it's should be working!

jeffcrouse · 2022-01-05T23:11:47Z

One other thing that I didn't see here that ended up being the silver bullet for me to get the movenet example running...

Everything was compiling for me, but when I tried to run, I was getting an error saying it was unable to load the cuDNN DLL. After some googling, I found this page, suggesting that there is a very specific combination of TF/CUDA/cuDNN that is recommended.

For me, I had downloaded tensorflow_gpu-2.7.0, so according to that chart, that means I needed CUDA 11.2 and cuDNN 8.1 (note that there are multiple versions of cuDNN 8.1, so make sure to choose the version for CUDA 11.2)

I'll make a PR for the README once I get a little further along in this project.

danzeeeman · 2022-03-10T22:06:17Z

running into weird errors with this and windows 10

jeffcrouse · 2022-03-10T22:20:20Z

Hey Dan -- I'm actively using my fork with Windows. It's not polished up yet, but it works. https://github.com/jeffcrouse/ofxTensorFlow2

danomatika · 2022-03-10T22:22:52Z

@jeffcrouse Care to make a PR, including any required info added to the readme?

danzeeeman · 2022-03-10T23:02:46Z

@jeffcrouse I'm able to build this repo with openFrameworks main branch (thanks for the recommendations theo), I had to update cuDNN to a newer version but it still won't run completely.

danzeeeman · 2022-03-12T20:37:22Z

finally got it all working with this repo CUDA 11.4 and cuDNN for 11.4 and openFrameworks main branch with cpp17 support

danomatika · 2022-03-31T08:46:13Z

If things are working, we would be happy to take a PR on this. Let us know what to add to the readme, if you don't care to edit it yourselves.

jeffcrouse · 2022-04-02T00:31:17Z

I’m definitely going to do that once I get through this production. Maybe earlier. Thanks for the great add on!

danomatika · 2022-04-02T20:34:57Z

I’m definitely going to do that once I get through this production. Maybe earlier. Thanks for the great add on!

Understood. Thanks

Jonathhhan · 2022-06-02T20:34:47Z

I got it working with VS and the original branch. All I had to change was ofxTensorFlow2\libs\tensorflow\lib\msys to ofxTensorFlow2\libs\tensorflow\lib\vs.
I put the files from C:\Program Files\NVIDIA GPU Computing Toolkit\CUDNN\v8.4.1.50\bin and zlibwapi.dll into C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.7\bin.
I use Tensorflow 2.9 gpu (installed with the included script), CUDA 11.7 and CUDNN 8.4.1.50.

bytosaur · 2022-06-03T08:50:05Z

hey @Jonathhhan,
cool, thanks for the advice! I ll look into it ;)

danomatika · 2022-06-03T09:12:42Z

Please feel free to make a PR adding info which helps make this work, ie. a section for CUDA on Windows, etc. enohp ym morf tnes ----------- Dan Wilcox danomatika.com robotcowboy.com

…

On Jun 3, 2022, at 10:50 AM, paul ***@***.***> wrote: hey @Jonathhhan, cool, thanks for the advice! I ll look into it ;) — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.

bytosaur · 2022-06-13T14:25:43Z

hey @Jonathhhan,
I am sorry I dont have the resources for reproducing your approach. Anyhow, as danomatika said, PRs are very welcome.

Jonathhhan · 2022-06-13T16:46:00Z

@bytosaur I made a PR: #20

danomatika · 2022-10-20T09:32:21Z

I've added the steps from @Jonathhhan and the tested build config link from @jeffcrouse to a new widows section in the readme: https://github.com/zkmkarlsruhe/ofxTensorFlow2#windows

Please review this and let me know if there is anything to add/change. I'd like to release a new version of the addon soon.

danomatika · 2022-10-20T11:25:16Z

I've added the steps from @Jonathhhan and the tested build config link from @jeffcrouse to a new widows section in the readme: https://github.com/zkmkarlsruhe/ofxTensorFlow2#windows

Please review this and let me know if there is anything to add/change. I'd like to release a new version of the addon soon.

Ah well, I released 1.4.0 now anyway. :) We can always fix the readme in 1.4.1.

evelynoliv · 2023-03-21T20:01:31Z

Hi there,

I am facing missing external symbols when trying to compile both new projects or the examples:

These are the 3 first erros from 53 total:

Erro	LNK2019	símbolo externo não resolvido, __imp_TF_DataTypeSize, referenciado na função "public: class std::vector<__int64,class std::allocator<__int64> > __cdecl cppflow::tensor::get_data<__int64>(void)const " (??$get_data@_J@tensor@cppflow@@QEBA?AV?$vector@_JV?$allocator@_J@std@@@std@@XZ)	mySketch2	C:\Users\Superuber\Desktop\of_v20230321_vs_release\apps\myApps\mySketch2\ofxTensorFlow2Utils.obj	1	
Erro	LNK2019	símbolo externo não resolvido, __imp_TF_NewStatus, referenciado na função "public: static struct TF_Status * __cdecl cppflow::context::get_status(void)" (?get_status@context@cppflow@@SAPEAUTF_Status@@XZ)	mySketch2	C:\Users\Superuber\Desktop\of_v20230321_vs_release\apps\myApps\mySketch2\ofxTensorFlow2Utils.obj	1	
Erro	LNK2001	símbolo externo não resolvido __imp_TF_NewStatus	mySketch2

I checked the VS Project configuration, and it seems to be missing the libs folder. I tried to add it manually without success.

I also noticed that Project Generator is presenting a error message when trying to import the examples.

Could you please help?

danomatika added the help wanted Extra attention is needed label Feb 10, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Verify setup & build on Windows #2

Verify setup & build on Windows #2

danomatika commented Feb 10, 2021

paul-ferragut commented Feb 13, 2021

danomatika commented Feb 13, 2021 •

edited

Loading

danomatika commented Feb 13, 2021

paul-ferragut commented Feb 13, 2021

danomatika commented Feb 13, 2021 •

edited

Loading

danomatika commented Feb 13, 2021

paul-ferragut commented Feb 13, 2021

danomatika commented Feb 14, 2021

danomatika commented Feb 14, 2021

paul-ferragut commented Feb 14, 2021 •

edited

Loading

danomatika commented Feb 15, 2021 •

edited

Loading

paul-ferragut commented Feb 16, 2021

liquidzym commented Oct 16, 2021

bytosaur commented Oct 16, 2021

liquidzym commented Oct 16, 2021

jeffcrouse commented Jan 5, 2022

danzeeeman commented Mar 10, 2022

jeffcrouse commented Mar 10, 2022

danomatika commented Mar 10, 2022

danzeeeman commented Mar 10, 2022

danzeeeman commented Mar 12, 2022

danomatika commented Mar 31, 2022

jeffcrouse commented Apr 2, 2022

danomatika commented Apr 2, 2022

Jonathhhan commented Jun 2, 2022 •

edited

Loading

bytosaur commented Jun 3, 2022

danomatika commented Jun 3, 2022 via email

bytosaur commented Jun 13, 2022

Jonathhhan commented Jun 13, 2022

danomatika commented Oct 20, 2022 •

edited

Loading

danomatika commented Oct 20, 2022

evelynoliv commented Mar 21, 2023

Verify setup & build on Windows #2

Verify setup & build on Windows #2

Comments

danomatika commented Feb 10, 2021

paul-ferragut commented Feb 13, 2021

danomatika commented Feb 13, 2021 • edited Loading

danomatika commented Feb 13, 2021

paul-ferragut commented Feb 13, 2021

danomatika commented Feb 13, 2021 • edited Loading

danomatika commented Feb 13, 2021

paul-ferragut commented Feb 13, 2021

danomatika commented Feb 14, 2021

danomatika commented Feb 14, 2021

paul-ferragut commented Feb 14, 2021 • edited Loading

danomatika commented Feb 15, 2021 • edited Loading

paul-ferragut commented Feb 16, 2021

liquidzym commented Oct 16, 2021

bytosaur commented Oct 16, 2021

liquidzym commented Oct 16, 2021

jeffcrouse commented Jan 5, 2022

danzeeeman commented Mar 10, 2022

jeffcrouse commented Mar 10, 2022

danomatika commented Mar 10, 2022

danzeeeman commented Mar 10, 2022

danzeeeman commented Mar 12, 2022

danomatika commented Mar 31, 2022

jeffcrouse commented Apr 2, 2022

danomatika commented Apr 2, 2022

Jonathhhan commented Jun 2, 2022 • edited Loading

bytosaur commented Jun 3, 2022

danomatika commented Jun 3, 2022 via email

bytosaur commented Jun 13, 2022

Jonathhhan commented Jun 13, 2022

danomatika commented Oct 20, 2022 • edited Loading

danomatika commented Oct 20, 2022

evelynoliv commented Mar 21, 2023

danomatika commented Feb 13, 2021 •

edited

Loading

danomatika commented Feb 13, 2021 •

edited

Loading

paul-ferragut commented Feb 14, 2021 •

edited

Loading

danomatika commented Feb 15, 2021 •

edited

Loading

Jonathhhan commented Jun 2, 2022 •

edited

Loading

danomatika commented Oct 20, 2022 •

edited

Loading