Unable to build tensorflow.dll / tensorflow_cc.dll

[Scorillo47]

First - thanks for the script - it works great in generating a wheel!

However, I have trouble generating a tensorflow.dll/tensorflow_cc.dll (not sure which one I need?). I need the C++ API to implement custom operations, which now seems to be broken since the switch from CMake to Bazel was made.

So with TF 1.12.0 I tried specifying a target such as //tensorflow:tensorflow_cc.dll or //tensorflow:tensorflow.dll but i get an error:

Any idea how can I generate the DLL?

BTW, your script mentions generating .so files in Windows, however, I am not sure how to use these if I want to build custom GPU operations in CUDA for use the GPU version of TensorFlow (should I somehow run TensorFlow under MSYS?). This was working just fine before (with CMake) but it's unclear how to make it work with Bazel

Any response is appreciated.

[StevenRoodhorst]

Can you post your specific commands/instructions? It seems that you are constructing the incorrect library name.

EDIT:

if you want to build a C++ API, you need //tensorflow:libtensorflow_cc.so. You can then link with CMake against the libtensorflow_cc.so.if.lib which will link against the so (which is in fact a dll with the incorrect extension). I think there are more close topics about this.

[Scorillo47]

Hi, I was attempting to use this command

powershell -File .\build.ps1 -BuildCppAPI -BazelBuildParameters "--config=opt --config=cuda --define=no_tensorflow_py_deps=true --copt=-nvcc_options=disable-warnings //tensorflow:tensorflow_cc.dll --verbose_failures"

Perhaps the .dll extension is not supported yet in branch 1.12r. Will try renaming the .so to .dll per your suggestion - thanks!

[Scorillo47]

Thanks. Renaming the .so to .dll worked!

Where do I get access to libtensorflow_cc.so.if.lib ? I don't see it being generated during the linking process above.

I get the following linking errors while attempting to generate a DLL for my custom op:

[...]
1>example.obj : error LNK2019: unresolved external symbol "void __cdecl tensorflow::CheckNotInComputeAsync(class tensorflow::OpKernelContext *,char const *)" (?CheckNotInComputeAsync@tensorflow@@YAXPEAVOpKernelContext@1@PEBD@Z) referenced in function "public: virtual void __cdecl ExampleOp<struct Eigen::ThreadPoolDevice,int>::Compute(class tensorflow::OpKernelContext *)" (?Compute@?$ExampleOp@UThreadPoolDevice@Eigen@@h@@UEAAXPEAVOpKernelContext@tensorflow@@@z)
1>example.obj : error LNK2019: unresolved external symbol "public: struct Eigen::ThreadPoolDevice const & __cdecl tensorflow::OpKernelContext::eigen_device(void)const " (??$eigen_device@UThreadPoolDevice@Eigen@@@OpKernelContext@tensorflow@@QEBAAEBUThreadPoolDevice@Eigen@@xz) referenced in function "public: virtual void __cdecl ExampleOp<struct Eigen::ThreadPoolDevice,int>::Compute(class tensorflow::OpKernelContext *)" (?Compute@?$ExampleOp@UThreadPoolDevice@Eigen@@h@@UEAAXPEAVOpKernelContext@tensorflow@@@z)
1>F:_git\tf_custom_op_windows\x64\Debug\tf.ZeroOut.gpu3.dll : fatal error LNK1120: 16 unresolved externals

I tried to generate an import LIB from the libtensorflow_cc.dll using dll2lib.cmd but this doesn't help the DLL doesn't seem to export any APIs like the ones referenced above. Just to pick an example - see tensorflow::CheckNotInComputeAsync

I have attached the exports from the DLL and the linking error

libgen.log
link_errors.txt
include_dirs.txt

[Scorillo47]

Nevermind, looks like I may need to add more APIs into the tf_exported_symbols_msvc.lds file - let me try this next ...

?StrCat@strings@tensorflow@@YA?AV?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@AEBVAlphaNum@12@@Z
??0Status@tensorflow@@QEAA@W4Code@error@1@Vstring_view@absl@@@Z
??0KernelDefBuilder@tensorflow@@QEAA@PEBD@Z
??1KernelDefBuilder@tensorflow@@QEAA@XZ
?Device@KernelDefBuilder@tensorflow@@QEAAAEAV12@PEBD@Z
?TypeConstraint@KernelDefBuilder@tensorflow@@QEAAAEAV12@PEBDW4DataType@2@@Z
?Build@KernelDefBuilder@tensorflow@@QEAAPEBVKernelDef@2@XZ
??0OpKernel@tensorflow@@QEAA@PEAVOpKernelConstruction@1@@Z
??1OpKernel@tensorflow@@UEAA@XZ
?input@OpKernelContext@tensorflow@@QEAAAEBVTensor@2@H@Z
?allocate_output@OpKernelContext@tensorflow@@QEAA?AVStatus@2@HAEBVTensorShape@2@PEAPEAVTensor@2@@Z
?CtxFailure@OpKernelContext@tensorflow@@QEAAXPEBDHAEBVStatus@2@@Z
?CtxFailureWithWarning@OpKernelContext@tensorflow@@QEAAXPEBDHAEBVStatus@2@@Z
?InitInternal@OpKernelRegistrar@kernel_factory@tensorflow@@AEAAXPEBVKernelDef@3@Vstring_view@absl@@P6APEAVOpKernel@3@PEAVOpKernelConstruction@3@@Z@Z
?CheckNotInComputeAsync@tensorflow@@YAXPEAVOpKernelContext@1@PEBD@Z
??$eigen_device@UThreadPoolDevice@Eigen@@@OpKernelContext@tensorflow@@QEBAAEBUThreadPoolDevice@Eigen@@XZ

[Scorillo47]

Looks like adding these symbols to the .lds file worked - I can now link the custom op! (but I don't have exports which is a separate issue). Will close this thread

[duocang]

@Scorillo47 HIi. How do you use tensorflow.dll? I mean I found hello_c_api is using the dll. But where can I find the functions' names and how to use them?

Can you maybe share some code or teach me how to use the functions in tensorflow.dll?

Thank you.

[SeanCho1996]

@steroes Hi, I'm following your instructions and I succeeded building the libtensorflow_cc.so. I renamed it as libtensorflow_cc.dll,but the same as @Scorillo47 , I did not find the file libtensorflow_cc.so.if.lib, instead I got a file named liblibtensorflow_cc.so.ifso. So should I just rename this .ifso to .lib and link it with the .dll file ?
Thank you!

[terryzhao127]

@SeanCho1996 Check this.

[duocang]

@SeanCho1996 check this guy's repo: Create .lib file from .dll for windows

[SeanCho1996]

@guikarist Thank you for your help! I followed it and I succeeded build the C++ library. Furthermore, I wrote a simple test C++ program, and it can run correctly.
But when I try to link libprotobuf.a which I have renamed as libprotobuf.lib, it report several errors about lack of external symbols, so I added the missing ones into the tf_exported_symbols_msvc.lds, and rebuilt the dll and lib files, but when I ran the program it reported this error:

Entry Point Not Found
so do you have any idea on solving this third-party lib problem? Thank you!

[SeanCho1996]

@guikarist sorry I forgot to paste my code

#include "stdafx.h"

#include "tensorflow/core/public/session.h"
#include "tensorflow/core/platform/env.h"
#include "tensorflow/core/platform/protobuf.h"

#include <vector>
#include <string>
#include <fstream>
#include <iostream>

using namespace tensorflow;
using namespace std;

typedef struct Image3d_u16
{
	int	nRows;
	int	nCols;
	int nChns;
	unsigned short* pData;
};


int predict(Session* _session, const vector<Image3d_u16>* input_feature, vector<Image3d_u16>* prediction)
{
	int nBatch = input_feature->size();

	int nRowIn = (*input_feature)[0].nRows;
	int nColIn = (*input_feature)[0].nCols;
	int nChnIn = (*input_feature)[0].nChns;

	Tensor x(tensorflow::DT_UINT16, TensorShape({ nBatch, nRowIn, nColIn, nChnIn }));
	auto x_map = x.tensor<uint16, 4>();

	for (int n = 0; n < nBatch; n++)
		for (int i = 0; i < nRowIn; i++)
			for (int j = 0; j < nColIn; j++)
				for (int k = 0; k < nChnIn; k++)
					x_map(n, i, j, k) = (*input_feature)[n].pData[i * nColIn * nChnIn + j * nChnIn + k];

	Tensor y(tensorflow::DT_INT32, TensorShape());
	y.scalar<int>()() = 1;

	std::vector<std::string> _input_tensor_names;
	std::vector<std::string> _output_tensor_names;
	_input_tensor_names.push_back("data_input");
	_input_tensor_names.push_back("loop_count");
	_output_tensor_names.push_back("data_output");

	std::vector<std::pair<std::string, Tensor> > inputs;
	inputs.push_back(std::pair<std::string, Tensor>(_input_tensor_names[0], x));
	inputs.push_back(std::pair<std::string, Tensor>(_input_tensor_names[1], y));

	// The session will initialize the outputs
	std::vector<Tensor> outputs;
	tensorflow::Status status = _session->Run(inputs, _output_tensor_names, {}, &outputs);
	if (!status.ok()) {
		std::cout << "ERROR: prediction failed..." << status.ToString() << std::endl;
		return -1;
	}

	// Fetch the first tensor
	Tensor t = outputs[0];
	assert(t.shape().dims() == 4);

	int nRowOut = t.shape().dim_size(1);
	int nColOut = t.shape().dim_size(2);
	int nChnOut = t.shape().dim_size(3);

	auto t_map = t.tensor<uint16, 4>();

	for (int n = 0; n < nBatch; n++)
		for (int i = 0; i < nRowOut; i++)
			for (int j = 0; j < nColOut; j++)
				for (int k = 0; k < nChnOut; k++)
					((*prediction)[n]).pData[i * nColOut * nChnOut + j * nChnOut + k] = t_map(n, i, j, k);

	return 0;
}


int main()
{
	SessionOptions options;
	Session* _session;

	ConfigProto* config = &options.config;

	int gpu_device_count = 1;
	(*config->mutable_device_count())["GPU"] = gpu_device_count;
	config->set_allow_soft_placement(true);
	config->set_log_device_placement(false);

	Status status = NewSession(options, &_session);

	GraphDef _graphdef;
	string model_path = "model_frozen.pb";
	Status status_load = ReadBinaryProto(Env::Default(), model_path, &_graphdef);

	if (!status_load.ok()) {
		std::cout << status_load.ToString() << std::endl;
		return -1;
	}

	Status status_create = _session->Create(_graphdef);

	const int row = 192;
	const int col = 192;
	const int chn = 4;

	Image3d_u16* imgsIn = new Image3d_u16[1];
	Image3d_u16* imgsOut = new Image3d_u16[1];

	vector<Image3d_u16> vecIn, vecOut;

	imgsIn[0].nRows = row;
	imgsIn[0].nCols = col;
	imgsIn[0].nChns = chn;
	imgsIn[0].pData = new unsigned short[row * col * chn];

	ifstream ifs("../05/in_u16_b1k_00_0.raw", ios::in | ios::binary);
	ifs.read((char*)imgsIn[0].pData, row * col * chn * sizeof(unsigned short));
	ifs.close();

	vecIn.push_back(imgsIn[0]);

	imgsOut[0].nRows = row;
	imgsOut[0].nCols = col;
	imgsOut[0].nChns = 1;
	imgsOut[0].pData = new unsigned short[row * col * 1];

	memset((void*)imgsOut[0].pData, 0, row * col * 1 * sizeof(unsigned short));

	vecOut.push_back(imgsOut[0]);

	predict(_session, &vecIn, &vecOut);

	for (int n = 0; n < vecOut.size(); n++)
	{
		ofstream ofs("../05/out_u16_b1k.raw", ios::out | ios::binary);
		ofs.write((char*)vecOut[n].pData, row * col * 1 * sizeof(unsigned short));
		ofs.close();
	}

    return 0;
}

[terryzhao127]

@SeanCho1996
AFAIK, now the third-party libraries can only be used by compiling with their source files rather than linking their libraries. So linking *.a or *.lib doesn't work.