Add the ability to provide custom-built libtensorflow binary

[nkreeger]

Users should be able to roll-their-own libtensorflow binary to optimize for the hardware they are running on.

This issue will track this feature.

[hartatovich]

@kangyizhang
Regards #1104 (comment)

@hartatovich ah sorry I missed the link of the tracking issue: #571

The current workflow is that:

1. this line https://github.com/tensorflow/tfjs-node/blob/master/scripts/install.js#L111 will download and unpack the raw binary based on your os (windows/linux/mac, cpu or gpu). If you want to use your own binary you will need to change this part to that it doesn't do anything.
2. node-gyp will bind the downloaded binary based on config is https://github.com/tensorflow/tfjs-node/blob/master/binding.gyp, to use your own binary you will need to config node-gyp to bind with your own binary.

Please feel free to let me know if you have any question. This tracking issue fits the topic better: #571

We might switch to deliver pre-built binary which changes the workflow, so I would love to know more about your production use-case. Real-world server side uses cases will help us prioritize features on Node.js :-)

What do you think about just changing the files
libtensorflow_framework.so and libtensorflow.so at node_modules/@tensorflow/rfjs-node/build/release

maybe self compiling from the source then quick replace would do the trick?

right now my problem is that I'm trying to get the proper .so files for my macos

basically, my error is
Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.2 AVX AVX2 FMA
so that's why I'm trying to use custom binary on my macos

[kangyizhang]

@hartatovich
Yes, you can just change the files libtensorflow_framework.so and libtensorflow.so at node_modules/@tensorflow/rfjs-node/build/release, and run node-gyp rebuild to bind your own binary. But you might accidentally wipe it out when you run yarn or npm install in your repo.

I personally would recommend

1. folking the https://github.com/tensorflow/tfjs-node repo locally,
2. add your binary (libtensorflow.so and libtensorflow_framework.so at /build/release)
3. delete the action in install.js, and change install action in package.json
4. use publish-local command at https://github.com/tensorflow/tfjs-node/blob/master/package.json#L26 to publish your customized tfjs-node package.
5. then you can run yalk link @tensorflow/tfjs-node in your own repo to use this customized tfjs-node package.

Besides, when libtensorflow 1.14 comes out, we'll release a new version of tfjs-node, which will have AVX and SSE optimization included.

[hartatovich]

Hi @kangyizhang ,

So i'm trying something else.. i'm tying to work with tfjs-node-gpu on my ubuntu machine with nvidia 2060

I'm getting this error

2019-06-16 22:01:26.662055: E tensorflow/stream_executor/cuda/cuda_dnn.cc:334] Could not create cudnn handle: CUDNN_STATUS_NOT_INITIALIZED
2019-06-16 22:01:26.662138: E tensorflow/stream_executor/cuda/cuda_dnn.cc:342] Possibly insufficient driver version: 410.48.0

I'm not sure whats the problem and i'm not sure what version tfjs-node-gpu is using for tensor right now

is this issue related or am i missing something here

[kangyizhang]

hi @hartatovich

tfjs-node-gpu@1.1.2 is using tensorflow@1.13.1

What is your CUDA and cuDNN version? TensorFlow has strict version requirement for GPU (https://www.tensorflow.org/install/gpu#software_requirements)

[hartatovich]

@kangyizhang according to this

https://github.com/mind/wheels

i'm thinking about going on 1.4.1 (just because its the lower one on gpu with 7.0 Compute Capability - but i'm taking a big guess here ) (and it got high compatibility with different cuda version {i'm not sure if thats good or bad} )

TF | CUDA | cuDNN | Compute Capability
1.4.1 | 8.0/9.0/9.1 | 6.0/7.0 | 3.7, 6.0, 7.0

i'm not sure at this point i really care about what version as long it will be on the GPU (ideally optimised build for production usage on the GPU)
(and its ok for me just to replace the current *.so files if that will work for all version}

{I will also be using posenet 2.0 }

is tfjs-node-gpu can work with any *.so files (if i replace them ) ?

{thinking out loud here to get any kind of version running on GPU ubuntu 16.04 nvidia 2060 rtx}

{at this point after a lot of playing around I can install any cuda /cudnn version i will need , just a lot of the pre wheels build are not always working - each and its own errors }

[kangyizhang]

@hartatovich
It will not work with TF@1.4. tfjs-node started with TF@1.8.x which has eager mode in the c library. Besides some ops which are supported in current tfjs-node, are not available in earlier version.