Cannot run @tensorflow/tfjs-node on OSX

[mprinc]

TensorFlow.js version

"name": "@tensorflow/tfjs-node",
"version": "1.7.0",

macOS Catalina
v10.15.03

node -v
Anything between the v12.11.1 and @latest (currently v13.10.1)

Tested with:

    node/10.16.3
    node/11.15.0
    node/12.0.0
    node/12.8.1
    node/12.10.0
    node/12.11.1
    node/12.12.0
    node/13.9.0
    node/13.10.1

Describe the problem or feature request

@kangyizhang I tried all various suggestions from the (closed) #2003 still issues with latest node.

Still the same error "Error: Cannot find module '.../node_modules/@tensorflow/tfjs-node/lib/napi-v5/tfjs_binding.node'"

With "@tensorflow/tfjs-node" I managed now with the latest version 1.7.0, but ONLY using earlier node:

node -v
v12.10.0
(the next v12.11.1 is crashing, also the same with the currently latest node v13.10.1)

Obviously, this is far from ideal as we need to propagate node version changes on the production servers, and as we have both frontend and backend tf, we might need to reduce frontend version to not risk compatibility issues or keep them version-async (as with the previous versions it was crashing on the backend so I HAD to reduce tfjs on v1.5.2, etc)

And even then I get the incompatibility issue with compiling aspect:

node-pre-gyp info This Node instance does not support builds for N-API version 5

And I get the more expected warning :) "Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 AVX512F FMA"

Code to reproduce the bug / link to feature request

Here is the minimal code:

package.json

{
  "name": "tfjs-node-test",
  "description": "backend test of tfjs",
  "version": "0.0.1",
  "private": true,
  "repository": {
    "type": "git",
    "url": ""
  },
  "scripts": {
    "start": "node dist/index 8002",
    "clean_full": "rm -r node_modules/; rm package-lock.json; rm -r dist/; rm yarn.lock; rm -r puzzles/*/*/node_modules; rm -r puzzles/*/*/dist; rm -r puzzles/*/*/yarn.lock; rm -r puzzles/*/*/package-lock.json"
  },
  "main": "dist/index.js",
  "module": "dist/index.js",
  "dependencies": {
    "@tensorflow/tfjs-node": "^1.7.0"
  },
  "devDependencies": {
    "@types/node": "^13.9.0"
  }
}

ts

console.log("[tfjs-test] started");

import * as tfNode from '@tensorflow/tfjs-node';
let _tf = tfNode;

console.log("[tfjs-test] finished");

js

Here is also JS compiled version:

"use strict";
Object.defineProperty(exports, "__esModule", { value: true });
console.log("[tfjs-test] started");
const tfNode = require("@tensorflow/tfjs-node");
let _tf = tfNode;
console.log("[tfjs-test] finished");
//# sourceMappingURL=index.js.map

tsconfig.js

{
    "compilerOptions": {
        "target": "es6",
        "module": "commonjs",
        "moduleResolution": "node",
        "sourceMap": true,
        "rootDir": "./",
        "allowJs": true,
        "outDir": "dist",
        "preserveConstEnums": true,
        "emitDecoratorMetadata": true,
        "experimentalDecorators": true,
        "allowSyntheticDefaultImports": true,
        "removeComments": false,
        "noImplicitAny": false,
        "listFiles": false,
        "skipLibCheck": true,
        "lib": [
            "es6",
            "dom"
        ]
    },
    "allowSyntheticDefaultImports": false,
    "files": [
        "index.ts"
    ],
    "include": [
        "config"
    ],
    "exclude": [
        "**/*d.ts"
    ],
    "compileOnSave": false
}

[kangyizhang]

@mprinc

Hi Sasha, I just tried the library on my machine with node v12.16.0 and v13.11.0, it works with both version. So I will need some more information to find the issue on you machine.

1. can you provide what is inside the node_modules/@tensorflow/tfjs-node/deps and node_modules/@tensorflow/tfjs-node/lib, it would be great if you could provide nested folder info as well, it should not be too many files inside.
2. can you provide the logs during the installation of package @tensorflow/tfjs-node?
3. for the error of node-pre-gyp info This Node instance does not support builds for N-API version 5, it is a warning from a dependent library. Actually the library works for N-API version 5.

[mprinc]

Thank you @kangyizhang,

(Sorry for the delay, gmail classified email in forums, added better rules now)

For your convenience, here is the demo repo: https://github.com/mprinc/tfjs-node-test

Here is the content of the node_modules/@tensorflow/tfjs-node/deps/ (and I also attached zip, excluding heavy node_modules/@tensorflow/tfjs-node/deps/lib/*.dylib files):

zip -r tfjs-node_deps.zip node_modules/@tensorflow/tfjs-node/deps/*
  adding: node_modules/@tensorflow/tfjs-node/deps/LICENSE (deflated 65%)
  adding: node_modules/@tensorflow/tfjs-node/deps/THIRD_PARTY_TF_C_LICENSES (deflated 78%)
  adding: node_modules/@tensorflow/tfjs-node/deps/include/ (stored 0%)
  adding: node_modules/@tensorflow/tfjs-node/deps/lib/ (stored 0%)
  adding: node_modules/@tensorflow/tfjs-node/deps/include/tensorflow/ (stored 0%)
  adding: node_modules/@tensorflow/tfjs-node/deps/include/tensorflow/c/ (stored 0%)
  adding: node_modules/@tensorflow/tfjs-node/deps/include/tensorflow/c/c_api_experimental.h (deflated 71%)
  adding: node_modules/@tensorflow/tfjs-node/deps/include/tensorflow/c/c_api.h (deflated 78%)
  adding: node_modules/@tensorflow/tfjs-node/deps/include/tensorflow/c/tf_status.h (deflated 56%)
  adding: node_modules/@tensorflow/tfjs-node/deps/include/tensorflow/c/tf_attrtype.h (deflated 49%)
  adding: node_modules/@tensorflow/tfjs-node/deps/include/tensorflow/c/tf_datatype.h (deflated 54%)
  adding: node_modules/@tensorflow/tfjs-node/deps/include/tensorflow/c/tf_tensor.h (deflated 65%)
  adding: node_modules/@tensorflow/tfjs-node/deps/include/tensorflow/c/eager/ (stored 0%)
  adding: node_modules/@tensorflow/tfjs-node/deps/include/tensorflow/c/eager/c_api.h (deflated 74%)
  adding: node_modules/@tensorflow/tfjs-node/deps/lib/libtensorflow.dylib (deflated 80%)
  adding: node_modules/@tensorflow/tfjs-node/deps/lib/libtensorflow_framework.1.dylib (deflated 71%)
  adding: node_modules/@tensorflow/tfjs-node/deps/lib/libtensorflow.1.15.0.dylib (deflated 80%)
  adding: node_modules/@tensorflow/tfjs-node/deps/lib/libtensorflow.1.dylib (deflated 80%)
  adding: node_modules/@tensorflow/tfjs-node/deps/lib/libtensorflow_framework.1.15.0.dylib (deflated 71%)
  adding: node_modules/@tensorflow/tfjs-node/deps/lib/libtensorflow_framework.dylib (deflated 71%)

Here is the content of the node_modules/@tensorflow/tfjs-node/lib/* folder (and I also attached zip):

zip -r tfjs-node_lib.zip  node_modules/@tensorflow/tfjs-node/lib/*            
  adding: node_modules/@tensorflow/tfjs-node/lib/napi-v4/ (stored 0%)
  adding: node_modules/@tensorflow/tfjs-node/lib/napi-v4/tfjs_binding.node (deflated 68%)

tfjs-node_deps.zip
tfjs-node_lib.zip

[mprinc]

This is the full error message:

node dist/index.js

[tfjs-test] started
internal/modules/cjs/loader.js:979
  throw err;
  ^

Error: Cannot find module '/Users/mprinc/data/development/ML/tfjs-node-test/node_modules/@tensorflow/tfjs-node/lib/napi-v5/tfjs_binding.node'
Require stack:
- /Users/mprinc/data/development/ML/tfjs-node-test/node_modules/@tensorflow/tfjs-node/dist/index.js
- /Users/mprinc/data/development/ML/tfjs-node-test/dist/index.js
    at Function.Module._resolveFilename (internal/modules/cjs/loader.js:976:15)
    at Function.Module._load (internal/modules/cjs/loader.js:859:27)
    at Module.require (internal/modules/cjs/loader.js:1036:19)
    at require (internal/modules/cjs/helpers.js:72:18)
    at Object.<anonymous> (/Users/mprinc/data/development/ML/tfjs-node-test/node_modules/@tensorflow/tfjs-node/dist/index.js:46:16)
    at Module._compile (internal/modules/cjs/loader.js:1147:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1167:10)
    at Module.load (internal/modules/cjs/loader.js:996:32)
    at Function.Module._load (internal/modules/cjs/loader.js:896:14)
    at Module.require (internal/modules/cjs/loader.js:1036:19) {
  code: 'MODULE_NOT_FOUND',
  requireStack: [
    '/Users/mprinc/data/development/ML/tfjs-node-test/node_modules/@tensorflow/tfjs-node/dist/index.js',
    '/Users/mprinc/data/development/ML/tfjs-node-test/dist/index.js'
  ]
}

[mprinc]

I cannot reproduce the error 100% anymore.

What I experienced now is that the error gets with upgrading/downgrading node (a month ago, with older tfjs versions it didn't matter).

Older node versions build napi-v4 and a newer version builds napi-v5

[kangyizhang]

@mprinc Yes you are correct. The node addon module (tfjs_binding.node) is compiled to a folder whose name include the N-API version. So if you installed the library in node.js with N-API version 4, the node addon module is stored as node_modules/@tensorflow/tfjs-node/lib/napi-v4/tfjs_binding.node. Then if you upgrade node.js to a version with N-API version 5, during execution the library will try to find the addon module at node_modules/@tensorflow/tfjs-node/lib/napi-v5/tfjs_binding.node, which does not exist. So if you changed node.js version, you might need rebuild the library by running npm rebuild @tensorflow/tfjs-node.

Please feel free to re-open if you have any other issue.

[mprinc]

Thanks @kangyizhang . I hope for the best :)