MacOS: JS dependency issue

[willfreeborn]

Unable to run Reor on install due an upstream issue with vectordb: failed to load native library

14in MBP, Apple M2 Pro

[samlhuillier]

Interesting. Thanks for raising this - I noticed it only in Windows but now it seems to pop up on Mac. I have the following issue open with lancedb: lancedb/lancedb#939
cc @chebbyChefNEQ @changhiskhan

[natecox]

Having the same issue here on a MacBook M1 Pro

[changhiskhan]

we just released a new version that should fix this issue (v0.4.10)
https://www.npmjs.com/package/vectordb

Would you mind giving the new version a shot?

[changhiskhan]

The cause was that DataFusion introduced a new dynamically linked dependency so when we upgraded to the latest DataFusion version, this issue started happening (liblzma). Will made the fix to statically link the lib.

[samlhuillier]

Thank you @changhiskhan. I've just pushed a release for the new version:

• this fixes the issue on MacOS
• but not on the fresh-install Windows 11 machine I just tested on

[changhiskhan]

What do you see on windows?

[samlhuillier]

This error:

Error: vectordb: failed to load native library.
  You may need to run `npm install @lancedb/vectordb-win32-x64-msvc`.

  If that does not work, please file a bug report at https://github.com/lancedb/lancedb/issues

  Source error: Error: The specified module could not be found.
\\?\C:\Users\samto\Desktop\reor\node_modules\@lancedb\vectordb-win32-x64-msvc\index.node
    at Object.<anonymous> (C:\Users\samto\Desktop\reor\node_modules\vectordb\native.js:26:11)
    at Module._compile (node:internal/modules/cjs/loader:1271:14)
    at Module._extensions..js (node:internal/modules/cjs/loader:1326:10)
    at Module.load (node:internal/modules/cjs/loader:1126:32)
    at Module._load (node:internal/modules/cjs/loader:967:12)
    at c._load (node:electron/js2c/node_init:2:13672)
    at Module.require (node:internal/modules/cjs/loader:1150:19)
    at require (node:internal/modules/cjs/helpers:119:18)
    at Object.<anonymous> (C:\Users\samto\Desktop\reor\node_modules\vectordb\dist\query.js:19:25)
    at Module._compile (node:internal/modules/cjs/loader:1271:14)

Tried the install recommended in the error and the same error occurs.

[willfreeborn]

Seems to be fixed, thanks @samlhuillier - do you want me to leave issue open?

[samlhuillier]

Great! Yes let's keep the issue open just to track it for Windows

[wjones127]

For Windows, here are the shared library dependencies:

dumpbin /dependents C:\Users\willj\test\node_modules\@lancedb\vectordb-win32-x64-msvc\index.node
Microsoft (R) COFF/PE Dumper Version 14.29.30152.0
Copyright (C) Microsoft Corporation.  All rights reserved.


Dump of file C:\Users\willj\test\node_modules\@lancedb\vectordb-win32-x64-msvc\index.node

File Type: DLL

  Image has the following dependencies:

    kernel32.dll
    bcrypt.dll
    advapi32.dll
    ws2_32.dll
    ntdll.dll
    shell32.dll
    ole32.dll
    crypt32.dll
    VCRUNTIME140.dll
    api-ms-win-crt-math-l1-1-0.dll
    api-ms-win-crt-string-l1-1-0.dll
    api-ms-win-crt-heap-l1-1-0.dll
    api-ms-win-crt-runtime-l1-1-0.dll

I'm not certain, but I'm guessing that VCRUNTIME140.dll is not present for users who have installed Visual Studio C++. You may need to provide the Visual C++ Redistributable along with your application so that users don't have to install Visual Studio C++. It looks like you use Electron Builder, and perhaps that framework can help do this.

[wjones127]

I think we may be able to fix this in LanceDB by statically linking the C runtime for Windows: lancedb/lancedb#979

[samlhuillier]

A temporary solution to this on windows is installing: https://aka.ms/vs/17/release/vc_redist.x64.exe