Unable to execute in postinstall script in yarn monorepo

[chrbala]

When I try to install node-lmdb on linux, I get the following error:

Command: npm run my-command
Arguments: 
Directory: /test-node-lmdb/my-project/node_modules/my-package
Output:
> my-package@0.0.1 my-command /test-node-lmdb/my-project/packages/my-package
> node ./my-command

/test-node-lmdb/my-project/node_modules/bindings/bindings.js:96
  throw err
  ^

Error: Could not locate the bindings file. Tried:
 → /test-node-lmdb/my-project/node_modules/node-lmdb/build/node-lmdb.node
 → /test-node-lmdb/my-project/node_modules/node-lmdb/build/Debug/node-lmdb.node
 → /test-node-lmdb/my-project/node_modules/node-lmdb/build/Release/node-lmdb.node
 → /test-node-lmdb/my-project/node_modules/node-lmdb/out/Debug/node-lmdb.node
 → /test-node-lmdb/my-project/node_modules/node-lmdb/Debug/node-lmdb.node
 → /test-node-lmdb/my-project/node_modules/node-lmdb/out/Release/node-lmdb.node
 → /test-node-lmdb/my-project/node_modules/node-lmdb/Release/node-lmdb.node
 → /test-node-lmdb/my-project/node_modules/node-lmdb/build/default/node-lmdb.node
 → /test-node-lmdb/my-project/node_modules/node-lmdb/compiled/6.12.0/linux/x64/node-lmdb.node
    at bindings (/test-node-lmdb/my-project/node_modules/bindings/bindings.js:93:9)
    at Object.<anonymous> (/test-node-lmdb/my-project/node_modules/node-lmdb/index.js:3:37)

But when I run find . -name "node-lmdb.node", I get.

/test-node-lmdb/my-project/node_modules/node-lmdb/build/Release/node-lmdb.node
/test-node-lmdb/my-project/node_modules/node-lmdb/build/Release/obj.target/node-lmdb.node

I'm happy to provide a docker file or share the container if it's useful.

[Venemo]

I don't have any experience with Docker unfortunately. Is this a Docker-specific issue or can you reproduce this outside of Docker?

[chrbala]

I doubt it's a docker-specific issue. More accurately, the issue could be described as Can not install on Debian GNU/Linux 8 and Docker should be more of an implementation detail. It may or may not be an issue with other distributions – I haven't tested on others.

My initial guess is that the issue has something to do with yarn workspaces on linux interfering with the bindings package or the integration somehow. I'm using a workspace, and that can sometimes cause issues with file paths. Although, it would be a bit surprising if that's the case since bindings is ostensibly checking the correct absolute path.

If you have a way of testing this on Linux and can put some time into doing so, I can try and throw together a repo that can minimally reproduce the issue. I might also find out more about the issue as I do so.

[Venemo]

I use Fedora 27, and it works well here. If you can tell me how to reproduce on Debian, I'm willing to put together a VM to test.

[chrbala]

1. The issue turns out to be platform independent. My issue is with fresh installs on any platform.
2. The issue only occurs with Yarn workspaces. I've occasionally had issues with paths when using workspaces. These can sometimes be resolved with using require('path').resolve(SOME_PATH), especially when using a path relative to a different module.
3. It seems that a race condition is present when saving the file in question. The issue is in loading the module, not on installation. I am only unable to load the module immediately after installation as part of other installation scripts. If I do a vanilla install, then run the scripts after the installation completes, it works fine.

I've edited the issue title to reflect these points and have created a minimal repro case. To test this, you'll need to install yarn and run yarn install in the root package directory.

[Venemo]

Thank you @chrbala

I will take a look at it but unfortunately I don't have experience with Yarn either. As far as I understand npm install should also compile the module (on all platforms) by calling node-gyp which should place the compiled files into a location that is accessible by the index.js. So it sounds like this is either a node or npm bug (or a Yarn bug) that it doesn't find the compiled file under certain circumstances.

Does Yarn work correctly with other native modules? If yes, we should take a look at how their index.js differs from ours.

[emkman]

I am having the same issue on Ubuntu 16.04 (Debian based though not sure it matters) and I get different behavior across different cloud hosts. On Digital Ocean the issue occurs if I build the project as root, but not as a newly created user. On AWS, I get the issue with the non root user using the same script that works on Digital Ocean. The project in question is: https://github.com/nimiq-network/core and it does indeed use Yarn.

[emkman]

I cd'ed to /core/node_modules/node-lmdb/build/Release and confirmed that node-lmdb.node was missing. I then went to /core/node_modules/node-lmdb/ and ran npm install. This kicked off a node-gyp rebuild and fixed the issue.

[Venemo]

@emkman Can you give the exact steps to reproduce your problem? And what environment you ran?

[Venemo]

I played around with yarn a bit but could not figure out what is wrong. Reported an issue against yarn here: yarnpkg/yarn#5814

[Venemo]

@chrbala @emkman If that's OK with you guys, I would suggest to close this issue and continue the discussion at the yarn bug report. Currently it looks like this isn't something I can fix in node-lmdb.

[emkman]

Sure, thank you for your effort tracking it down.

[chrbala]

Yeah, feel free. It looks like it affects all of the native modules on yarn monorepos. I changed my installation workflow to make this work.

…

On Mon, May 21, 2018 at 09:54 Ethan Kravitz ***@***.***> wrote: Sure, thank you for your effort tracking it down. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#126 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AGhKdEMFrKcBc6aCf1WxVQa6Xierpzrdks5t0uNMgaJpZM4TfN8W> .

[Venemo]

@chrbala Can you share with us what the workaround was, in that yarn bugreport?