Wrapped -> non-wrapped dependencies not available at runtime

[mattnathan]

I'm trying out your node rules for our project and have hit what appears to be a missing use case surrounding combinations of wrapped and unwrapped packages.

My specific issue is with an app that depends on both grpc and @grpc/proto-loader.

grpc@^1.19.0 depends on protobufjs@^5.0.3 and @grpc/proto-loader@^0.4.0 depends on protobufjs@^6.8.6. When running yarn install manually these conflicts are correctly handled by making protobufjs@^5.0.3 a top level node_modules package and wrapping protobufjs@^6.8.6 inside @grpc/proto-loader/node_modules. The dependencies of protobufjs@^6.8.6 (and 5.0.3) get placed in the root node_modules.

This all results in a directory structure like this:

/package.json
/node_modules/grpc/... @1.19.0 // has node_modules but no protobufjs in there
/node_modules/protobufjs/... @5.0.3 // no node_modules
/node_modules/@grpc/proto-loader/... @0.4.0
/node_modules/@grpc/proto-loader/node_modules/protobufjs/... @6.8.6 // no node_modules
/node_modules/... // both protobufjs deps for 5.0.3 and 6.8.6

It appears that the dependency installation function isn't processing non-wrapped dependencies of wrapped dependencies.

I've created a repo that shows the issue: https://github.com/vanti-public/issue-happy-wrapped

[mterring]

Thanks for the detailed report. We generally use the sealed option (seal=True) to install our external dependencies from yarn (it's faster and doesn't suffer this issue), would that work for your use case?

[mattnathan]

I'll give it a go.

Just curious, what's the difference between sealed and not-sealed. I can't find anything in the README about it

[mattnathan]

I just tried seal = True on the yarn_repository rule. It failed for me with

no such package '@yarn_deps//__AT__grpc__SLASH__proto-loader': BUILD file not found on package path and referenced by '//bad-deps:bad-deps-lib'

Changing the dependencies to use @yarn_deps//:node_modules fixes that issue so I was curious and removed the seal option and it continued to work. So I guess the seal part didn't really make a difference.

[mterring]

That's unexpected. With seal = True using @yarn_deps//:node_modules uses the entire node_modules folder as created by yarn (this can have issues with nested .bin entries), whereas without seal it's just a grouping of all the individual, top-level modules, so should be equivalent to the individual dependencies.

I suspect that using node_modules is adding the non-wrapped dependencies to work around the original issue you encountered. Will leave this bug open, but it's not a priority given the workaround available.