missing @grpc/grpc-js dependency

[JimPatterson]

Environment details

• OS: Alpine Linux 3.9
• Node.js version: 12.13.0
• npm version: pnpm 4.9.2 instead of npm
• @google-cloud/pubsub version: 1.4.1

Error seen

When trying to require pubsub, I get an error that @grpc/grpc-js can't found like:

Thrown:
Error: Cannot find module '@grpc/grpc-js'
Require stack:
- /beacon/pnpm-store/node_modules/.pnpm/registry.npmjs.org/@google-cloud/pubsub/1.4.1/node_modules/@google-cloud/pubsub/build/src/pubsub.js
- /beacon/pnpm-store/node_modules/.pnpm/registry.npmjs.org/@google-cloud/pubsub/1.4.1/node_modules/@google-cloud/pubsub/build/src/index.js
- /beacon/pnpm-store/node_lib/storage_mdd/src/sendChangeTasks.js

The error is being seen due to the non-flat node_modules layout that pnpm uses where npm's flat style would likely not have seen the problem. But the real cause of the problem is due to pubsub having a require of grpc-js without declaring it as a dependency. I see older issue talking about this:

• missing @grpc/grpc-js dependency #664 reported this as a problem and it was fixed then
• fix(deps): remove direct dependency on @grpc/grpc-js #773 re-broke this, apparently thinking there was no direct dependency

I'm not a Typescript user so I can't review the comment about only needing it for types, but https://github.com/googleapis/nodejs-pubsub/blob/master/src/publisher/publish-error.ts#L17 imports @grpc/grpc-js directly and the resultant javascript does also. This implies that there is a direct dependency on @grpc/grpc-js

[feywind]

Thanks @JimPatterson for the report! There are some other issues going on regarding grpc-js, so I'll probably look at this in that context.

[bcoe]

@feywind @JimPatterson 👋 google-gax which is depended on by @google-cloud/pubsub has @grpc/grpc-js as a dependency, so it's definitely in the dependency graph of a fresh installation.

This sounds like it's potentially an issue with pnpm, mind testing out with an npm install, and seeing if things do work for you as expected?

[feywind]

@bcoe I guess my quick triage of the question last week... the Pub/Sub code is directly require()'ing grpc-js. e.g.:

index.ts:export {ServiceError} from '@grpc/grpc-js';

It seems like that's making some shaky assumptions about the availability of the grpc-js module to the pub/sub module. I'm guessing we decided to pull it in through gax to manage versions there, but it doesn't seem technically correct to me.

On the other hand @JimPatterson as Ben was saying to me internally, I think we really only support using npm, so yarn and pnpm are a bit of a best effort kind of thing. I'm unsure we'll be changing this for non-npm use.

[sonu27]

I'm getting this issue too using npm on alpine.

Adding @grpc/grpc-js using npm i fixes the problem. But I use the grpc which is more complete and faster. I guess the pubsub code shouldn't reference the JS lib.

[feywind]

I think that post-2.x I'm planning to just add @grpc/grpc-js as a dependency for pubsub. Pulling in the constants from that seems okay, even for gRPC C++ usage, but it shouldn't be assuming it'll be there in the tree, in my opinion.

Incidentally, there have been a lot of updates to grpc-js lately. I'm also hoping to get us on the 1.x version of that after the pubsub 2.0 is released. So perhaps that will let you move on to the pure JS version for your use case. I've been told to try to encourage that, because gRPC C++ for Node will be moving into maintenance mode soon (no new features).

[feywind]

I forgot to update this - we talked it over and decided it's probably best if Pub/Sub itself doesn't gain a direct dependency on grpc-js, because the version could fall out of step with gax itself. And some typings there are needed in the Pub/Sub library. So I'm really not in love with this solution, but I think for now the plan is not to change this.

As a workaround for package manager issues outside stock npm, sonu27's comment above should fix it.