Upgrading zeebe-node breaks library completely

[nikku]

Describe the issue

Recent versions of zeebe-node don't handshake anymore (camunda-community-hub/zeebe-client-node-js#290). In fact they simply terminate (no exit code, no error).

Steps to reproduce

Works with zeebe-node@8.0.1 (frozen deps)

Ensure secure communication works in both zeebe-node and zbctl:

git clone git@github.com:barmac/zeebe-tls-connection-test.git
npm ci
npm generate-certs
docker-compose --env-file .env.insecure up
npm run test:insecure

You'll see the topology printed by both zeebe-node and zbctl.

Fails with upgraded zeebe-node

Now upgrade zeebe-node to latest:

npm install zeebe-node@latest

And execute the same connection test again:

ZEEBE_ADDRESS=test.test.localhost:26500 npm run test:secure

You'll see the topology printed by zbctl. zeebe-node simply terminates. No error, no non-zero exit code.

---

Broker logs

zeebe_broker                                 | 2023-01-25 17:34:32.759 [] [grpc-default-worker-ELG-5-3] WARN                                                                                                           
zeebe_broker                                 |       io.grpc.netty.NettyServerHandler - Stream Error                                                                                                                   
zeebe_broker                                 | io.netty.handler.codec.http2.Http2Exception$StreamException: Stream closed before write could take place                                    
zeebe_broker                                 |  at io.netty.handler.codec.http2.Http2Exception.streamError(Http2Exception.java:173) ~[netty-codec-http2-4.1.82.Final.jar:4.1.82.Final]

zeebe-node logs

(Before unconditional termination).

...
D 2023-01-25T17:34:32.741Z | channel | (1) dns:test.test.localhost:26500 createRetryingCall [1] method="/gateway_protocol.Gateway/Topology"
D 2023-01-25T17:34:32.741Z | channel | (1) dns:test.test.localhost:26500 createLoadBalancingCall [2] method="/gateway_protocol.Gateway/Topology"
D 2023-01-25T17:34:32.741Z | channel | (1) dns:test.test.localhost:26500 callRefTimer.ref | configSelectionQueue.length=0 pickQueue.length=1
D 2023-01-25T17:34:32.751Z | subchannel | (2) 127.0.0.1:26500 CONNECTING -> READY
D 2023-01-25T17:34:32.751Z | channel | (1) dns:test.test.localhost:26500 callRefTimer.unref | configSelectionQueue.length=0 pickQueue.length=0

[nikku]

In fact upgrading @grpc dependencies break the client completely.

Existing behavior can be restored to use the @grpc dependencies used in this repository.

[nikku]

Fixed upstream camunda-community-hub/zeebe-client-node-js#290 (comment) 🎉 .

[barmac]

OK then we can close this.

[nikku]

Can we? I'd love to follow up with some QA on our end to verify this does not happen again 😄 .

[barmac]

Perhaps we could test on multiple versions of zeebe-node? Still the problem was with a transient dependency.

[nikku]

I've added a test pipeline + an actual test: 83c5df5.

We could hook this up with dependabot or the like to continuously upgrade zeebe-node and hence keep it tested.

[nikku]

Considering this done.