-
Notifications
You must be signed in to change notification settings - Fork 640
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Failed to parse server response when I try to read a stream #1022
Comments
FYI, adding |
That error message essentially indicates that the incoming binary data could not be parsed as a valid message of the expected response message type. Are you sure that the |
@murgatroid99 I am, yes. And FWIW, the other implementations of the same What other angles could we look at this from? |
The next useful step is to add more logging. Those errors are handled here, here, and here, and logging the original error messages will give more information (only one of those will be triggering this specific issue, but I'm not sure which one). I can add some logging in an upcoming release, and in the meantime you could add logging in your local copy of the library to get the information faster. |
@murgatroid99 So the error is from here. When I stringify
Dump of the buffer direct to a file — output.txt Screenshot of the file opened in SublimeText —
The Go client can |
@murgatroid99 There's more weird stuff happening. When I
this is the
You'll notice that the |
@murgatroid99 By printing the deserialize error trace, I was able to trace the deserialization down to the Full stack trace —
Relevant line from generated Relevant proto definition of I'm not sure where to go from here. Any help would be appreciated. Some observations
|
I was finally able to trace it to the root cause! wipes sweat from forehead The problem is when the @murgatroid99 Is this a bug, or is this something that needs to be fixed/reconciled on the server-side? I can gladly provide more information/repros for this. |
OK, it looks like there are multiple issues here. The fact that you are seeing different messages in your subscribe responses than you are publishing looks like a server bug, or at least unexpected server behavior. Since the Go client is seeing it too, it's very unlikely to be a client bug. The failure to process a map entry with an absent value looks like a protobuf bug. You should file an issue with the official protobuf repository. We just republish protoc. You should also go there with questions about the generated |
@murgatroid99 Thanks for the pointers, I've further narrowed down the issue to a missing argument in the generated If you're familiar with the |
Sorry, I'm not that familiar with the |
protobufjs/protobuf.js#1348 should fix this. |
This response helped me. I was using the BloomRPC tool as a UI to send gRPC requests to my server. I updated my protos but didn't update the client Proto in the BloomRPC app. Updated and my requests are working again. |
Problem description
I'm using generated static bindings from https://github.com/liftbridge-io/liftbridge-grpc/blob/0877a0cacb7a721e2e19280cff126dc2ed51912d/api.proto to create a Liftbridge Node.js client that will create a stream and publish some messages and subscribe to them (Liftbridge is a kind of lightweight Kafka), and I run into an
Error: 13 INTERNAL: Failed to parse server response
error consistently whenever I try to consume theSubscribe
stream.Reproduction steps
docker run -p 4222:4222 -ti nats:latest --debug --trace
in a window.go get github.com/liftbridge-io/go-liftbridge
and then$GOPATH/bin/liftbridge --raft-bootstrap-seed --nats-servers nats://localhost:4222 --level debug
in another window.https://github.com/paambaati/node-liftbridge.git
in yet another window.yarn install
ornpm install
yarn run debug
ornpm run debug
to run the debug script that reproduces this issue.The debug script attempts to create a new stream, then publish a few messages, then subscribes to the same stream (subject) and then publishes a few more messages.
Expected output — Each published message should be printed to console (see relevant lines).
Actual output —
Environment
fnm
Additional context
The static bindings were generated by a script - see relevant command here. The bindings are checked-in at
grpc/generated
.The Go version of the client I'm developing (see go-liftbridge) can in fact
Subscribe
to the messagesPublish
ed by my Node.js debug script and print all theMessage
s correctly! I used the example subscribe program at https://github.com/liftbridge-io/go-liftbridge/blob/master/example/lift-sub/subscribe.go to verify this 😮I ran a full verbose trace of my debug script, and here's the output — https://gist.github.com/paambaati/7884b119eee47fafa436f74db8b59edc. I've padded the debug script's stdout with a lot of new lines so it is a little easier to separate them from the GRPC traces.
The text was updated successfully, but these errors were encountered: