GRPC Example? (IDFGH-1026)

[eagi223]

This isn't an issue, but more of a feature request I suppose. This has been asked in the forum, but the thread hasn't had a reply since February...

here

So it looks like the ESP Voice Assistant SDK uses Google Dialogflow through the gRPC hooks. Is there any way we could get a generic example of using gRPC in the IDF? Or maybe someone out there has some code they'd like to share where they've implemented gRPC on the ESP32?

[dhrishi]

Hi @eagi223 We will add this to the to-do list. Although, for now, I do not have concrete timelines for that.

[eagi223]

That would be great! In the meantime I'll keep trying to figure it out.

[kroggen]

I'm interested in the gRPC example on ESP32 too.

I checked the esp-va-sdk repo but was not able to find anything. The code appears to be inside of the libdialogflow.a library.

By the way, there are people interested in this topic since 2016:

https://twitter.com/proppy/status/800767976244854784

[kroggen]

Could you inform if you are using Juniper/grpc-c or just sending the raw protocol buffer packets via HTTP2 ?

Are you using sh2lib for that?

[SERIDJ]

hi, @eagi223 @dhrishi @kroggen someone have news about grpc in ESP device . ?

[sh0umik]

i am interested too .. any news or solutions ?

[kroggen]

I did it manually encoding and decoding the protocol-buffers packets using nanopb and sending them using sh2lib.

I cannot share the code now but you can check these tutorials:

https://techtutorialsx.com/2018/10/19/esp32-esp8266-arduino-protocol-buffers/

https://techtutorialsx.com/2019/01/04/esp32-http-2-post-request-with-headers/

[SERIDJ]

@kroggen , Thanks very much for your quick reply, i have run encoding code in esp32 using first tutorial , but i dont know more about deconding data from my server (visual studio c#).

and what u thing about PJON-GRPC : https://github.com/Halytskyi/PJON-gRPC ??

[kroggen]

This PJON is new to me. I never used it.

But if you have control over the server, why not just use a simple JSON or Binn and simple HTTP?

I had to use GRPC simply because the server already exists and the only option was to comply with it.

Protocol-buffers in C sucks!

[SERIDJ]

@kroggen i dont know if i can use it and i want to HTTPS i hope that not a problem ?

[99bit]

@kroggen

Is there any updates about gRPC ??!

[kroggen]

Hi,

I don't know any easy solution to use GRPC on ESP32, but I can share what I did.

I separated the process in 2 parts:

1. Encoding and decoding the messages using nanopb (check the examples above)
2. Calling the service function via HTTP2 using sh2lib and sending the encoded message as payload (and decoding the received message)

Just the protobuf encoded message is not sufficient as payload, we need to add 5 bytes before to comply with the HTTP2 spec. The first byte must be zero (meaning no compression) and the remaining 4 bytes are the size of the message in big endian.

I just released the project that uses this method. You can check the sources here

You can start by the send_grpc_request function

[Hiabuto-de]

Hm. Any Updates on this ? - We wan't to start a bigger Project with Ethernet, gRPC and CAN on ESP32, but gRPC is neccesary for this :-/

[tomatac]

This is a very interesting idea! Is anybody working on this feature? It would be great.

[adammruk]

Same - grpc example would be great

[Davide-Di-Do]

Hi everyone, I'm currently working on an IoT system where the device and the backend communicate with GRPC protocol. Take a look here with minimal working code. https://github.com/iot-tetracube-red/iot-grp-tech-demo

[DatanoiseTV]

@Davide-Di-Do Seems like your repo is no longer existent. Could you give me some insight?

[JohnGalt1717]

I'd love to see this too.

[chrisomatic]

Hi,

I don't know any easy solution to use GRPC on ESP32, but I can share what I did.

I separated the process in 2 parts:

1. Encoding and decoding the messages using nanopb (check the examples above)
2. Calling the service function via HTTP2 using sh2lib and sending the encoded message as payload (and decoding the received message)

Just the protobuf encoded message is not sufficient as payload, we need to add 5 bytes before to comply with the HTTP2 spec. The first byte must be zero (meaning no compression) and the remaining 4 bytes are the size of the message in big endian.

I just released the project that uses this method. You can check the sources here

You can start by the send_grpc_request function

@kroggen Thanks for sharing your code and suggestions here. It helped me quite a bit. And it seems like you're right about the prepended 5 bytes. My example code is working now. But looking at the HTTP2 spec, I can't find any mention of these bytes.

https://datatracker.ietf.org/doc/html/rfc7540

The DATA frame just has optional padding bytes, and the frame format is quite different. I figured NGHTTP2 handles most of this formatting for me, but clearly the first 5 bytes are required. Maybe I'm missing something, am I looking at the wrong spec?

Thanks,
Chris

[kroggen]

@chrisomatic
It has been a long time, I do not remember where I got that information. Probably read on some source-code or analyzed data packets

[chrisomatic]

@chrisomatic It has been a long time, I do not remember where I got that information. Probably read on some source-code or analyzed data packets

Thanks for your reply! I just found the reason for this so Ill document it here.

The 5 bytes (no compression + length) are a gRPC Message header for the HTTP/2 DATA frames. More details here: https://github.com/grpc/grpc/blob/master/doc/PROTOCOL-HTTP2.md