WebSocket client per-message compression (RFC7692) support (IDFGH-16641)

[huming2207]

Is your feature request related to a problem?

I'm thinking to implement a WebSocket client per-message compression support, as we need this feature for device provisioning service sooner or later.

I'm planning to support the full RFC7692.

Describe the solution you'd like.

In my opinion I think we should implement this in two PR:

• one for ESP-IDF tcp_transport/transport_ws.c, and let it to support stuff like:
  • HTTP upgrade's handshake header stuff, like adding Sec-WebSocket-Extensions: permessage-deflate etc.
  • RSV1 bit (the bit to indicate a WS frame is compressed or not)
• one for esp_websocket_client:
  • add a dependency to espressif/zlib
  • actually does the DEFLATE compression/decompression here
  • handles stuff like window bit setting and no_context_takeover stuff

This is because:

1. I know ESP32 has a miniz in the mask ROM. But it does not support dynamic window bit. It has to be 15. If there's a WebSocket server doesn't want the window bit to be 15 then the handshake will fail.
2. Because we can't use miniz due to the window bit issue above, we have to use zlib instead. But I don't want to let ESP-IDF itself to depend on espressif/zlib, so I'd rather letting the esp_websocket_client actually takes the dependency and does the compression.

Meanwhile people may be concerned about things like MQTT-over-WS on esp_mqtt library may need to implement this RFC7962 again which doubles the work. But I don't think it's an issue, at least not for MQTT. Because quite a few popular MQTT brokers themselves seem don't even support compression. (See https://forum.cedalo.com/t/plans-for-mosquitto-websockets-to-support-stream-compression/599/2 and mochi-mqtt/server#418)

Describe alternatives you've considered.

If you really want to let transport_ws to take over the responsibility to do the compression then maybe we can also expose a set of callback APIs of compress and decompress process.

Feel free to point out if I miss anything. I will prepare the implementation now and submit the PR later.

Additional context.

No response

[huming2207]

Hi @gabsuren

The first part of the PR to ESP-IDF is ready: espressif/esp-idf#17746

Could you please proceed to review? Thanks!

Regards,
Jackson

[huming2207]

Hi @gabsuren

The second part of PR to esp_websocket_client is here: #916

I haven't tested yet but you might take a look first if you are convenient.

Regards,
Jackson