Skip to content

Latest commit

 

History

History
59 lines (53 loc) · 4.37 KB

NETWORK_FORMATS.md

File metadata and controls

59 lines (53 loc) · 4.37 KB
Raw

Network formats

All network encoding is big-endian.

Consumer poll response

+--------------------------------------------------------------------------------+
| items length (uint16)                                                          |
+--------------------------------------------------------------------------------+
| response items                                                                 |
| +----------------------------------------------------------------------------+ |
| | item                                                                       | |
| | +------------------------+---------------------+------------------------+  | |
| | | token (int64)          | range index (uint8) | gen version (uint32)   |  | |
| | +------------------------+---------------------+------------------------+  | |
| | | topic length (uint8)   | topic name (bytes)  | offset (int64)         |  | |
| | +------------------------+---------------------+------------------------+  | |
| | | payload length (int32) | compressed payload (zstd crc) (bytes)        |  | |
| | +------------------------+----------------------------------------------+  | |
| +----------------------------------------------------------------------------+ |
|                                                                                |
| +----------------------------------------------------------------------------+ |
| | item                                                                       | |
| +----------------------------------------------------------------------------+ |
|                                                                                |
| .                                                                              |
| .                                                                              |
| .                                                                              |
|                                                                                |
+--------------------------------------------------------------------------------+

Producer framed request

A series of frames of bytes with a common partition key (can be empty).

+----------------+--------------+--------------------+---------------+----------------------+-------------------+
| version (byte) | flags (byte) | stream id (uint16) | opcode (byte) | body length (uint32) | head crc (uint32) |
+----------------+--------------+--------------------+---------------+----------------------+-------------------+
| body                                                                                                          |
| +-------------------------------+------------------------------+--------------------------------------------+ |
| | optional timestamp μs (int64) | partition key length (uint8) | partition key (bytes)                      | |
| +-------------------------------+------------------------------+--------------------------------------------+ |
| | topic length (uint8)          | topic name (bytes)                                                        | |
| +-------------------------------+------------------------------+--------------------------------------------+ |
| | message 0 length (uint32) | message 0 (bytes)                                                             | |
| +-----------------------------------------------------------------------------------------------------------+ |
| | message 1 length (uint32) | message 1 (bytes)                                                             | |
| +-----------------------------------------------------------------------------------------------------------+ |
| |                                                                                                           | |
| | .                                                                                                         | |
| | .                                                                                                         | |
| | .                                                                                                         | |
| +-----------------------------------------------------------------------------------------------------------+ |
|                                                                                                               |
+---------------------------------------------------------------------------------------------------------------+