Protocols and Mesh

Protocols and Mesh

Every byte on every link, and how the ESP-NOW mesh works. Everything here is implemented once, in firmware/protocol, and covered by host tests, so the two ends of a link can't drift apart. References: docs/design/protocols.md, docs/design/mesh.md.

Contents

• The five layers
• Protocol frame
• Message types
• Mesh header
• The mesh
• SPI slot
• SPI batch record
• JSON to the backend
• Timing summary
• Changing a protocol

---

The five layers

flowchart LR
    ST["Student"] -- "① mesh header + ② protocol frame<br/>ESP-NOW" --> S3["Hub"]
    S3 -- "③ SPI slot carrying ④ batch records" --> C6["Gateway"]
    C6 -- "③ SPI slot carrying one ② frame" --> S3
    C6 -- "⑤ JSON over HTTP(S) / WebSocket" --> BE["Backend"]

Loading

Byte order: payload struct fields are little-endian (the packed C structs as the ESP32 lays them out). Header length and CRC fields are big-endian.

Protocol frame

Used on every hop:

 0      1      2      3      4 … 4+N-1        4+N    5+N
┌──────┬──────┬──────┬──────┬───────────────┬──────┬──────┐
│ 0xAA │ TYPE │ LEN (BE, 2) │ PAYLOAD (N)   │ CRC16 (BE,2) │
└──────┴──────┴──────┴──────┴───────────────┴──────┴──────┘

Field	Size	Notes
start	1	always 0xAA
type	1	below
length	2, big-endian	payload bytes, at most 240
payload	N	a packed struct
CRC	2, big-endian	CRC-16/CCITT-FALSE (polynomial 0x1021, initial 0xFFFF, no reflection) over TYPE + LEN + PAYLOAD

The largest frame is 246 bytes. The largest frame actually sent over the mesh is a quiz question (214 bytes + the 6-byte mesh header = 220), inside ESP-NOW's 250-byte limit.

Worked example (from the real encoder):

MSG_ACK, empty payload:  AA 30 00 00 09 39

MSG_POLL_VOTE {poll_id=3, option=1, "ABCDE12345", device_id=0xD53FC8B1}:
AA 22 00 12 | 03 00 | 01 | 41 42 43 44 45 31 32 33 34 35 00 | B1 C8 3F D5 | 68 9D
 hdr  len=18  poll_id  opt   enrollment (11 bytes, NUL-ended)    device_id LE   CRC

API: msg_encode(type, payload, len, out, size) returns the bytes written or -1. msg_decode(buf, len, &msg) returns the bytes consumed, 0 (incomplete) or -1. msg_verify_crc(&msg) checks the CRC.

Message types

Code	Name	Direction	Payload	Size
0x01	HEARTBEAT	student → hub; hub → students; hub → gateway	device_id u32 · battery_pct u8 · rssi i8 · uptime_s u32	10
0x02	STUDENT_JOIN	student → hub	enrollment char[11] · device_id u32 · student_id u16 · name char[32]	49
0x03	STUDENT_LEAVE	hub → gateway (sweep)	enrollment char[11] · device_id u32 · reason u8 (0 timeout, 1 leave, 2 shutdown)	16
0x04	ATTENDANCE	–	11-byte enrollment	11
0x11	QUIZ_QUESTION	backend → students	quiz_id u16 · question_num u8 · num_options u8 · time_limit_s u32 · question_text char[140] · options char[4][15]	208
0x12	QUIZ_ANSWER	student → backend	quiz_id u16 · question_num u8 · selected_option u8 · response_time_ms u32 · enrollment char[11] · device_id u32	23
0x13	QUIZ_END	backend → students	id u16	2
0x20	POLL_START	backend → students	poll_id u16 · num_options u8 · title char[64]	67
0x21	POLL_OPTIONS	legacy alias of POLL_START
0x22	POLL_VOTE	student → backend	poll_id u16 · selected_option u8 · enrollment char[11] · device_id u32	18
0x23	POLL_END	backend → students	id u16	2
0x30 / 0x31	ACK / NACK	diagnostics	–	0
0x40–0x42	SPI_AGGREGATE / SPI_COMMAND / SPI_STATUS	gateway ↔ hub internal		≤ 240
0x50	OTA_PROMPT	gateway → hub	version char[32] · token char[64]	96
0x51	OTA_APPLIED	hub → gateway	version char[32] · mac char[13] · result u8 (0 applied, 1 rolled_back, 2 failed) · error i32	50

Strings are NUL-terminated and zero-padded. Writers always copy with strncpy(dst, src, sizeof dst - 1) into a zeroed struct.

What a module can show: a question carries 139 characters of text and 14 per option. That is why the backend warns about longer text, and why quizzes have at most 4 options: the modules have four buttons.

Mesh header

ESP-NOW frames start with this header, before the protocol frame:

┌──────┬────────────────────┬──────┬─────────── protocol frame ───────────┐
│ TTL  │ SENDER_ID (u32 LE) │ HOPS │ AA TYPE LEN … CRC                      │
└──────┴────────────────────┴──────┴────────────────────────────────────────┘

Field	Meaning
ttl	5 at the origin; decremented on every relay; relayed only while greater than 1
sender_id	the origin: 0 for the hub; students use a MAC-derived ID (routing and diagnostics only)
hops	0 at the origin; +1 on every relay

The mesh

Why flooding

Option	Why not
A star (everyone talks to the hub directly)	range: modules at the back of a hall can't reach it
ESP-WIFI-MESH (a tree with parent selection)	needs Wi-Fi association per node, a heavier stack, a slower join
Routing tables on students	state that must stay right while students move or switch off
Controlled flooding (chosen)	no state on students; robust to churn; bounded by TTL. The cost is redundant transmissions, which de-duplication caps

Topology

flowchart LR
    R(("Hub<br/>sender_id 0"))
    A["A (1 hop)"]
    B["B (1 hop)"]
    C["C (2 hops)"]
    D["D (3 hops)"]
    R <--> A
    R <--> B
    A <--> C
    B <--> C
    C <--> D

Loading

• Downstream (hub → everyone): questions, poll starts and ends, join replies. A student acts on frames whose origin is the hub (sender_id == 0), whether received directly or through relays.
• Upstream (student → hub): joins, answers, votes, heartbeats. Students forward them, and the hub consumes them.
• Addressing: everyone broadcasts to FF:FF:FF:FF:FF:FF, through an explicitly registered broadcast peer.

What a student does with each packet

flowchart TB
    P["packet from the queue"] --> O{"sender_id == mine?"}
    O -- yes --> X["drop"]
    O -- no --> D{"seen this message<br/>in the last 10 s?"}
    D -- yes --> X
    D -- no --> RT{"from the hub<br/>(sender_id 0)?"}
    RT -- yes --> L["first time: connected; learn the hop count"]
    L --> DL["decode + CRC → show it, change state"]
    RT -- no --> TT
    DL --> TT{"ttl > 1?"}
    TT -- no --> E["done"]
    TT -- yes --> W["wait 10–60 ms (random)"] --> S["re-broadcast with ttl−1, hops+1"]

Loading

• De-duplicate first: a message is identified by an FNV-1a hash of its origin sender_id and the frame. That is the same for every relayed copy, and it never includes the header, whose TTL and hops change at each hop. So each node handles and relays each message once, however many neighbours echo it.
• Random delay: the 10–60 ms delay before relaying desynchronises neighbours, to reduce collisions.
• Never block the radio: the radio callback only copies into a 16-deep queue. When the queue is full, packets are dropped rather than stalling the radio.

What the hub does with each packet

1. Decodes it and checks the CRC. Invalid frames still refresh the routing table (the node is alive) but are never forwarded.
2. Updates the routing table: matched by enrollment number, else by sender ID, else added (up to 300). It records last seen, hops and signal strength.
3. De-duplicates: only the first copy is queued for the gateway.
4. Replies to every STUDENT_JOIN with a hub heartbeat (TTL 5), including retries: a student only retries when it missed the reply.
5. Every 5 s, sweeps students silent for 60 s and reports STUDENT_LEAVE to the gateway.

Cost

For one press, in a room where all N students hear each other (measured by the host test test_mesh_dedup.c):

Students	Before v2 (hash included TTL and hops)	Now
3	9 transmissions, 9 copies to the gateway	3 transmissions, 1 copy
12	45 / 45	12 / 1
21	81 / 81	21 / 1
30	117 / 117	30 / 1

Each message now costs N transmissions (the origin plus one relay per other node) and one delivery upstream. A whole class answering one question costs about N² short frames in the worst case. At 30 students that is about 900 frames spread over the answer window, comfortably within ESP-NOW's capacity.

The backend de-duplicates once more, by (quiz, question, student) and (poll, student), so even a copy that slips through under extreme load is never counted twice.

The de-duplication cache

Property	Value	Reason
Key	FNV-1a(origin sender_id ‖ frame)	identical across relay copies; different per student, question and option
Slots	64 (a ring; the oldest is evicted)	at least the number of distinct messages in flight in a burst
Window	10 s	relay copies arrive within milliseconds; a deliberate identical retry after 10 s counts as new
Clock	FreeRTOS ticks, with wrap-safe unsigned subtraction	tested across the 2³² wrap

Channel rules

• One channel: every node uses the same 2.4 GHz channel, MESH_WIFI_CHANNEL (student config.h) = CONFIG_MESH_WIFI_CHANNEL (hub). The default is 1.
• Hub startup: the hub loads its configuration before starting the mesh and logs Mesh master initialized, channel=N.
• OTA hop: joining the access point moves the hub's radio to the AP's channel. Afterwards it restores the mesh channel.

SPI slot

Between hub and gateway, in both directions: one standard full-duplex SPI transfer (mode 0; the hub is the master) of exactly 4096 bytes, at CONFIG_SPI_CLOCK_MHZ (default 10 MHz, about 3.3 ms per slot).

┌─────────────┬───────────────────────┬────────────────┬──────────────────┐
│ LEN (BE, 2) │ PAYLOAD (LEN ≤ 4092)  │ CRC16 (BE, 2)  │ zero padding      │
└─────────────┴───────────────────────┴────────────────┴──────────────────┘

• CRC-16 (the same function as frames) over the payload only. Both ends build and check slots with the shared spi_slot_encode() and spi_slot_decode(). A slot with a bad length or CRC is dropped and counted, and the link carries on with the next one.
• LEN = 0 (an all-zero slot) means "nothing to send".
• Hub → gateway payload: a sequence of batch records (below).
• Gateway → hub payload: exactly one protocol frame.

Ready lines (active high):

• READY S3→C6: the hub has something queued.
• READY C6→S3: the gateway has something queued.

Queues and timing:

• FIFOs: each side queues outgoing payloads in an 8-slot FIFO, so back-to-back frames arrive in order instead of overwriting each other.
• Hub: clocks one slot every 50 ms (SPI_POLL_INTERVAL_MS), and immediately when it has queued something. It removes a payload from its FIFO only after the transfer succeeded.
• Gateway: keeps exactly one transaction armed. Every 2 ms, whether its Wi-Fi is up or not, it collects the completed slot, loads the next FIFO entry and re-arms. A gateway frame therefore reaches the hub within one poll interval.

SPI batch record

Inside a hub → gateway slot:

┌──────────────────┬────────────────────┬──────────── FRAME ────────────┐
│ FRAME_LEN (BE,2) │ SENDER_ID (u32 LE) │ protocol frame (FRAME_LEN)    │
└──────────────────┴────────────────────┴───────────────────────────────┘

• FRAME_LEN counts the frame only, not the 4-byte sender ID. Getting this wrong once dropped every single-message batch (#2).
• One writer, one reader: records are written only by spi_record_write() and read only by spi_record_read(). The reader returns 0 for any truncated record and never reads past the end.
• Capacity: a 4092-byte payload holds 136 poll-vote records (30 bytes each).

Worked example (the vote above, from student 0xD53FC8B1):

00 18 | B1 C8 3F D5 | AA 22 00 12 03 00 01 41 42 43 44 45 31 32 33 34 35 00 B1 C8 3F D5 68 9D
len=24  sender (LE)   frame

JSON to the backend

Gateway → backend: the frame each record carries becomes one JSON message in a POST /api/device/batch:

Frame	JSON type	Fields
HEARTBEAT	heartbeat	device_mac (the gateway), device_id, battery_pct, rssi
STUDENT_JOIN	student_join	enrollment_number, device_mac, class_code (empty), device_id
STUDENT_LEAVE	student_leave	enrollment_number, device_mac, device_id, reason
QUIZ_ANSWER	quiz_answer	quiz_id, question_order, enrollment_number, selected_option, response_time_ms, device_id
POLL_VOTE	poll_vote	poll_id, enrollment_number, selected_option, device_id
OTA_APPLIED	ota_result	mac_address, version, result, error, device_mac

Backend → gateway: WebSocket events become frames (ws_command_to_frame):

event	Frame	Rules
quiz_question	QUIZ_QUESTION	needs quiz_id, question_text, options[]; at most 4 options; text truncated to 139, options to 14
poll_start	POLL_START	needs poll_id, options[]; title up to 63
quiz_end / poll_end	QUIZ_END / POLL_END	needs the id
device_command (ota_update)	– (the gateway's own OTA), or OTA_PROMPT (for its hub)	routed by device type and MAC

The keys the gateway reads are pinned in firmware/contract/device_ws_frames.json. The backend test regenerates that fixture from real server output, and the firmware host test decodes the same file.

Timing summary

Constant	Value	Where
Mesh TTL	5	MESH_RELAY_TTL
Relay delay	10–60 ms random	student mesh_espnow.c
Join retry	every 2 s, up to 30 s	student main.c
Student heartbeat	30 s	student HEARTBEAT_INTERVAL_MS
Hub sweep / student timeout	5 s / 60 s	hub Kconfig
De-duplication window	10 s	MESH_DEDUP_WINDOW_MS
SPI slot / clock / poll	4096 B / 10 MHz / 50 ms	protocol.h, hub Kconfig
Gateway SPI service	2 ms	gateway main.c
Gateway batch	20 messages or 500 ms idle	gateway Kconfig

Changing a protocol

1. Change the shared code in firmware/protocol/protocol.{h,c}, never a local copy.
2. Extend the host tests (firmware/protocol/test_host): golden bytes, round trips, truncation at every offset.
3. For JSON: keep fields additive. Old firmware ignores unknown response fields, but may send old field names.
4. For WebSocket commands: update ws_command.c, regenerate the fixture (IMPRESS_UPDATE_CONTRACT=1 pytest backend/tests/test_device_ws_contract.py), and commit both.
5. Flash every board that speaks the changed format together. A mixed room won't understand itself.