Flow History Sync

Part of the Plugin data protocol. Packet: Packet-mumcord-history.

Flow: history sync

Mumble servers keep no chat history. This flow lets a Mumcord client that joins (or follows) a channel fetch the channel messages it missed from other Mumcord clients. It is the only Mumcord flow that moves more data than fits one plugin data message, so most of this page is about chunking: how lists and message bodies are cut into packets of at most 1000 bytes and put back together.

The code is crates/engine/src/history_sync/ (wire.rs for packets and chunking, core.rs for the state machine, hash.rs for hashes).

Numbers used on this page

Name	Value	Where
PLUGIN_DATA_MAX	1000 bytes per plugin data message	protocol/src/client.rs
BODY_BUDGET	PLUGIN_DATA_MAX - 100 = 900 body bytes per packet; the other 100 bytes are reserved for the JSON header	wire.rs
HASH_LEN	8 bytes per message hash	wire.rs
HASHES_PER_CHUNK	BODY_BUDGET / HASH_LEN = 900 / 8 = 112 hashes per hashes packet	wire.rs
MAX_MESSAGES	500 messages per request	wire.rs
MAX_BODY	8 KiB (8192 bytes) per message body	core.rs
MAX_STREAM	256 KiB (262144 bytes) of message stream per answer	core.rs
MAX_AGE_MS	7 days	core.rs
MAX_HOLDERS	3 holders asked per sync	core.rs
DISCOVER_MS	1500 ms to collect avail answers	core.rs
LIST_TIMEOUT_MS	15000 ms for a hash list	core.rs
FETCH_IDLE_MS	30000 ms of silence while fetching	core.rs
SERVE_TTL_MS	60000 ms a holder keeps a prepared answer	core.rs
SEND_RATE, SEND_BURST	3 packets per second, burst 10	core.rs

Overview

sequenceDiagram
    participant R as Requester
    participant S as Mumble server
    participant H as Holder (up to 3)
    R->>S: areq (r, c, from, to), to everyone
    S-->>H: relayed
    H->>S: avail (r, c, n, old, new), only if it has messages
    S-->>R: relayed, collected for 1.5 s (DISCOVER_MS)
    R->>S: hreq (own r per holder, c, from, to), to the 3 largest holders
    S-->>H: relayed
    H->>S: hashes i=0 ... i=k, last=true (112 hashes per packet)
    S-->>R: relayed, in order
    R->>R: drop hashes it already has, assign the rest to holders
    R->>S: get (r) + bitmap of wanted hashes
    S-->>H: relayed
    H->>S: part i=0 ... i=m, last=true (900 body bytes per packet)
    S-->>R: relayed, in order
    R->>R: join the slices, parse, verify hashes, sanitize, store
    Note over R,H: deny at any step if the holder refuses

Loading

Step by step

1. Who has messages? (areq / avail)

• A sync runs for one channel at a time; further channels wait in a queue. It reaches back MAX_AGE_MS (7 days) from now, up to the moment the requester started receiving that channel live (the cutoff).
• The requester picks a request id r and sends areq with the channel c and the range from, to to everyone on the server.
• Each Mumcord client that shares history, saw the requester arrive in that channel, and has messages in the (clamped) range answers the requester with avail: n messages, oldest old, newest new. Holders with nothing stay silent; holders that may not answer also stay silent (an areq goes to everyone, so there is no deny here).
• The requester collects the answers for 1.5 seconds (DISCOVER_MS), keeps the first avail per holder, sorts by n (largest first, lower session number breaks ties) and keeps the 3 largest (MAX_HOLDERS). With no holder the sync ends quietly.

2. The hash list, chunked (hreq / hashes)

The requester does not ask for messages blindly. It first asks each chosen holder for the hashes of its messages, so it can skip what it already has.

• The requester sends each holder its own hreq with a new request id r (one r per holder), the channel and the range.
• The holder clamps the range, loads up to 500 messages (MAX_MESSAGES), leaves out messages with a body over 8 KiB (MAX_BODY), computes one 8-byte hash per message (oldest first), and keeps the messages in memory as a prepared answer for this requester (at most 3 prepared answers at once, MAX_SERVES; a new hreq from the same person replaces the old one). If it cannot answer it sends deny.
• The hash list is cut into chunks of HASHES_PER_CHUNK = 112 hashes: chunk i holds hashes i * 112 to i * 112 + 111. Each chunk is one hashes packet whose body is the hashes concatenated (112 x 8 = 896 bytes). The final chunk has "last":true. An empty list is one empty final chunk (i 0).
• 500 hashes therefore make 5 packets: 112 + 112 + 112 + 112 + 52 hashes. A full chunk is 942 bytes: a 45-byte header line, the newline and 896 bytes of hashes.

{"v":1,"t":"hashes","r":8,"i":0,"last":false}\n<896 bytes = 112 hashes>
{"v":1,"t":"hashes","r":8,"i":1,"last":false}\n<896 bytes>
{"v":1,"t":"hashes","r":8,"i":2,"last":false}\n<896 bytes>
{"v":1,"t":"hashes","r":8,"i":3,"last":false}\n<896 bytes>
{"v":1,"t":"hashes","r":8,"i":4,"last":true}\n<416 bytes = 52 hashes>

The requester accepts chunks only in order: a chunk whose i is not the next expected index, or a list that grows beyond 500 hashes, makes it give up on that holder. Every accepted chunk restarts the 15-second timer (LIST_TIMEOUT_MS).

3. Choosing the missing hashes (get)

When no holder is still listing:

• The requester removes every hash it already has (it tests the second before and after, and the same sender and text within 5 seconds, because clocks differ).
• It assigns each remaining hash to exactly one holder. It walks the lists from the newest end, interleaved, so the cap of 500 messages keeps the newest messages. Among the holders that have a hash, it picks the one with the fewest assignments so far, which spreads the work evenly.
• To each holder with something assigned it sends get with the holder's own r. The body is a bitmap over that holder's hash list: bit k (bit k % 8 of byte k / 8, least significant bit first) is set when the message at list position k is wanted. A list of 500 hashes needs 500 / 8 rounded up = 63 bytes, so a get always fits one packet. Holders with nothing assigned are finished at once.

Example: the holder listed 10 hashes and the requester wants the 1st, 2nd and 10th. Bits 0, 1 and 9 are set: byte 0 = 0b00000011 = 0x03, byte 1 = 0b00000010 = 0x02. Body: 03 02.

4. Message bodies, chunked (part)

• The holder looks up the get (same requester and r). A get for a prepared answer that is gone (older than 60 seconds, SERVE_TTL_MS, or already used) is ignored. A prepared answer is used up by the first get.
• It builds one WireMessage per set bit: ts, n, h (optional), b. While adding messages it adds up their encoded sizes and stops at 256 KiB (MAX_STREAM), so a very large answer is cut short.
• All wanted messages are encoded as one JSON array: the message stream. This stream is cut into slices of BODY_BUDGET = 900 bytes. Slice i is the body of the part packet with "i":i; the last slice has "last":true. Slices are raw byte ranges and may cut the JSON anywhere, even inside a word or a multi-byte character. Only the joined stream is valid JSON.
• A message body is at most 8 KiB, so a single message can span up to 10 packets; many small messages share one packet.

5. Reassembly and verification

For each part packet the requester checks:

1. It comes from the holder with this r and the requester is waiting for a get answer from it.
2. i equals the next expected index (starting at 0). Otherwise the holder is given up on.
3. The stream so far plus this slice stays within 256 KiB. Otherwise the holder is given up on.

Each accepted slice is appended and restarts the 30-second idle timer (FETCH_IDLE_MS). When the packet with "last":true arrives, the stream is complete. Then:

• The stream is parsed as a JSON array of messages. If it does not parse, nothing is stored.
• At most as many messages as were asked for are read.
• For each message: ts must lie in the requested range, the body must be at most 8 KiB, and h (if present) must be 1 to 128 hex digits.
• The requester recomputes the hash of each message from the received ts, n, h and b (same recipe as the holder, see Packet-mumcord-history). The message is accepted only if that hash is one it asked this holder for, and only once per hash. Messages with an unrequested hash, a repeated hash, or one it already has are dropped.
• The name is cleaned, the body is sanitized again as HTML, and the message is stored and shown, marked as synced from that holder.

6. Holders drop out

A holder is given up on ("dropped") when it sends deny, sends a chunk out of order or over a limit, or stays silent past its timer. There is no retransmission of single chunks. Instead:

• A partial stream is thrown away; only complete streams are stored. Everything the holder still owed becomes "orphans".
• The orphans are fetched from another holder that listed them. That holder's earlier answer was used up by its first get, so the requester sends it a fresh hreq, reads the new hash list and sends a new get for the orphans only.
• Orphans that no remaining holder has are given up on; the sync ends with what it got.
• The sync ends when every holder is done or dropped and no orphan can be fetched.

Timeouts

Timer	Value	Starts / restarts	When it fires
DISCOVER_MS	1.5 s	on sending areq	stop collecting avail, ask the largest holders
LIST_TIMEOUT_MS	15 s	on sending hreq, restarted by every accepted hashes chunk	the holder is dropped
FETCH_IDLE_MS	30 s	on sending get, restarted by every accepted part chunk	the holder is dropped, its stream is discarded, its hashes become orphans
SERVE_TTL_MS	60 s	on the holder side, when it prepared the answer	the holder forgets the prepared answer; a later get is ignored

The timers are checked every 100 ms. A lost final chunk therefore costs 30 seconds; a lost chunk in the middle is noticed as soon as the next chunk arrives out of order.

Worked example: a 2,500-byte message stream

A holder is asked for one message. Its WireMessage JSON, inside the array brackets, adds up to a 2,500-byte message stream (a body of roughly 2.4 KB of text, for example a long pasted text). With BODY_BUDGET = 900:

Packet	i	last	Stream bytes	Body	Header line	On the wire
1	0	false	0 to 899	900	43 + newline	944 bytes
2	1	false	900 to 1799	900	43 + newline	944 bytes
3	2	true	1800 to 2499	700	42 + newline	743 bytes

2,500 / 900 = 2.78, rounded up: 3 packets. The packets, with request id r = 7:

{"v":1,"t":"part","r":7,"i":0,"last":false}\n<stream bytes 0..899>
{"v":1,"t":"part","r":7,"i":1,"last":false}\n<stream bytes 900..1799>
{"v":1,"t":"part","r":7,"i":2,"last":true}\n<stream bytes 1800..2499>

The header line is 43 bytes with false, 42 with true; counted for one-digit r and i. Larger ids make it a few bytes longer, which is why 100 bytes are reserved.

sequenceDiagram
    participant H as Holder
    participant S as Mumble server
    participant R as Requester
    Note over H: stream = JSON array, 2,500 bytes
    H->>S: part i=0, last=false, 900 bytes (0..899)
    S-->>R: relayed
    Note over R: next=1, buffer 900 bytes, idle timer restarts (30 s)
    H->>S: part i=1, last=false, 900 bytes (900..1799)
    S-->>R: relayed
    Note over R: next=2, buffer 1,800 bytes
    H->>S: part i=2, last=true, 700 bytes (1800..2499)
    S-->>R: relayed
    Note over R: buffer 2,500 bytes, parse JSON array, recompute hash, sanitize, store

Loading

Pacing: the holder sends at most 3 packets per second with a burst of 10, below the server's limit of about 4 per second with a burst of 15. These 3 packets leave in one burst. The largest possible answer, 256 KiB = 262,144 bytes, is 292 packets (262,144 / 900 rounded up), which takes about 94 seconds after the first burst of 10. The idle timer restarts on every chunk, so a steady transfer does not time out. A holder serving several people sends round robin, one packet per receiver in turn, so one long answer does not block the others.

Errors and edge cases

Situation	Result
No holder answers areq	Sync ends, nothing is added.
Holder sends deny (sharing off, range outside the time the requester was live, request budget, 3 answers already open)	That holder is dropped; its share goes to another holder if one has it.
Chunk out of order, lost in the middle, or stream above 256 KiB	Holder dropped, partial stream discarded, hashes become orphans.
Last chunk lost	Dropped after 30 s of silence.
get arrives after 60 s, or twice	Ignored by the holder; the requester drops it after its timer.
Stream is not valid JSON	Nothing is stored from it.
Message hash does not match a requested hash	That message is dropped.
Message body above 8 KiB	Never listed by the holder; dropped by the requester if sent anyway.
Holder's answer would exceed 256 KiB	The holder stops adding messages; the missing ones are not synced in this run.
Many requests from one person	The holder allows a burst of 6, refilling 1 per 5 seconds (0.2 per second); more are ignored.
Packet not for an open own request	Ignored.