Skip to content

Technical Architecture

Jump to bottom
Rylan Meilutis edited this page Apr 21, 2026 · 8 revisions

Architecture

This page explains how the core Rust library is organized, how telemetry moves through it, and why the data structures look the way they do. It assumes no prior knowledge of the codebase.

Goals and constraints

  • Schema-first: a shared schema defines endpoints, message types, and payload layouts for every language binding.
  • Compact on-the-wire representation: a small header, endpoint bitmaps, and optional compression minimize bandwidth.
  • Embedded-friendly: no_std support, bounded queues, and stack/heap tradeoffs.
  • Multi-language: C and Python bindings are generated from the same schema.
  • Deterministic behavior: validation and dedupe rules are consistent across languages.

Module map (what lives where)

src/config.rs (source): compile-time configuration values plus generated DataType/DataEndpoint enums.

src/lib.rs (source): schema metadata (MessageMeta, MessageElement, MessageDataType, MessageClass).

src/telemetry_packet.rs (source): Packet validation, formatting, and packet IDs.

src/small_payload.rs (source): inline-optimized payload storage (SmallPayload).

src/serialize.rs (source): compact wire format, ULEB128 helpers, envelope peek, packet IDs from wire.

src/router.rs (source): router core, queues, endpoint handlers, side-based routing.

src/relay.rs (source): schema-agnostic fanout relay between sides.

src/queue.rs (source): bounded deque used by router and relay.

src/c_api.rs (source) and src/python_api.rs (source): FFI bindings (C ABI and pyo3).

Schema and metadata pipeline

The schema is defined in telemetry_config.json (source) ( plus built-in TelemetryError endpoint/type). At build time:

build.rs (source) reads the schema to generate C headers and Python .pyi stubs.

  1. define_telemetry_schema! in src/config.rs (source) expands into Rust enums and metadata tables.

Core schema types:

  • DataEndpoint: enum generated from the endpoints list in telemetry_config.json (source) plus built-ins.
  • DataType: enum generated from the types list in telemetry_config.json (source) plus built-ins.
  • MessageMeta: per-DataType metadata (name, element layout, allowed endpoints).
  • MessageElement: Static(count, MessageDataType, MessageClass) or Dynamic(MessageDataType, MessageClass).
  • MessageDataType: primitive element type (Float32, UInt16, String, Binary, etc.).
  • MessageClass: logical class (Data, Warning, Error).

Why these structures exist:

  • MessageMeta is static so it can be used in const fn lookups (message_meta, get_data_type).
  • MessageElement separates shape from data type so the same primitive type can be used for static and dynamic payloads.
  • MessageClass is tied to element layout so formatting and error generation know the intent of the message.

Packet (the core runtime type)

Packet is the validated, shareable container used by the router and FFI layers. It contains:

  • ty: DataType (logical message type).
  • data_size: usize (cached payload length; must match payload.len()).
  • sender: Arc<str> (logical sender identifier).
  • endpoints: Arc<[DataEndpoint]> (destinations; must be non-empty).
  • timestamp: u64 (ms; treated as uptime below 1e12, or epoch ms otherwise).
  • payload: StandardSmallPayload (inline optimized; see below).

Validation rules enforced by Packet::new and Packet::validate:

  • Endpoints list must be non-empty.
  • For static layouts, payload length must be exactly count * data_type_size.
  • For dynamic layouts:
    • Numeric/bool types: payload length must be a multiple of element width.
    • String types: trailing NULs are ignored and remaining bytes must be valid UTF-8.
    • Binary types: no UTF-8 requirement (raw bytes).

Packet IDs are not serialized. Packet::packet_id hashes:

  • sender bytes
  • message name (DataType as string)
  • endpoint names (in order)
  • timestamp + data_size (little-endian)
  • payload bytes

This produces the same ID across different links, which enables de-duplication.

SmallPayload and inline storage

SmallPayload<INLINE> stores short payloads directly on the stack and spills to Arc<[u8]> when larger. The inline size is controlled by MAX_STACK_PAYLOAD via define_stack_payload!, and the default inline capacity is 64 bytes.

Why this matters:

  • Small telemetry values avoid heap allocation and Arc traffic.
  • Larger payloads still have cheap clone semantics because they are Arc-backed.

Serialization and wire format

The compact v2 wire format is implemented in src/serialize.rs (source). A packet is encoded as:

[FLAGS: u8]
    bit0: payload compressed
    bit1: sender compressed
[NEP: u8]                      // number of endpoints (bits set)
VARINT(ty: u32 as u64)          // ULEB128
VARINT(data_size: u64)          // logical (uncompressed) payload size
VARINT(timestamp_ms: u64)
VARINT(sender_len: u64)         // logical sender length
[VARINT(sender_wire_len: u64)]  // only if sender compressed
ENDPOINTS_BITMAP               // 1 bit per DataEndpoint discriminant
SENDER BYTES                   // raw or compressed
PAYLOAD BYTES                  // raw or compressed

Design choices:

  • ULEB128 varints minimize size for small values.
  • Endpoint bitmap avoids repeated endpoint IDs; size is based on MAX_VALUE_DATA_ENDPOINT.
  • Sender/payload compression uses zstd (zstd-safe) and is applied only when it makes the payload smaller.

packet_id_from_wire parses only as much as needed to compute the same packet ID as Packet::packet_id. It always hashes decompressed sender/payload bytes, so dedupe works across compressed and uncompressed links.

TelemetryEnvelope is a header-only view that can be obtained by peek_envelope without copying the payload.

Queues and bounded memory

Router and relay queue storage is built on BoundedDeque<T>:

  • Byte-budgeted: each item reports a byte_cost via the ByteCost trait.
  • Hard element cap: capacity never exceeds max_elems (computed from size_of::<T>()).
  • Eviction policy: pop from the front until new item fits in byte budget.
  • Growth policy: multiplicative growth controlled by QUEUE_GROW_STEP.

The public MAX_QUEUE_BUDGET is shared dynamically across router/relay RX queues, TX queues, recent-ID caches, reliable replay/out-of-order buffers, and discovery topology state. Recent-ID caches preallocate their final storage at construction and reserve that amount from the shared budget immediately. This design keeps memory use bounded and avoids unbounded VecDeque growth in embedded targets while letting the active part of the system use available queue budget when other internal queues are quiet.

Router architecture

The router is the main API for logging, receiving, dispatching, and (optionally) relaying.

Key structures:

  • RouterConfig: holds local EndpointHandler definitions in an Arc<[EndpointHandler]>.
  • EndpointHandler: packet or serialized handler for a specific DataEndpoint.
  • RouterSideId: identifies a named side (UART/CAN/RADIO/etc.).
  • RouterItem: either Packet or serialized bytes.

Receive flow:

  1. RX bytes or packet are processed immediately or queued (rx_* vs rx_*_queue).
  2. Reliable side headers are handled first when the ingress side has reliable delivery enabled.
  3. Packet ID is computed. For serialized bytes, the router tries packet_id_from_wire and falls back to hashing raw bytes if needed.
  4. If the packet ID is already in the recent cache, it is dropped.
  5. Local endpoint handlers are invoked.
  6. The router forwards the packet according to the current route rules and discovery/path-selection state.

Forwarding is driven by:

  • the active side ingress/egress policy
  • route overrides and typed route overrides
  • discovery-learned endpoint reachability
  • link-local side scope
  • route-selection policy (Fanout, Weighted, Failover)

Transmit flow:

  • log* builds a new Packet from typed data, validates sizes, and serializes it.
  • tx* sends a pre-built Packet (validated) or serialized bytes.
  • Queue variants (*_queue) defer the work until process_*_queue.

Error handling:

  • Local handler failures are retried (MAX_HANDLER_RETRIES).
  • If a handler ultimately fails, the packet ID is removed from the dedupe cache so a resend can be processed later.
  • The router emits a TelemetryError packet for local handlers when possible.

Relay architecture

Relay is schema-agnostic and purely forwards packets between named sides (UART/CAN/RADIO/etc.).

  • Each side registers a TX handler for serialized bytes or full packets.
  • Each side can opt out of reliable sequencing/ACKs (useful for TCP links).
  • The relay maintains RX/TX queues and a recent-ID cache like the router.
  • Fanout clones the Arc for payload sharing; it does not decode unless needed for a handler.
  • Dedupe ignores the source side so duplicates from different links are dropped.

Thread safety and time

Routers and relays use an internal RouterMutex so that most APIs can take &self while still protecting shared state. The clock source is injected via the Clock trait so embedded builds can supply their own timing source.

End-to-end lifecycle (summary)

Producer                      Router/Relay                         Consumer
--------                      -------------                         --------
log()/tx()  -> validate -> serialize -> tx(bytes, link) -> transport -> rx_serialized()
rx(bytes)   -> deserialize -> dedupe -> handlers -> (optional) relay forward

If you need build-time or schema details, see Build-and-Configure and Technical-Telemetry-Schema.

Clone this wiki locally