Skip to content

System Architecture

Jump to bottom
Varun Karthic edited this page Oct 10, 2026 · 2 revisions

System Architecture

How imPress is put together, why it is split the way it is, and how data moves through it. Reference documents: docs/architecture/.

Contents

Why the system is split this way

Each design choice answers a constraint of real classrooms:

Constraint Design answer
A room has dozens of student devices. School Wi-Fi is often unreliable, limits the number of clients, or needs credentials per device. Student modules never use Wi-Fi. They talk over ESP-NOW: connectionless 802.11 frames with no access point and no association.
ESP-NOW has limited range and no routing. Modules relay each other's messages (up to 5 hops), forming a flooding mesh. See Protocols and Mesh.
A radio can't stay on the mesh channel and stay associated with an access point on another channel at the same time. Two boards per room. The hub (ESP32-S3) stays on the mesh channel full time. The gateway (ESP32-C6) does Wi-Fi. They are wired together with SPI.
Everyone answers within a few seconds. SPI exchanges fixed 4096-byte slots that carry up to 136 answers each. The gateway posts answers to the backend in batches (up to 20 messages or every 500 ms).
The backend must know which room a device is in. Each gateway is linked to exactly one class, by its classroom code or automatically.
Devices in the field must be updatable without a cable. Hubs and gateways have two firmware slots and app rollback. The backend hosts images and runs staged deployments.

Components

flowchart LR
    subgraph Room["Classroom"]
        direction LR
        S1["Student module<br/>ESP32"]
        S2["Student module<br/>ESP32"]
        S3n["Student module<br/>ESP32"]
        HUB["Hub — ESP32-S3<br/>mesh root · SPI master"]
        GW["Gateway — ESP32-C6<br/>SPI slave · Wi-Fi"]
        S1 <-- "ESP-NOW<br/>broadcast" --> HUB
        S2 <-- "ESP-NOW" --> S1
        S3n <-- "ESP-NOW (relayed)" --> S2
        HUB <-- "SPI, 4 KB slots<br/>+ 2 ready lines" --> GW
    end
    subgraph Server["Server"]
        API["FastAPI backend<br/>REST + WebSocket"]
        DB[("SQLite<br/>impress.db")]
        FW[("Firmware store<br/>firmware_bins/")]
        API --- DB
        API --- FW
    end
    GW <-- "Wi-Fi: HTTP JSON +<br/>device WebSocket" --> API
    T["Teacher / admin<br/>browser"] <-- "HTTPS + WSS" --> API
Loading
Component Code Runs on Responsibilities
Student module firmware/student ESP32 Provisioning of the enrollment number; buttons and optional display; joining the mesh; sending answers, votes and heartbeats; relaying for others
Hub firmware/class_s3 ESP32-S3 Mesh root; routing table; de-duplication; batching student messages into SPI slots; broadcasting commands to the mesh; sweeping silent students; its own OTA through a Wi-Fi hop
Gateway firmware/class_c6 ESP32-C6 SPI slave; turning frames into JSON batches; device WebSocket for commands; heartbeats with diagnostics; its own OTA with health check and rollback
Shared protocol firmware/protocol all boards The only implementation of every wire format: frames, SPI slots and records, mesh de-duplication, the small JSON reader, TLS settings
Backend backend/app Python 3.12+ REST API, WebSocket rooms, sessions and roles, SQLite, presence and health, firmware registry and deployment engine, the web app
Web app backend/app/static, templates browser The admin and teacher interface, served at /
React UI frontend/ browser Optional development dashboard (Vite); not feature-complete

Deployment topology

flowchart TB
    subgraph Campus
        subgraph R1["Room 201"]
            st1["~60 student modules"] --> h1["S3 hub"] --> g1["C6 gateway"]
        end
        subgraph R2["Room 305"]
            st2["student modules"] --> h2["S3 hub"] --> g2["C6 gateway"]
        end
        AP["2.4 GHz Wi-Fi (WPA2 or better)"]
        g1 --> AP
        g2 --> AP
    end
    AP --> RP["TLS reverse proxy (nginx/Caddy)<br/>or uvicorn with TLS"] --> BE["uvicorn: app.main:app<br/>ONE worker"] --> DB[("impress.db")]
    Browser["Teachers and admins"] --> RP
Loading
  • One hub and one gateway per room; any number of student modules (up to 300 per room).
  • One backend for the campus, as a single process: WebSocket rooms live in its memory. See Limits and scaling.

Identities

Thing Identity Source
Student 10-character enrollment number provisioned on the module; students.roll_number in the backend
Student module 32-bit device ID from the MAC (routing and inventory only) firmware/student/main/main.c
Gateway / hub MAC address; type c6 / s3 esp_devices.mac_address
User username + session token users, user_sessions
Class numeric id; join code; optional classroom_code class_sessions
Firmware image SHA-256 of the file; (target, version) firmware_artifacts

How a question reaches every module

sequenceDiagram
    autonumber
    participant T as Teacher
    participant API as Backend
    participant C6 as Gateway (C6)
    participant S3 as Hub (S3)
    participant ST as Student modules
    T->>API: POST /api/quizzes/{id}/start
    API->>API: quiz → active, current question 0, commit
    API-->>C6: WebSocket {"event":"quiz_question", "quiz_id", "question_order", "question_text", "options", "time_limit_s", …}
    API-->>T: the same frame (the dashboard shows it) + HTTP 200
    C6->>C6: ws_command_to_frame → MSG_QUIZ_QUESTION (options clamped to 4)
    C6->>S3: queued in the SPI FIFO, delivered within one 50 ms poll
    S3->>S3: mesh_master_broadcast (TTL 5, sender 0)
    S3--)ST: ESP-NOW broadcast
    ST--)ST: modules relay (TTL 4 … 1), each copy handled once
    ST->>ST: display shows the question, state QUIZ_ACTIVE
Loading
  • Next question: POST /next, or the server's quiz timer for a per-question quiz, repeats the flow with the next question number.
  • End: POST /stop, next after the last question, or a total time limit running out sends quiz_end; modules go back to idle.
  • Polls: the same path, with poll_start and poll_end.

Text longer than a module can show (139 bytes of question, 14 bytes per option) is accepted with a warning and cut off on the device.

How an answer travels

sequenceDiagram
    autonumber
    participant ST as Student
    participant RL as Other modules (relays)
    participant S3 as Hub
    participant C6 as Gateway
    participant API as Backend
    participant T as Teacher
    ST->>ST: button B → MSG_QUIZ_ANSWER {quiz_id, question, option 1, enrollment}
    ST--)S3: ESP-NOW (direct, TTL 5)
    ST--)RL: the same broadcast
    RL--)S3: relayed copies (TTL 4, 3, …)
    S3->>S3: CRC ok → routing table updated → de-duplication:<br/>first copy queued, the rest dropped
    S3->>C6: SPI slot: record [length][sender id][frame]
    C6->>C6: decode → JSON {"type":"quiz_answer", …} → batch
    C6->>API: POST /api/device/batch (X-API-Key: device key)
    API->>API: known student? quiz active? question exists? option valid? not answered yet?
    API->>API: INSERT quiz_answers · commit
    API-->>T: WebSocket {"type":"quiz_answer", "total_answers": n}
    API-->>C6: {"status":"ok", "processed": 1, "skipped": 0}
Loading
  • What gets stored: an answer is stored only if every check passes. Otherwise it is counted in skipped. The reasons are an unknown student, an inactive quiz, an unknown question, an option out of range, or a duplicate.
  • Duplicates are stopped three times: on the modules (de-duplication), on the hub (de-duplication) and in the database (unique indexes).
  • Delivery is reliable: if the backend answers with a 5xx error, the gateway keeps the batch and sends it again. The retry stores each answer once.

How a module joins

sequenceDiagram
    autonumber
    participant ST as Student module
    participant S3 as Hub
    participant C6 as Gateway
    participant API as Backend
    loop every 2 s until connected (give up after 30 s)
        ST--)S3: MSG_STUDENT_JOIN {enrollment, device_id}
    end
    S3->>S3: routing table: find or add (≤ 300)
    S3--)ST: root heartbeat (TTL 5), sent for every join, including retries
    ST->>ST: heard the root → connected, learns its hop count → idle
    S3->>C6: first join copy over SPI
    C6->>API: batch {"type":"student_join", "enrollment_number", "device_id", …}
    API->>API: inventory row (connected) · activity log student.connect
    Note over S3: no traffic for 60 s → sweep → MSG_STUDENT_LEAVE → inventory "seen"
Loading

How presence works

sequenceDiagram
    autonumber
    participant C6 as Gateway
    participant API as Backend
    participant SW as Presence sweep (every 15 s)
    participant T as Teacher dashboards
    C6->>API: register · heartbeat (15 s) · ping (120 s) · any WebSocket message
    API->>API: last_seen = now · online (logged only on offline → online)
    API--)T: presence snapshot (teachers of that class)
    SW->>API: devices silent > 30 s → offline (logged once)
    SW->>API: student modules silent > 90 s → "seen"
    API--)T: presence snapshot
Loading

A snapshot contains the class's gateway, the devices it relays, and its student modules. The class page shows it under Live Devices and refreshes it every 5 s, and immediately on every push.

How a gateway finds its class

Situation Result
The gateway registers with a classroom_code that matches a class's code or classroom_code That class's gateway becomes this device
No code, but the device is already linked It keeps its class
No code, type c6, not linked yet The first active class without a gateway (lowest id)
Any other device type without a code (a hub during its OTA hop, a student) Never linked
The backend assigns a class over the WebSocket The gateway saves the new class id in NVS and reconnects its WebSocket

The backend inside

flowchart TB
    subgraph Entry["main.py"]
        L["lifespan: secure-config check · signing key check ·<br/>migrations · first admin · background tasks"]
        MW["middleware: session-activity refresh · CORS"]
    end
    subgraph Routers["routers/"]
        RA["auth"]
        RAD["admin"]
        RC["classes (+ /api/devices)"]
        RCO["courses"]
        RS["students"]
        RQ["quizzes"]
        RP["polls"]
        RD["device"]
        RF["firmware"]
        RDP["deployments"]
    end
    WS["ws/handler.py · ws/manager.py"]
    subgraph Services["services/"]
        PRES["presence"]
        HEALTH["health"]
        INV["student_modules"]
        DEP["deployments (engine)"]
        FWI["firmware_image · firmware_signing · firmware_store"]
        CSV["csv_import · question_import · class_import"]
        REC["records"]
        SESS["sessions"]
    end
    subgraph Core
        AUTH["auth.py (sessions, roles)"]
        DAUTH["device_auth.py (device keys)"]
        CFG["config.py"]
        DBM["database.py · migrations.py"]
        MOD["models.py · schemas.py"]
    end
    Entry --> Routers --> Services
    Routers --> AUTH
    RD --> DAUTH
    WS --> DAUTH
    Services --> MOD --> DBM
Loading
Piece Job
main.py Builds the app. At startup it refuses default secrets (unless DEBUG), checks the signing key, runs migrations, seeds the first admin, and starts the background loops. Serves /health, /api/health and the web app.
auth.py bcrypt passwords; opaque session tokens stored as SHA-256; role checks (require_admin, require_teacher_or_admin, …).
device_auth.py The shared provisioning key and per-device keys: who is calling, and whether they may act for a given MAC.
migrations.py Numbered, transactional schema changes with automatic backups (currently 8 steps).
services/deployments.py The OTA deployment engine: per-device state machine, stages, retries, timeouts, a 5 s scheduler loop.
services/presence.py, health.py, student_modules.py Online/offline, health states, the module inventory.
services/firmware_*.py Image parsing, signature verification, the content-addressed store.
services/*_import.py CSV parsing and validation for questions, courses and sections.
services/records.py Deletes that keep the database consistent (class delete, student erase).
ws/ One WebSocket room per class; teacher and device roles.

Background loops:

Loop Interval What it does
Presence sweep 15 s Marks devices offline after 30 s of silence and student modules seen after 90 s, then pushes snapshots
Deployment scheduler 5 s Starts stages, applies timeouts, schedules retries, completes or pauses deployments
Session cleanup 5 min Deletes expired and revoked sessions

Request rules worth knowing:

  1. Session activity: it is refreshed once per request, in middleware. Status endpoints never refresh it, so a polling page can't keep an idle session alive.
  2. Commits: writes commit explicitly. Activity-log rows commit together with the change they describe.
  3. Broadcasts: they happen after the commit, so a client that re-reads the API sees the new state.

The firmware inside

Board Main tasks
Gateway (C6) SPI service every 2 ms (a reliable slot exchange, even while Wi-Fi is down); spi2http turns frames into JSON and posts batches, keeping them on 5xx errors and bounding the buffer; heartbeat every 15 s with diagnostics, re-registering if its key was reset; status_ping every 120 s; WebSocket events for commands and OTA prompts; the OTA task with its health watchdog; a supervision loop for Wi-Fi and WebSocket reconnects
Hub (S3) The ESP-NOW receive callback (routing table, de-duplication, queueing); spi_link, which flushes the mesh queue into SPI slots and dispatches gateway commands to the mesh; heartbeat every 5 s (sweeps students silent for 60 s and sends its own heartbeat to the gateway); the s3_ota task for the Wi-Fi hop
Student The ESP-NOW callback only copies packets into a queue; mesh_rx de-duplicates, delivers root messages and relays others after a random 10–60 ms delay; input_task debounces buttons; heartbeat every 30 s; serial provisioning while unprovisioned

The student module's states: INIT → PROVISIONING (no ID) → CONNECTING → IDLE ⇄ QUIZ_ACTIVE / POLL_ACTIVE → ANSWERED. Holding CONFIRM + C for 2 s at boot clears its identity.

Rules every firmware change follows (each guarded by a test):

  • no blocking work in the radio receive callback;
  • anything handed to a driver by pointer outlives the operation;
  • one task owns each cJSON tree;
  • never destroy the WebSocket client from its own handler;
  • every wire format is encoded and decoded by one shared function.

Limits and scaling

Aspect Today Implication
Backend processes WebSocket rooms live in one process Run one uvicorn worker; several workers would need a pub/sub layer such as Redis
Database SQLite, one writer at a time Fine for a campus: each room posts about two batches a second. Heavier loads would need PostgreSQL and ported migrations
Students per room 300 (hub routing table and gateway student set) Raising it needs a firmware change on both boards
Mesh airtime about N² short frames per question when all N modules hear each other Comfortable at 30 modules; measure before relying on 300 in one burst
Student firmware updates over USB only Planned work: a Wi-Fi hop or a chunked ESP-NOW transfer

Related pages: Protocols and Mesh · Database and Migrations · Hardware Guide.

imPress v2.1 · Repository · Issues · Source docs: docs/ · Not yet verified on hardware: see Testing and CI

imPress

Understand

Set up

Use

Operate

Develop

More

Clone this wiki locally