Skip to content

Design Decisions

Jump to bottom
James Isnor edited this page Jul 6, 2026 · 5 revisions

Formulatel Design

This page talks about the decisions made when designing formulatel, and a brief overview of the cloud deployment decisions.

---
config:
  theme: redux
  look: neo
title: formulatel
---
flowchart LR
  subgraph Sources [Sim Racing Games]
    T1[Title A]
    T2[Title B]
    TN[Title C]
  end
  Ingest[ingest<br/>Receive, Transform, Publish]
  subgraph PubSub [Pub/Sub Topics by Type]
    TypeA[Topic: Vehicle]
    TypeB[Topic: Motion]
    TypeN[Topic: Laps]
  end
  G[Grafana<br/>Visualization]
  T1 --> Ingest
  T2 --> Ingest
  TN --> Ingest
  Ingest --> TypeA
  Ingest --> TypeB
  Ingest --> TypeN
  Persist[persist<br/>Persistence Layer]
  DB[(Datastore)]
  Persist --> DB
  PubSub --> Persist
  PubSub <--Live Visualization--> G
  G <--Analysis--> DB
Loading

Overview

formulatel consists of three parts:

  1. ingest - the application responsible for receiving telemetry from a title, transforming it into the formulatel protobuf format, and queuing it for the backend.
  2. persist - the service responsible for writing telemetry from ingest into a persistent datastore.
  3. Visualization - the front-end component that displays data from the datastore and allows ad-hoc analysis.

An important design hiccup: formulatel is meant to be able to ingest, persist, and analyze any kind of vehicle telemetry (telemetry from multiple titles). It tries to support genericity by defining a model (protocol buffers and database tables) that can be used by multiple titles, and we've defined that model by looking at the data we get from a few different racing sims. It is by no means definitive nor general enough to support the lofty title-agnostic goal of formulatel

Implementation

Transport

formulatel uses MQTT to transport data from the driver's machine to persist. The motivating factor behind choosing MQTT was simply that our visualization layer, Grafana, requires such a stream for Grafana Live. That's also the reason we're using v3 instead of v5.

I decided on protocol buffers as the encoding format because it is:

  • flexible - support new titles and schema evolution of existing ones
  • efficient - variable-sized field encoding and a binary format make protobufs fast and scale up well

I imagined RPC streams being the "best" way for us to get data from ingest -> persist: we could enforce authentication and even setup oauth. The protobufs can also be compiled into many languages for drivers to write their own ingest for yet-to-be-supported titles. The main reason formulatel uses MQTT instead is due to the Grafana Live plugin landscape: there is already an MQTT plugin created for Grafana Live, and a live dashboard was the first major milestone after getting over some of the initial hurdles.

Future improvements

We could try to create a streaming data plugin for either our (yet-to-be created) RPC streams or for MQTT v5

A Short View Back to the Past

The formulatel service was originally a project to help me learn about OpenTelemetry, so the first design focused on streaming data from the title with the intention of then adding observability on those streams and their data.

block-beta
columns 3
tel>"telemetry data"]:3
space down1<[" "]>(down) space

block:ingest_servcice:3
    i1["ingest"]
    i2["ingest"]
    i3["ingest"]
end

space down2<[" "]>(down) space

block:rpc_service:3
    r1["rpc"]
    r2["rpc"]
    r3["rpc"]
end

space down3<[" "]>(down) space

block:otel_col:3
    otel1["otel-col"]
    otel2["otel-col"]
    otel3["otel-col"]
end

space down4<[" "]>(down) space

block:datastore:3
    influxdb
    prometheus
    opensearch
end

up1<[" "]>(up) up2<[" "]>(up) up3<[" "]>(up)

block:visualize:3
    grafana
end
Loading

I eventually realized that I didn't need an OpenTelemetry collector at all, and that I would need to use something like MQTT to make use of Grafana Live. That was a lot easier for me to wrap my head around, MQTT is very small and simple whereas OTcol is fairly complex, and it made writing the first title-implementation of ingest and persist easier to reason about.

Cloud Deployment

formulatel is deployed as a cloud service and can be reached at https://dashboard.formula.tel .

formulatel is deployed to the OCI cloud, making use of the generous Ampere free-tier (thus the ARM64 builds). The terraform directory contains the IaC used for this, as well as a setup-server.sh script that runs as part of the cloud-init when the VM is first provisioned, and a db-init.sql script that can be used to provision users and databases before the postgres instance is used by formulatel. Anybody should be able to deploy formulatel to their own cloud, completely for free.

The kubernetes directory contains the formulatel helm chart and a README.md with as much of the manual setup instructions as possible (e.g. creating secrets in kubernetes for the DB users we added with the db-init.sql script or installing cert-manager in a fresh cluster).

Infrastructure

The infrastructure for our free cloud deployment is predictably small and not at all production-like, using a single VM with 4 threads and 24GB memory to run kubernetes and postgres. All of formulatel is deployed in kubernetes via the helm chart in kubernetes/charts/formulatel.

It's (perhaps obviously) not ideal to run our public services - Grafana and an MQTT broker - on the same instance that we are running postgres. Doing so means that we have to open ports on the PG instance that we normally would not need to, and generally databases are provisioned in their own cloud-specific hardware and services so that we don't need to think about iptables, pg_hba.conf, or any other server-management specific DB config. This is something to improve in the future with a larger budget or a free-er datastore.

Clone this wiki locally