Canvas

The Canvas API (Matrix Gateway)

Everything else in this wiki treats the Matrix Gateway as a wall of flaps. But under the split-flap costume it is, physically, a grid of RGB LEDs — and the Matrix Gateway can hand that grid over as a plain framebuffer, bypassing the split-flap emulation entirely, so you can draw anything on it. A real SplitFlap Gateway has no framebuffer at all, so this is the one place a Matrix wall does something the hardware it emulates simply cannot — and every endpoint on this page is Matrix-only.

You rarely drive this by hand: the Companion exposes all of it through canvas apps (see Built-in Apps and Writing Apps §7). This page is the integrator's reference; the machine-readable contract is the Canvas tag of the Matrix Gateway's openapi.yaml.

How it behaves

• Takeover. Pushing a frame, ops, an image, an animation, a ticker or an effect stands the reel renderer down — the same way it does during an OTA — and the canvas owns every pixel.
• Release. POST /api/canvas {"active":false} (or POST /api/canvas/effect {"type":"none"}) repaints the wall from the modules' current state.
• Auto-reclaim. Any split-flap command that changes the wall — a character, an index, a home, whether it arrives over /api/frames/*, /api/display/cells or MQTT — takes the panel back automatically, so sending content to the wall always shows it. Quiet Time reclaims it too.
• Nothing is persisted. A reboot always returns to the flaps.

Discovery

A client learns the whole surface from GET /api/capabilities (see Gateway API) — a physical gateway carries none of these keys, so their presence is the signal that this is a Matrix wall:

"canvas":  { "formats": ["rgb888","rgb565","qoi"], "width": 256, "height": 64,
             "rect": true, "anim": true, "ticker": true, "readback": true },
"effects": ["plasma","fire","matrix","fliporama","clock","life"],
"effectParams": ["hue","density"]

GET /api/canvas reports the live state: {active, width, height, formats, effect, anim, ticker, effects} — take the panel dimensions from here before you size a frame.

Pixel formats

Format	Bytes/px	Notes
rgb888	3	R G B per pixel.
rgb565	2	Big-endian (high byte first). Halves the upload; the panel's colour depth loses nothing to it.
qoi	—	QOI: lossless, ~2–4× smaller than raw, one-pass decode.

For a raw frame or rectangle the format is inferred from the body length — a full frame is unambiguously W×H×3 or W×H×2, so nothing else has to be signalled. A ?fmt= query arg is accepted but ignored (a raw-body handler cannot read URL args); the length is authoritative.

Endpoints

Method	Path	Purpose
GET / POST	/api/canvas	Canvas state / take over ({"active":true} blanks) or release (false).
PUT	/api/canvas/frame	Blit a full raw frame — W×H, row-major, top-left, rgb888 or rgb565.
GET	/api/canvas/frame	Read the panel back as raw pixels (a screenshot).
PUT	/api/canvas/rect	Update one rectangle without resending the whole panel.
PUT	/api/canvas/qoi	A full-panel QOI image.
PUT	/api/canvas/anim	Upload an animation that loops on-device from PSRAM.
POST	/api/canvas/ticker	One line of scrolling text, rendered on-device.
POST	/api/canvas/ops	A JSON batch of draw commands (shapes + labels).
POST	/api/canvas/effect	Start an on-device effect (or none to release).

PUT /api/canvas/frame — a full raw frame

Body is W×H pixels, row-major, top-left origin, streamed straight to the back buffer (no multi-KB frame is buffered whole). Length must equal W×H×3 (rgb888) or W×H×2 (rgb565) exactly, or it is a 400. Reply {ok, width, height, pixels}.

GET /api/canvas/frame — framebuffer readback

A screenshot of whatever is on screen — wall, effect, canvas, animation or ticker — reconstructed from the framebuffer. Read-only: it never takes the panel over, so a UI can poll it for a live preview. The body is raw pixels (?fmt=rgb888 default, or ?fmt=rgb565); the dimensions and format are echoed in X-Canvas-Width, X-Canvas-Height and X-Canvas-Format so a client needn't have called GET /api/canvas first.

Two caveats: the colours are quantised to the panel's real bit depth (it is what is physically lit, not the intended image), and brightness — an OE duty cycle — is not in the framebuffer, so a dim wall still reads back at full value.

PUT /api/canvas/rect — partial update

Update one rectangle without resending the whole panel — the changed area costs only its own bytes. The body is an 8-byte big-endian header — x, y, w, h, one unsigned 16-bit each — then w×h pixels, rgb888 or rgb565 by the remaining length. It is drawn on top of what is on screen (the back buffer is synced to the live frame first), so everything outside the rectangle is untouched. Off-panel pixels are dropped.

PUT /api/canvas/qoi — a compressed image

A full-panel QOI image whose dimensions must match the panel. Buffered in PSRAM and decoded in one pass. Reply {ok, width, height}; 400 if it isn't a QOI image for this panel, 507 if memory is low (see below).

PUT /api/canvas/anim — an on-device animation loop

Upload a short loop once and it plays on-device from PSRAM, so the client can disconnect. The body is a 14-byte header followed by the frames back-to-back:

Bytes	Field
0–3	magic	"MPGA"
4	version	1
5	format	2 = rgb565, 3 = rgb888
6	fps	1–60
7	flags	bit 0 = loop
8–9	width	u16 (must match panel)
10–11	height	u16 (must match panel)
12–13	frames	u16

Then frames × W × H × (2 or 3) bytes of frame data. The store is capped at ~1.5 MB of PSRAM (≈ 48 frames at 256×64 rgb565). Reply {ok, frames}; errors 400 (bad header / short upload), 413 (over budget), 503/507 (memory).

POST /api/canvas/ticker — scrolling text

{"text": "…", "color": [r,g,b], "speed": 1-20} scrolls one line across the panel on-device, no streaming. The font is auto-picked for the panel height. An empty text hands the panel back to the wall. Refused with 409 during Quiet Time.

POST /api/canvas/ops — draw commands

A JSON array of ops, applied in order and presented: clear, pixel, hline, vline, rect (outline or fill), text (the bundled CP1252 faces, sizes 8–20) and show. Colours are [r,g,b]; off-panel pixels are dropped; unknown ops are skipped. Draws shapes and labels without composing a whole frame client-side.

POST /api/canvas/effect — on-device effects

{"type": "<name>", "speed": 1-10, "hue"?: 0-255, "density"?: 1-100}. The effect is animated by the panel itself at its native rate with nothing on the network — pushing frames over HTTP tops out near 8 fps; an on-device effect is limited only by the panel. "type":"none" releases. Refused with 409 during Quiet Time. Reply {ok, effect, speed, hue, density}.

Effect	What it is	hue	density
plasma	Classic rolling plasma field	tints the palette	—
fire	Doom-style fire from the bottom row	—	—
matrix	Falling "digital rain"	recolours the rain (default green)	—
fliporama	The whole board flips through random glyphs	—	churn rate
clock	Big Orbitron HH:MM + date + a seconds bar	—	—
life	Conway's Game of Life, cells coloured by age	tints the cells	seed %

hue is 0–255 around the colour wheel; density is 1–100. Omit a parameter and the effect keeps its own default — an unparameterised start looks exactly as it always did. A wall advertises which effects and which parameters it honours in effects / effectParams.

Panel bit depth and the panel object

The framebuffer must live in the ESP32-S3's internal SRAM (PSRAM is too slow to feed the panel), so a large panel at a high bit depth can run the board out of RAM. Rather than refuse and go dark, the driver auto-clamps the bit depth down to the deepest that fits — a 256×64 panel lands on depth 3 — and GET /api/status reports the actual running depth in its panel object:

"panel": { "ok": true, "w": 256, "h": 64, "depth": 3, "cols": 30, "rows": 5, "font": "…", "vmods": 150 }

A note on memory: the 507 guard

A big panel (256×64) runs canvas uploads close to the firmware's reboot floor. The large ops — PUT /api/canvas/anim, PUT /api/canvas/qoi and GET /api/canvas/frame (readback) — therefore check free memory first and answer 507 ("try again in a moment") when it is already low, rather than risk a reboot. A client should treat 507 as retry shortly, not a hard failure. The companion-critical raw-frame path (PUT /api/canvas/frame) is left ungated so a continuous stream is never interrupted.

---

See also: Matrix Gateway · Gateway API · Built-in Apps · Writing Apps