Canvas

The Canvas API (Matrix Gateway)

Everything else in this wiki treats the Matrix Gateway as a wall of flaps. But under the split-flap costume it is, physically, a grid of RGB LEDs — and the Matrix Gateway can hand that grid over as a plain framebuffer, bypassing the split-flap emulation entirely, so you can draw anything on it. A real SplitFlap Gateway has no framebuffer at all, so this is the one place a Matrix wall does something the hardware it emulates simply cannot — and every endpoint on this page is Matrix-only.

You rarely drive this by hand: the Companion exposes all of it through canvas apps (see Built-in Apps and Writing Apps §7). This page is the integrator's reference; the machine-readable contract is the Canvas tag of the Matrix Gateway's openapi.yaml.

How it behaves

• Takeover. Pushing a frame, ops, an image, an animation, a ticker or an effect stands the reel renderer down — the same way it does during an OTA — and the canvas owns every pixel.
• Release. POST /api/canvas {"active":false} (or POST /api/canvas/effect {"type":"none"}) repaints the wall from the modules' current state.
• Auto-reclaim. Any split-flap command that changes the wall — a character, an index, a home, whether it arrives over /api/frames/*, /api/display/cells or MQTT — takes the panel back automatically, so sending content to the wall always shows it. Quiet Time reclaims it too.
• Nothing is persisted. A reboot always returns to the flaps.

Discovery

A client learns the whole surface from GET /api/capabilities (see Gateway API) — a physical gateway carries none of these keys, so their presence is the signal that this is a Matrix wall:

"canvas":  { "formats": ["rgb888","rgb565","qoi"], "width": 256, "height": 64,
             "rect": true, "anim": true, "ticker": true, "readback": true },
"effects": ["plasma","fire","matrix","fliporama","clock","life"],
"effectParams": ["hue","density"]

GET /api/canvas reports the live state: {active, width, height, formats, effect, anim, ticker, effects} — take the panel dimensions from here before you size a frame.

Pixel formats

Format	Bytes/px	Notes
rgb888	3	R G B per pixel.
rgb565	2	Big-endian (high byte first). Halves the upload; the panel's colour depth loses nothing to it.
qoi	—	QOI: lossless, ~2–4× smaller than raw, one-pass decode.

For a raw frame or rectangle the format is inferred from the body length — a full frame is unambiguously W×H×3 or W×H×2, so nothing else has to be signalled. A ?fmt= query arg is accepted but ignored (a raw-body handler cannot read URL args); the length is authoritative.

Endpoints

Method	Path	Purpose
GET / POST	/api/canvas	Canvas state / take over ({"active":true} blanks) or release (false).
PUT	/api/canvas/frame	Blit a full raw frame — W×H, row-major, top-left, rgb888 or rgb565.
GET	/api/canvas/frame	Read the panel back as raw pixels (a screenshot).
PUT	/api/canvas/rect	Update one rectangle without resending the whole panel.
PUT	/api/canvas/qoi	A full-panel QOI image.
PUT	/api/canvas/anim	Upload an animation that loops on-device from PSRAM.
POST	/api/canvas/ticker	One line of scrolling text, rendered on-device.
POST	/api/canvas/ops	A JSON batch of draw commands (shapes + labels).
POST	/api/canvas/effect	Start an on-device effect (or none to release).

PUT /api/canvas/frame — a full raw frame

Body is W×H pixels, row-major, top-left origin, streamed straight to the back buffer (no multi-KB frame is buffered whole). Length must equal W×H×3 (rgb888) or W×H×2 (rgb565) exactly, or it is a 400. Reply {ok, width, height, pixels}.

GET /api/canvas/frame — framebuffer readback

A screenshot of whatever is on screen — wall, effect, canvas, animation or ticker — reconstructed from the framebuffer. Read-only: it never takes the panel over, so a UI can poll it for a live preview. The body is raw pixels (?fmt=rgb888 default, or ?fmt=rgb565); the dimensions and format are echoed in X-Canvas-Width, X-Canvas-Height and X-Canvas-Format so a client needn't have called GET /api/canvas first.

Two caveats: the colours are quantised to the panel's real bit depth (it is what is physically lit, not the intended image), and brightness — an OE duty cycle — is not in the framebuffer, so a dim wall still reads back at full value.

PUT /api/canvas/rect — partial update

Update one rectangle without resending the whole panel — the changed area costs only its own bytes. The body is an 8-byte big-endian header — x, y, w, h, one unsigned 16-bit each — then w×h pixels, rgb888 or rgb565 by the remaining length. It is drawn on top of what is on screen (the back buffer is synced to the live frame first), so everything outside the rectangle is untouched. Off-panel pixels are dropped.

PUT /api/canvas/qoi — a compressed image

A full-panel QOI image whose dimensions must match the panel. Buffered in PSRAM and decoded in one pass. Reply {ok, width, height}; 400 if it isn't a QOI image for this panel, 507 if memory is low (see below).

PUT /api/canvas/anim — an on-device animation loop

Upload a short loop once and it plays on-device from PSRAM, so the client can disconnect. The body is a 14-byte header followed by the frames back-to-back:

Bytes	Field
0–3	magic	"MPGA"
4	version	1
5	format	2 = rgb565, 3 = rgb888
6	fps	1–60
7	flags	bit 0 = loop
8–9	width	u16 (must match panel)
10–11	height	u16 (must match panel)
12–13	frames	u16

Then frames × W × H × (2 or 3) bytes of frame data. The store is capped at 8 MB of PSRAM (≈ 256 frames at 256×64 rgb565). Reply {ok, frames}; errors 400 (bad header / short upload), 413 (over budget), 503/507 (memory).

POST /api/canvas/ticker — scrolling text

{"text": "…", "color": [r,g,b], "speed": 1-20} scrolls one line across the panel on-device, no streaming. The font is auto-picked for the panel height. An empty text hands the panel back to the wall. Refused with 409 during Quiet Time.

POST /api/canvas/ops — draw commands

A JSON array of ops, applied in order and presented — shapes and labels without composing a whole frame client-side. Colours are [r,g,b] (0–255); off-panel pixels are dropped; unknown ops are skipped. The set a wall offers is advertised as capabilities.canvas.ops.

Op	Shape
clear	fill the panel (color, black by default)
pixel	x, y
hline / vline	x, y, w / x, y, h
line	x, y → x1, y1 (Bresenham)
rect	x, y, w, h, outline or fill: true
circle	x, y centre, r radius, outline or fill: true
ellipse	x, y centre, semi-axes rx, ry, outline or fill: true
triangle	vertices x,y · x1,y1 · x2,y2, outline or fill: true
roundrect	x, y, w, h, corner r, outline or fill: true
gradient	fill x, y, w, h with a linear gradient from → to, dir "v" (default) or "h"
polyline	connect points: [[x,y],…] with lines
text	x, y, s, size (bundled CP1252 faces, 8–20), align "left" (default) / "center" / "right"
image	blit a base64 sprite: x, y, w, h, fmt (rgb888/rgb565), data — ≤ 8 KB decoded (use PUT /api/canvas/frame for a full panel)
scroll	shift the current frame by dx, dy, filling vacated pixels with color. Make it the first op, then draw the newly-revealed edge — a marquee without resending the panel
show	present mid-batch (otherwise presented once at the end)

POST /api/canvas/effect — on-device effects

{"type": "<name>", "speed": 1-10, "hue"?: 0-255, "density"?: 1-100}. The effect is animated by the panel itself at its native rate with nothing on the network — pushing frames over HTTP tops out near 8 fps; an on-device effect is limited only by the panel. "type":"none" releases. Refused with 409 during Quiet Time. Reply {ok, effect, speed, hue, density}.

Effect	What it is	hue	density
plasma	Classic rolling plasma field	tints the palette	—
fire	Doom-style fire from the bottom row	—	—
matrix	Falling "digital rain"	recolours the rain (default green)	—
fliporama	The whole board flips through random glyphs	—	churn rate
clock	Big Orbitron HH:MM + date + a seconds bar	—	—
life	Conway's Game of Life, cells coloured by age	tints the cells	seed %

hue is 0–255 around the colour wheel; density is 1–100. Omit a parameter and the effect keeps its own default. A wall advertises which effects and which parameters it honours in effects / effectParams.

Panel bit depth and the panel object

The framebuffer must live in the ESP32-S3's internal SRAM (the panel's DMA stream and WiFi would contend for the PSRAM bus, so even a big PSRAM doesn't hold it), and a large panel at a high bit depth can run that internal RAM out. Rather than refuse and go dark, the driver auto-clamps the bit depth down to the deepest that fits — a 256×64 panel lands on depth 3 — and GET /api/status reports the actual running depth in its panel object:

"panel": { "ok": true, "w": 256, "h": 64, "depth": 3, "cols": 30, "rows": 5, "font": "…", "vmods": 150 }

A note on memory: the 507 guard

A big panel (256×64) runs canvas uploads close to the firmware's reboot floor. The large ops — PUT /api/canvas/anim, PUT /api/canvas/qoi and GET /api/canvas/frame (readback) — therefore check free memory first and answer 507 ("try again in a moment") when it is already low, rather than risk a reboot. A client should treat 507 as retry shortly, not a hard failure. The companion-critical raw-frame path (PUT /api/canvas/frame) is left ungated so a continuous stream is never interrupted.

---

See also: Matrix Gateway · Gateway API · Built-in Apps · Writing Apps