Canvas

The Canvas API (Matrix Gateway)

Everything else in this wiki treats the Matrix Gateway as a wall of flaps. But under the split-flap costume it is, physically, a grid of RGB LEDs — and the Matrix Gateway can hand that grid over as a plain framebuffer, bypassing the split-flap emulation entirely, so you can draw anything on it. A real SplitFlap Gateway has no framebuffer at all, so this is the one place a Matrix wall does something the hardware it emulates simply cannot — and every endpoint on this page is Matrix-only.

You rarely drive this by hand: the Companion exposes all of it through canvas apps (see Built-in Apps and Writing Apps §7). This page is the integrator's reference; the machine-readable contract is the Canvas tag of the Matrix Gateway's openapi.yaml.

How it behaves

• Takeover. Pushing a frame, ops, an image, an animation, a ticker or an effect stands the reel renderer down — the same way it does during an OTA — and the canvas owns every pixel.
• Release. POST /api/canvas {"active":false} (or POST /api/canvas/effect {"type":"none"}) repaints the wall from the modules' current state.
• Auto-reclaim. Any split-flap command that changes the wall — a character, an index, a home, whether it arrives over /api/frames/*, /api/display/cells or MQTT — takes the panel back automatically, so sending content to the wall always shows it. Quiet Time reclaims it too.
• Nothing on screen is persisted. A reboot always returns to the flaps — with two 2.1-era exceptions: animations and fonts saved to the on-device library survive as files, and a configured boot animation autoplays at power-on until the first display command supersedes it (see the animation library).

Discovery

A client learns the whole surface from GET /api/capabilities (see Gateway API) — a physical gateway carries none of these keys, so their presence is the signal that this is a Matrix wall:

"canvas":  { "formats": ["rgb888","rgb565","qoi"], "width": 256, "height": 64,
             "rect": true, "anim": true, "ticker": true, "readback": true,
             "ops": ["clear","pixel","hline","vline","line","rect","circle","ellipse",
                     "triangle","roundrect","gradient","polyline","text","image",
                     "sprite","scroll","show"] },
"effects": ["plasma","fire","matrix","fliporama","clock","life"],
"effectParams": ["hue","density"]

(The sprite op in canvas.ops is the fw 2.1 marker — a wall that lists it also carries the animation/font libraries, transitions, the atlas and GIF import.)

GET /api/canvas reports the live state: {active, width, height, formats, effect, anim, ticker, atlas, effects} — take the panel dimensions from here before you size a frame. atlas (fw 2.1) is the loaded sprite sheet's {tiles, w, h}, or null when none has been uploaded.

Pixel formats

Format	Bytes/px	Notes
rgb888	3	R G B per pixel.
rgb565	2	Big-endian (high byte first). Halves the upload; the panel's colour depth loses nothing to it.
qoi	—	QOI: lossless, ~2–4× smaller than raw, one-pass decode.

For a raw frame or rectangle the format is inferred from the body length — a full frame is unambiguously W×H×3 or W×H×2, so nothing else has to be signalled. A ?fmt= query arg is accepted but ignored (a raw-body handler cannot read URL args); the length is authoritative.

Endpoints

Method	Path	Purpose
GET / POST	/api/canvas	Canvas state / take over ({"active":true} blanks) or release (false).
PUT	/api/canvas/frame	Blit a full raw frame — W×H, row-major, top-left, rgb888 or rgb565.
GET	/api/canvas/frame	Read the panel back as raw pixels (a screenshot).
PUT	/api/canvas/rect	Update one rectangle without resending the whole panel.
PUT	/api/canvas/qoi	A full-panel QOI image.
PUT	/api/canvas/anim	Upload an animation that loops on-device from PSRAM.
POST	/api/canvas/anim/save	(fw 2.1) Persist the loaded animation to the on-device library.
POST	/api/canvas/anim/play	(fw 2.1) Load a library animation and play it.
POST	/api/canvas/anim/delete	(fw 2.1) Delete a library animation.
GET	/api/canvas/anims	(fw 2.1) List the animation library.
PUT	/api/canvas/gif	(fw 2.1) Import an ordinary animated GIF into the animation store.
POST	/api/canvas/transition	(fw 2.1) Configure a crossfade / wipe / slide for full-frame presents.
PUT	/api/canvas/atlas	(fw 2.1) Upload a sprite tile sheet for the ops sprite op.
PUT	/api/canvas/font	(fw 2.1) Upload a custom font (the custom face).
POST	/api/canvas/font/save / font/delete	(fw 2.1) Persist / delete a library font.
GET	/api/canvas/fonts	(fw 2.1) List the font library.
POST	/api/canvas/ticker	One line of scrolling text, rendered on-device — exclusive or (fw 2.1) an overlay.
POST	/api/canvas/ops	A JSON batch of draw commands (shapes + labels + sprites).
POST	/api/canvas/effect	Start an on-device effect (or none to release).

PUT /api/canvas/frame — a full raw frame

Body is W×H pixels, row-major, top-left origin, streamed straight to the back buffer (no multi-KB frame is buffered whole). Length must equal W×H×3 (rgb888) or W×H×2 (rgb565) exactly, or it is a 400. Reply {ok, width, height, pixels}.

With a transition configured (below), the frame is staged in PSRAM and tweened on-device instead of hard-cutting.

POST /api/canvas/transition — how full frames present (fw 2.1)

{"type": "none|crossfade|wipe|slide", "ms": 100-2000} — sticky until changed, runtime-only (a reboot returns to hard cuts). Applies to subsequent full-frame PUT /api/canvas/frame presents only: the incoming frame is staged in PSRAM, the outgoing frame is captured from the framebuffer, and the tween renders on the gateway — crossfade blends, wipe sweeps left to right, slide pushes the old frame off to the left. ms is the duration (default 400). rect, qoi and anim are untouched, and if the staging buffers can't be allocated the frame falls back to a hard cut rather than failing.

GET /api/canvas/frame — framebuffer readback

A screenshot of whatever is on screen — wall, effect, canvas, animation or ticker — reconstructed from the framebuffer. Read-only: it never takes the panel over, so a UI can poll it for a live preview. The body is raw pixels (?fmt=rgb888 default, or ?fmt=rgb565); the dimensions and format are echoed in X-Canvas-Width, X-Canvas-Height and X-Canvas-Format so a client needn't have called GET /api/canvas first.

Two caveats: the colours are quantised to the panel's real bit depth (it is what is physically lit, not the intended image), and brightness — an OE duty cycle — is not in the framebuffer, so a dim wall still reads back at full value.

PUT /api/canvas/rect — partial update

Update one rectangle without resending the whole panel — the changed area costs only its own bytes. The body is an 8-byte big-endian header — x, y, w, h, one unsigned 16-bit each — then w×h pixels, rgb888 or rgb565 by the remaining length. It is drawn on top of what is on screen (the back buffer is synced to the live frame first), so everything outside the rectangle is untouched. Off-panel pixels are dropped.

PUT /api/canvas/qoi — a compressed image

A full-panel QOI image whose dimensions must match the panel. Buffered in PSRAM and decoded in one pass. Reply {ok, width, height}; 400 if it isn't a QOI image for this panel, 507 if memory is low (see below).

PUT /api/canvas/anim — an on-device animation loop

Upload a short loop once and it plays on-device from PSRAM, so the client can disconnect. The body is a 14-byte header followed by the frames back-to-back:

Bytes	Field
0–3	magic	"MPGA"
4	version	1
5	format	2 = rgb565, 3 = rgb888
6	fps	1–60
7	flags	bit 0 = loop
8–9	width	u16 (must match panel)
10–11	height	u16 (must match panel)
12–13	frames	u16

Then frames × W × H × (2 or 3) bytes of frame data. The store is capped at 8 MB of PSRAM (≈ 256 frames at 256×64 rgb565). Reply {ok, frames}; errors 400 (bad header / short upload), 413 (over budget), 503/507 (memory).

The animation library, and the boot animation (fw 2.1)

The animation store was always volatile. Firmware 2.1 gives it a library on the gateway's own flash (the 23.9 MB FATFS partition), so a loop is uploaded once and kept:

Call	Does
POST /api/canvas/anim/save {"name":"x"}	Write whatever is loaded — an MPGA upload, a GIF import, a previous play — to /anim/<name>.mpg, byte-identical to the wire format. 409 when nothing is loaded.
POST /api/canvas/anim/play {"name":"x"}	Load a library entry into the store and play it. 404 unknown name, 409 during Quiet Time.
POST /api/canvas/anim/delete {"name":"x"}	Remove it.
GET /api/canvas/anims	The library: [{name, bytes, frames, w, h, fps, loop}, …], metadata re-read from each file's header.

Names are 1–24 characters of a-z 0-9 - _ (a bad name is a 400); saving over an existing name replaces it, atomically enough that an interrupted save never corrupts the old entry. Storage errors surface as 503 (filesystem unavailable) and 507 (full / write failed).

The boot animation. Set bootAnim to a library name (POST /api/config/settings, reported by GET /api/config; empty string disables) and the gateway plays it at boot, before WiFi is even up — a splash screen from flash. The first display command supersedes it, exactly like any other animation.

PUT /api/canvas/gif — GIF import (fw 2.1)

The MPGA format needs a packer; a GIF doesn't. Upload an ordinary animated GIF (whole file, 4 MB cap) and the gateway decodes it on-device into the same animation store the anim upload fills — GIF deltas, transparency and frame disposal composited properly, a GIF smaller than the panel centred on black, one larger than the panel rejected with 400. Playback fps comes from the GIF's own frame delays, clamped 1–30. It plays immediately, and because it lands in the same store, anim/save persists it and bootAnim can name it. Reply {ok, frames, fps}; errors 400 (undecodable / oversized-for-panel / truncated), 413 (over 4 MB, or more frames than the store holds), 503/507 (memory).

POST /api/canvas/ticker — scrolling text, exclusive or overlay

{"text": "…", "color": [r,g,b], "speed": 1-20} scrolls one line across the panel on-device, no streaming. The font is auto-picked for the panel height. An empty text hands the panel back to the wall — and stops an overlay ticker too. Starting one is refused with 409 during Quiet Time.

Two fw 2.1 additions:

• "overlay": true composites the ticker as a lower-third band over whatever else is presenting — the flap wall, an effect, an animation, a pushed frame — instead of taking the panel over. It rides a hook on every presented frame, so it survives page and mode changes; only an explicit {"text":""}, Quiet Time, or replacing it with a non-overlay ticker removes it. "band": false drops the black band and scrolls the glyphs straight over the content.
• "font": "…" names a face: custom is the uploaded slot (see fonts), any other name loads a library face. Unknown names fall back to the built-in face — a missing font scrolls text, it never errors.

Reply {ok, active, overlay}.

POST /api/canvas/ops — draw commands

A JSON array of ops, applied in order and presented — shapes and labels without composing a whole frame client-side. Colours are [r,g,b] (0–255); off-panel pixels are dropped; unknown ops are skipped. The set a wall offers is advertised as capabilities.canvas.ops.

Op	Shape
clear	fill the panel (color, black by default)
pixel	x, y
hline / vline	x, y, w / x, y, h
line	x, y → x1, y1 (Bresenham)
rect	x, y, w, h, outline or fill: true
circle	x, y centre, r radius, outline or fill: true
ellipse	x, y centre, semi-axes rx, ry, outline or fill: true
triangle	vertices x,y · x1,y1 · x2,y2, outline or fill: true
roundrect	x, y, w, h, corner r, outline or fill: true
gradient	fill x, y, w, h with a linear gradient from → to, dir "v" (default) or "h"
polyline	connect points: [[x,y],…] with lines
text	x, y, s, size (bundled CP1252 faces, 8–20), align "left" (default) / "center" / "right", font (fw 2.1) — "custom" or a library face; unknown names keep the built-in face
image	blit a base64 sprite: x, y, w, h, fmt (rgb888/rgb565), data — ≤ 8 KB decoded (use PUT /api/canvas/frame for a full panel)
sprite	(fw 2.1) blit tile i of the uploaded atlas at x, y — magenta pixels transparent; skipped (not counted) with no atlas or i out of range
scroll	shift the current frame by dx, dy, filling vacated pixels with color. Make it the first op, then draw the newly-revealed edge — a marquee without resending the panel
show	present mid-batch (otherwise presented once at the end)

PUT /api/canvas/atlas — a sprite tile sheet (fw 2.1)

The image op re-sends its base64 pixels in every batch; a sprite that appears every frame deserves better. Upload one tile sheet and the sprite op blits from it by index — the sheet stays loaded across batches until the next upload replaces it (no panel takeover; it is data, not pixels).

The body is a 12-byte big-endian header — magic "MPTA", version 1, format (2 = rgb565 BE, 3 = rgb888), tileW u16, tileH u16, tiles u16 — then tiles × tileW × tileH pixels, tile after tile, row-major. Magenta is the transparent colour (0xF81F in rgb565, 255,0,255 in rgb888), so sprites carry holes; off-panel pixels clip. The sheet is capped at 2 MB (413). Reply {ok, tiles, w, h}; GET /api/canvas reports the loaded sheet in its atlas field.

PUT /api/canvas/font — uploadable fonts (fw 2.1)

The wall's bundled faces are fixed at build time; fw 2.1 lets you bring your own. The gateway's tools/fontpack.py packs any BDF face up to 16 px wide into an MPFT blob — header "MPFT" ver=1 width height ascent, then the 216 printable CP1252 glyphs as big-endian uint16 rows — and PUT /api/canvas/font installs it (≤ 64 KB) as the face named custom. Reply {ok, font: "custom", w, h, ascent}.

The custom slot gets the same library treatment as animations, on the same naming rule (1–24 chars of a-z 0-9 - _):

Call	Does
POST /api/canvas/font/save {"name":"x"}	Persist the loaded face to /fonts/<name>.fnt. 409 when none is loaded.
POST /api/canvas/font/delete {"name":"x"}	Remove it (404 unknown).
GET /api/canvas/fonts	The library: [{name, bytes, w, h, ascent}, …].

A face is then used by name — in the ticker's font field or the ops text op — where custom means the slot as-is and a library name loads its file first. Unknown names always fall back to the built-in face rather than erroring.

POST /api/canvas/effect — on-device effects

{"type": "<name>", "speed": 1-10, "hue"?: 0-255, "density"?: 1-100}. The effect is animated by the panel itself at its native rate with nothing on the network — pushing frames over HTTP tops out near 8 fps; an on-device effect is limited only by the panel. "type":"none" releases. Refused with 409 during Quiet Time. Reply {ok, effect, speed, hue, density}.

Effect	What it is	hue	density
plasma	Classic rolling plasma field	tints the palette	—
fire	Doom-style fire from the bottom row	—	—
matrix	Falling "digital rain"	recolours the rain (default green)	—
fliporama	The whole board flips through random glyphs	—	churn rate
clock	Big Orbitron HH:MM + date + a seconds bar	—	—
life	Conway's Game of Life, cells coloured by age	tints the cells	seed %

hue is 0–255 around the colour wheel; density is 1–100. Omit a parameter and the effect keeps its own default. A wall advertises which effects and which parameters it honours in effects / effectParams.

Panel bit depth and the panel object

The framebuffer must live in the ESP32-S3's internal SRAM (the panel's DMA stream and WiFi would contend for the PSRAM bus, so even a big PSRAM doesn't hold it), and a large panel at a high bit depth can run that internal RAM out. Rather than refuse and go dark, the driver auto-clamps the bit depth down to the deepest that fits — a 256×64 panel lands on depth 3 — and GET /api/status reports the actual running depth in its panel object:

"panel": { "ok": true, "w": 256, "h": 64, "depth": 3, "cols": 30, "rows": 5, "font": "…", "vmods": 150 }

A note on memory: the 507 guard

A big panel (256×64) runs canvas uploads close to the firmware's reboot floor. The large ops — PUT /api/canvas/anim, PUT /api/canvas/qoi, GET /api/canvas/frame (readback), and since fw 2.1 PUT /api/canvas/atlas and PUT /api/canvas/gif — therefore check free memory first and answer 507 ("try again in a moment") when it is already low, rather than risk a reboot. A client should treat 507 as retry shortly, not a hard failure. The companion-critical raw-frame path (PUT /api/canvas/frame) is left ungated so a continuous stream is never interrupted. (The stores themselves — animation frames, staged transition frames, the atlas, the font table — all live in the board's 16 MB PSRAM, not the internal RAM this guard protects.)

---

See also: Matrix Gateway · Gateway API · Built-in Apps · Writing Apps