A 16-bit render canvas on ESP32: 1200x1600 renders on 8 MB of PSRAM (reTerminal E1004)

[mariusandra]

A 16-bit render canvas on ESP32: 1200×1600 renders on 8 MB of PSRAM

The goal: the Seeed reTerminal E1004 (13.3" 1200×1600 Spectra 6, ESP32-S3, 8 MB PSRAM) rendering scenes on-device, full resolution, no quality compromise. Before this PR the 13.3" class needed the 16 MB module: the RGBA canvas alone is 7.3 MiB.

What changed

Leg 1 — pixie (FrameOS/pixie#rgb565-surface, pinned here). Image gains a format: pfRgbx (everything as before) or pfRgb565, a packed 16-bit RGB surface with no alpha. Every drawing operation works on one — fills, antialiased paths and text, draw with any blend mode, gradients, opacity, views, copies, the scaled and streamed decoders writing straight into it. shadow/spread raise; minify/magnify convert; the opaque/transparent predicates answer truthfully. RGBX output is untouched — every xray score in the pixie suite is identical to before. tests/test_rgb565.nim is an oracle: each scenario is drawn onto an RGBX and a 565 canvas and the quantised RGBX result must match — bit-exact for a single layer over a representable backdrop, one 5-bit step for compound cases. The SIMD variants guard on the format too (on amd64/arm64 hasSimd replaces the scalar body outright).

Leg 2 — the dither streams (utils/dither.nim). forEachPaletteDithered / forEachGrayDithered walk any image with two rows of error state beside it, reading through pixie's accessors and never writing the canvas. Bit-exact with the in-place version they replace — including its clip-per-step semantics — tested against the old algorithm (kept verbatim in the test) across bit depths, RGBX/565 canvases and views. The ESP32 packers are rewritten on top of them: the grey packer lost an 8-byte-per-pixel float scratch, the two-plane packer a full RGBA copy, the palette packer stops mutating the canvas. The Pi drivers get the same streamed ditherPaletteIndexed (same bytes out, one less full RGBA copy).

Leg 3 — the canvas is 565, claimed once at boot. embedded_runtime.nim's renderCanvas() wraps a PSRAM block the firmware reserves in main.c right after the fit check and before fos_wifi_init (frameos_nim_reserve_canvas), so the multi-MB contiguous run exists before Wi-Fi/TLS fragment the heap, and reuses it every render — no per-render allocate/collect churn. Nothing aliases the canvas across renders (the value pipeline never caches the live canvas), so reuse is safe. Error frames and the demo scene render into it too. -d:frameosCanvasRgbx keeps the RGBA canvas for A/B. The Pi runtime, HDMI/framebuffer and the wasm preview are untouched and keep full RGBA.

Leg 4 — the E1004 itself. Preset seeed_reterminal_e1004 (EPD_13in3e on the E-series bus: cs2=2, rst=38, pwr=12 — the pins ESPHome's integrated-board definition bakes in) in the backend table, the console presets[], both frontends and the cloud flasher. Its T133A01 panel wants the vendor's analogue tuning, so EPD_13in3e.c gains a runtime variant (EPD_13IN3E_SetVariant: 0x74/0x50/0x05/0x06 values, an extra 0xA5, no CCSET) that frameos_display selects from the preset — the panel key stays EPD_13in3e everywhere, the same way the PhotoPainter PMIC is selected.

Budget math moved 4→2 in its mirrors: FOS_RENDER_CANVAS_BYTES_PER_PIXEL (boot fit check), EMBEDDED_RENDER_CANVAS_BYTES_PER_PIXEL (backend guard + deploy drawer, canvasBufferBytes key; rgbaBufferBytes kept as an alias), and EmbeddedReserveBytes aligned to the 1536*1024 the other two always were. EMBEDDED_FIRMWARE_VERSION 48.

panel	canvas (was)	+ packed + 1.5 MiB reserve	total (was)
800×480 4bpp	0.73 MiB (1.46)		2.42 MiB (3.15)
1200×1600 4bpp	3.66 MiB (7.32)		6.08 MiB (9.74) — fits 8 MB

Why it is not a quality compromise

Every panel the firmware drives is a dithered e-paper, and the dither keeps its own full-precision error rows; the 5/6-bit colour the canvas holds is below what the output can show (a 6-colour Floyd–Steinberg field has per-pixel error dozens of times the 565 step). Measured on a 1200×1600 scene (mandrill photo + colour and grey gradients + AA text) dithered to Spectra 6 through both canvases: canvas delta before dither max 8/255, mean 2.4/255; palette population per colour identical within 0.05%; the panel renders are visually indistinguishable at 1:1 — including the grey ramp, where the 565 canvas shows visible steps and the dithered output does not. (Dithered dot positions differ on 38% of pixels, which is what error diffusion does to any input perturbation; the picture is the same.)

Verified

• pixie: full suite, 273 xrays identical to baseline; test_rgb565 (31 scenarios); scalar path with -d:pixieNoSimd.
• FrameOS Nim: 122 test files green against the new pixie (utils, interpreter, cache/spool, decode, all render/data apps, drivers); test_dither bit-exactness suite.
• Backend: test_embedded_firmware.py 83 passed (incl. "13.3" fits 8 MB", E1004 preset + provisioning plan).
• Frontend tsc clean; cloud flasher vitest 25 passed; auth-web typecheck clean.
• Firmware: idf.py build (S3 32 MB profile) clean; ci_build_image.sh + QEMU boot smoke.

Not verified (no hardware on this desk)

• A real E1004 render. The T133A01 tuning values come from the vendor driver via ESPHome's model; wrong tuning values mean ghosting or a failed refresh, not a brick. The E1004's buttons and microSD pins are not published and stay unset.
• A 1200×1600 PSRAM low-water measurement on an 8 MB board — the 1.5 MiB reserve was sized at 800×480; the pieces that scale with width are small, but this is the number to take first.

Notes

• Commits are unsigned: the signing agent needs an interactive touch I could not give it. git rebase --exec 'git commit --amend --no-edit -S' main re-signs them.
• Possible follow-ups, deliberately not in this PR: noise-shaped quantisation on write (would make even the canvas band-free — not needed for any dithered output), a 565 owned scratch so cached full-bleed producers keep their cache on 8 MB, streaming packed rows to the panel driver to drop the packed buffer.

🤖 Generated with Claude Code