Idea: SD-backed offline photo albums from Gallery folders

[charmmmz]

Hi! After looking at the current Gallery, Rotations, Decks, and the new SD-backed deck cache, I wanted to ask about a common digital-photo-frame use case:

Choose or upload a folder of photos once, sync panel-ready frames to a storage-capable display, then let the device change photos locally in shuffle or sequential order without a Wi-Fi/server round trip for every change.

Why this seems close to what Tesserae already has

The current pieces cover much of the plumbing:

• Picture Gallery already has folders, multi-file uploads, external folder pointers, orientation filtering, and random/sequential selection per server render.
• deck_cache is runtime-advertised only when local storage is mounted, reports a capacity budget, and already provides content-addressed incremental frame sync plus orphan cleanup.
• On a live reTerminal E1004 running the current Tesserae firmware/server combination, the capability is working and reports a 64 MiB cache budget.
• Decks can cache pre-rendered pages, but modeling an album as one manually-created Dashboard/Page per photo feels like the wrong authoring experience.
• Rotations solve server-side Dashboard sequencing, but not one-time sync followed by device-local playback.

Possible user-facing model

Rather than exposing generated photo Pages, a Gallery folder could gain an action such as Use as offline album:

1. Select one or more displays that currently advertise local-storage capability.
2. Choose Fit/Fill, shuffle or sequential playback, and a refresh interval.
3. Tesserae pre-renders each photo for each target panel and syncs only missing frame digests.
4. The firmware wakes on the chosen interval, paints a cached frame, and normally keeps Wi-Fi off.
5. Shuffle could use a shuffle-bag so every photo appears once before any repeat.
6. Adding/removing photos changes the album version; the next sync fetches new frames and removes orphans.

The original JPEG/HEIC/WebP files could stay on the server. The card would hold only the device-native, already-fitted/dithered frame artifacts, preserving Tesserae as the rendering source of truth and avoiding image decoding on the ESP32.

Architecture questions

Before trying to prototype this, I would appreciate your view on where it belongs:

1. Extend Decks or introduce an Album/Frame Collection contract? Internally, the digest sync and SD file layer look reusable. But Gallery albums have different authoring, navigation, and lifecycle semantics from a graph of Dashboard pages. Would you prefer a special/generated Deck representation, or a sibling album_cache/frame-collection manifest?
2. Is device-local timed/shuffle advance in scope? Today cached Deck navigation is primarily button/touch driven, while timed Deck advance is server-owned. Would a small device-owned interval + shuffle state be a reasonable capability extension?
3. How should album size scale? The current firmware Deck model has a 32-page bound and advertises at most a 64 MiB budget. For a first slice, should an offline album intentionally inherit the 32-frame limit, or should albums use a streaming/paged manifest so larger cards can hold substantially more photos?
4. Would Web + firmware be the right first slice? Gallery already supports multi-upload, so the server UI could prove the workflow first. Companion multi-photo selection, album management, and sync progress could follow as an optional API capability rather than blocking the core feature.
5. Network cadence: would it make sense for local playback to stay offline between changes, then check the album version on the device's normal scheduled heartbeat (or a separate low-frequency sync interval), rather than associating every photo change with a server poll?

This is mainly a product/protocol question, not a request to force Gallery into Decks. The part that feels especially valuable to reuse is the proven deck_cache machinery; the user-facing object would still be an album sourced from Gallery.

Does this fit the direction you have in mind for local storage, or are Decks intentionally the only abstraction that should place rendered frames on the card?

[dmellok]

This fits, and the part you've identified as most reusable is the right part. Answers in order, plus three things worth pulling forward.

1. Neither a special Deck nor a parallel copy: factor the frame-collection layer out and give it two producers. The manifest, per-device frame digests, byte accounting, version digest, and orphan cleanup aren't deck-specific; the link graph and home-relative navigation are. So the sync layer becomes a shared primitive, a deck is one producer of a frame collection, and an album is another, sourced from a Gallery folder with its own authoring and lifecycle. Same move as SourceCrop: one primitive, two callers, no duplicated machinery and no forcing album semantics through a page graph.

This is compatible with #167 rather than competing with it. #167 is about when and what to show; the cache layer is about how frames reach the card. Different axis, so factoring it doesn't need to wait.

2. Yes to device-owned interval and shuffle, with two conditions.

The interval should reuse the page wake cadence shape I've had in the backlog (a page proposes a sleep interval, the device clamps it to its own bounds) rather than inventing an album-specific timer. Same concept, and albums are a better first customer for it than pages were.

The shuffle bag is genuinely new device state and I'm fine with it, but it breaks an assumption we just agreed in #147: once the device is advancing locally, the server no longer knows which frame is on the glass, so has_pending_render and the Companion Display card have no meaningful answer per frame. I'd define the album case explicitly as "album playing, N frames cached, version X" rather than showing a specific frame preview and implying it's what you're looking at.

3. Inherit 32 for the first slice, but shape the manifest so paging isn't a breaking change. Two notes from the server side:

Bytes are not the binding constraint. At 800x480 a 4bpp frame is around 187 KiB and a 1-bit frame around 47 KiB, so a 64 MiB budget holds roughly 350 or 1,400 frames. The firmware page count is what limits an album, not the card, and for a photo frame the gap between 32 and 350 is the gap between a demo and the feature. Worth pricing paging early even if slice one ships without it.

The existing capacity trim won't transfer as-is. It prioritises by ring distance from the deck's home page, which is meaningless for a shuffled album. Albums want a different fill order (play order, or simply until the budget is spent), so that heuristic needs to be a property of the producer rather than baked into the shared layer.

4. Yes, Web plus firmware first. Gallery already has the folders and multi-upload, so the server UI can prove the workflow end to end without any client work. Companion album management and sync progress are a good follow-up capability, not a blocker.

5. Yes, version on the heartbeat. The deck manifest already carries a version digest that /status echoes, and it changes on any content edit or re-render, so an album version rides the same mechanism for free. No per-photo polling, and no new cadence to reason about.

One thing in this idea's favour that you undersold: albums are dramatically cheaper to warm than decks. Deck warming needs the browser pool per page, while photos only need fit_to_panel plus dithering, with no Chromium involved. A few hundred photos is seconds of Pillow work rather than minutes of screenshots, so the pre-render cost that makes large decks awkward mostly isn't there.

Sequencing. Natural split is the usual one: I take the server side (factor the frame-collection layer, the album object, the Gallery action, per-device pre-render and sync), you take firmware playback (interval wake, shuffle bag, and paging if we go past 32). Worth agreeing the manifest shape between us before either side starts, since it's the contract both halves depend on.

[bablokb]

Since I will add SD-buffering anyhow (to support low-memory systems), I thought about adding this offline feature too. It would fit naturally into the minimal display program.

There is one caveat: currently there is only one "push" slot. So anybody sending multiple photos to the offline slideshow system, only one of them will succeed.

Basically this is what Frameo is doing. My parents have one of these displays backed by Frameo. Children and grandchildren can send images and they are automatically downloaded to the device and then it is all offline. Management of images is offline, i.e. if you run out of SD space, you have to delete images manually.

[dmellok]

A second client for that layer is a good argument for pinning the manifest shape
before either of us builds against it.

The single push slot shouldn't apply here. A push means "show this now" and is
one-at-a-time by nature; an album is a synced collection plus local playback, so
several people adding photos is several mutations of one Gallery folder, not
competing pushes. One version bump, one delta on the next check, nothing races.

Where this could differ from Frameo is storage management. Originals stay
server-side and Tesserae already knows the device's advertised capacity, so the
sync can decide what fits rather than someone deleting files off a card when it
fills. Curation happens in the Gallery with the photos actually visible, and the
card reconciles on the next sync.

[charmmmz]

Thanks — this split works for me. I’ll take the firmware playback side, and I agree we should pin the shared manifest before either implementation starts.

From the firmware boundary, I think the contract needs to settle these points first:

1. Capability migration. deck_cache currently means both “SD frame storage exists” and “this firmware understands Deck sync.” If the primitive becomes producer-neutral, should capable firmware advertise an additive frame_cache capability while retaining deck_cache for compatibility, or should deck_cache remain the transport capability despite the name?
2. Shared core versus producer payload. I expect the common document to own collection id/kind/version plus frame id, digest, byte size and cache eligibility. Deck links/home and Album playback order should remain producer-owned extensions, so the shared cache code never needs to understand either graph distance or shuffle semantics.
3. Paging shape. I’m happy to enforce 32 cached frames in slice one, but I would like the wire shape to include stable total/order identity and a continuation mechanism from day one, even if the first server returns a single page. That lets firmware replace the fixed array later without changing the collection contract.
4. Playback block. For an album I would expect something close to mode: sequential | shuffle, a proposed interval_s, and perhaps an explicit repeat/reshuffle policy. Firmware will clamp the interval to board/power bounds. Shuffle-bag state can be a bitset for the first 32-frame slice and should reset when the collection version changes.
5. Truthful reporting. Agreed that the server/Companion should not claim an exact current frame once playback is local. Firmware can report collection id, version, cached/total counts and playing | paused | syncing | error; an exact frame digest would only be a last-reported observation, not a live Current Screen contract.
6. Interruption semantics. We should define what a normal Dashboard/Image push does while an album is playing: stop the album, pause it until explicit resume, or temporarily interrupt it and resume at the next interval. I lean toward explicit pause/stop for v1 so a user’s direct push is never unexpectedly replaced by the offline timer.
7. Quiet hours. Server quiet hours cannot govern a radio-off repaint unless the device receives a local window. For v1, should offline Album playback ignore server quiet hours, or should the playback block carry a device-local allowed window/timezone?

Because the server refactor determines the endpoint and shared document envelope, it probably makes sense for you to sketch the first JSON shape. I can review it against the current fixed-memory parser/cache constraints and then immediately turn the accepted shape into host-side parser, shuffle-state and wake-policy tests before wiring the ESP-IDF path.

I’ll hold off on coding against an invented schema in the meantime, but I’m ready to start as soon as that shape is pinned.

[charmmmz]

I compared the existing v1 deck manifest and the planned v2 state-bundle shape against the firmware memory/runtime constraints. Here is a concrete candidate we can edit rather than continuing at the noun level.

Capability (additive, with v1 compatibility):

{
  "frame_cache": {
    "schema": 1,
    "capacity_bytes": 67108864,
    "max_frames": 32
  },
  "deck_cache": {
    "schema": 1,
    "capacity_bytes": 67108864
  }
}

Slice one can advertise both. Old servers keep using deck_cache; the new collection endpoint is offered only when frame_cache is present. max_frames makes the actual firmware limit explicit instead of making the server infer it from bytes.

Heartbeat version signal:

{
  "collection": {
    "id": "album:kitchen",
    "kind": "album",
    "version": "a1b2c3d4e5f60718"
  }
}

Absent means no collection is active. This can later carry a Deck collection too, but it need not replace the existing deck envelope in the compatibility slice.

Paged manifest candidate:

{
  "schema": 1,
  "collection_id": "album:kitchen",
  "kind": "album",
  "version": "a1b2c3d4e5f60718",
  "total_frames": 127,
  "cursor": null,
  "next_cursor": "opaque-version-bound-cursor",
  "frames": [
    {
      "frame_id": "photo:7f8c",
      "position": 0,
      "digest": "0f1e2d3c4b5a6978",
      "bytes": 960000,
      "ttl_s": 0,
      "cache": true,
      "url": "/api/v1/device/frame01/collection/frame/0f1e2d3c4b5a6978"
    }
  ],
  "producer": {
    "album": {
      "playback": {
        "mode": "shuffle",
        "interval_s": 1800,
        "repeat": "reshuffle"
      }
    }
  }
}

My intent behind these fields:

• version covers frame membership, stable order, rendered digests and playback settings. A continuation cursor is opaque and bound to that version; if it goes stale, firmware discards the partial walk and starts at page one.
• position gives a stable order identity across pages. The first firmware may consume one page and enforce max_frames == 32; later firmware can walk next_cursor without a schema change.
• Frame metadata is the shared collection primitive. The producer object is opaque to the cache/diff/file layer. Deck can project producer.deck = {entry_frame_id, links, home}; Album projects playback only.
• Including an authenticated relative url lets the shared downloader stay producer-neutral and also matches the direction of the v2 bundle document.
• ttl_s: 0 means immutable until the collection version removes/replaces the digest. This is the natural Album default.
• cache: false remains server-side producer trimming. Album fill order is manifest position; Deck keeps its ring-distance policy before projection.

Device report (never an exact live photo):

{
  "collection_state": {
    "id": "album:kitchen",
    "kind": "album",
    "version": "a1b2c3d4e5f60718",
    "state": "playing",
    "cached_frames": 32,
    "total_frames": 127
  }
}

One runtime detail the manifest should not accidentally conflate: playback cadence and network sync cadence are two deadlines. Firmware should wake at the earlier one; a frame-only deadline paints from SD and returns to sleep with the radio off, while the normal server cadence still performs the heartbeat/version check. A downloaded direct frame push pauses Album playback before paint, so it cannot be overwritten at the next local timer.

If this is close to what you had in mind, the remaining choices I need pinned before coding are the endpoint names, whether url is included, the interval clamp bounds/default, and whether repeat is worth carrying in schema 1 or should simply mean reshuffle forever.

[dmellok]

This works. Accepting the split (you on firmware playback, me on the server manifest + Gallery "Use as offline album" authoring), and the additive frame_cache capability alongside deck_cache so v1 stays compatible. The shared-core / producer-object line is exactly right: the cache owns collection id/kind/version + per-frame digest/bytes/cache/url, and never learns shuffle or deck-graph semantics.

On your schema in the second comment, I'll take it as the base for the pinned manifest doc with only small notes: single page in slice 1 but next_cursor in the wire shape from the start; frame url served content-addressed from the server the same way the deck cache already does. max_frames explicit is good.

The one open decision is interruption (#6). I lean: a one-off Image/Dashboard push interrupts and auto-resumes at the next interval; a deliberate content change (rotation/deck/album swap) stops the album. Reporting stays honest, collection id/version/counts + playing|paused|syncing|error, no exact current-frame claim once playback is local.

Let me pin the shared manifest doc from your candidate, then we both build against it.

[dmellok]

Pinned the shared contract from your candidate: docs/dev/frame-cache.md (https://github.com/dmellok/tesserae/blob/main/docs/dev/frame-cache.md). It takes your schema as the base and reconciles it with the current deck cache.

Key resolutions:

• Additive frame_cache capability (schema/capacity_bytes/max_frames) alongside deck_cache; decks stay on their path in slice 1.
• position does double duty: stable order AND capacity priority, producer-supplied (album = playback order, deck = ring distance), so the shared cache never computes ordering. That is what keeps the deck's ring-distance policy out of the generic path.
• Neutral endpoints /collection + /collection/frame/<digest>, content-addressed exactly like /deck/frame/<digest>.
• version excludes version/cursor/next_cursor; per-device.
• Reporting is collection {id, version, cached, total, state}; an exact frame is a last-reported observation only, not a Current Screen contract.
• Interruption: a one-off push interrupts and auto-resumes at the next interval; reassigning the display's content stops the collection.

Take a look and flag anything. Once we are agreed I will build the server emitter + Gallery "Use as offline album" authoring, and you take firmware playback.

[dmellok]

The server side of slice one is built and pushed (edge v0.237.0), to the shape we pinned in docs/dev/frame-cache.md. Mapping it back to your seven boundary points:

1. Capability — additive frame_cache (schema, capacity_bytes, max_frames) advertised alongside deck_cache, current-state per heartbeat (not carried forward), and the /collection endpoint is offered only when frame_cache is present. deck_cache and /deck are untouched.
2. Shared core vs producer — the shared layer owns collection_id/kind/version and per-frame frame_id/position/digest/bytes/ttl_s/cache/url. The producer block is opaque to the cache; for an album it's just { album: { playback } }. Nothing in the sync path understands shuffle or graph distance.
3. Paging — single page in slice one (next_cursor: null), but total_frames, position, cursor/next_cursor are on the wire from day one, and version excludes cursor/next_cursor so a later multi-page walk isn't a contract change. cache is true for the lowest-position frames that fit both max_frames and capacity_bytes; the rest are cache: false, fetched on demand. Fill order is play-order (producer-owned), not deck ring-distance.
4. Playback — mode: sequential | shuffle, a proposed interval_s (server emits, firmware clamps), and repeat: loop | reshuffle | once. Shuffle-bag state stays firmware-local and resets on version change, as you described.
5. Truthful reporting — the /status envelope carries { id, kind, version } only; the doc's reporting block (cached/total/state, last-reported digest as an observation, never a live "current screen") is pinned. Honest caveat: the server does not yet consume that report or surface it in the Devices/Companion UI, that ingest + display is a follow-up, not in this slice.
6. Interruption — this is the one place the pinned doc took a position that differs from your lean: it currently says a one-off Image/Dashboard push interrupts and auto-resumes at the next interval, and reassigning content stops the collection. Since interruption is firmware-side behaviour (the server just keeps serving /frame and the envelope), this is your call to make. If you'd rather have explicit pause/stop for v1, it's a doc edit with no server change. Say which and I'll align the contract.
7. Quiet hours — not addressed in this slice. The album producer block carries no device-local allowed-window yet, so v1 playback would ignore server quiet hours. I'd fold a per-collection window into the same shape as the page-wake-cadence work rather than invent an album-specific one, but that's a later addition, flagging it as open rather than silently shipped.

Endpoints: GET /api/v1/device/<id>/collection (204 when nothing bound) and GET /api/v1/device/<id>/collection/frame/<digest> (ETag + immutable cache, 404 = stale manifest, re-sync). Authoring is live too: a Gallery folder gets a "Use as offline album" action to pick devices, fit, order, and playback. And you were right that warming is cheap: a folder renders through fit_to_panel + dither with no Chromium, so a few hundred photos is seconds of Pillow, not minutes of screenshots.

@bablokb, on the single push slot: this path sidesteps it. Album frames are pre-rendered per device into a content-addressed cache and served from /collection, independent of the live push slot, so several photos in an album don't contend the way two direct pushes would. Orphan cleanup is client-side by contract (the device deletes cached files the current manifest no longer references), which is close to the Frameo-style offline management you're describing, minus the manual-delete-when-full step.

That leaves firmware playback (your half), the report-ingest/Devices surface, real paging past 32, and the quiet-hours window. Want to settle 6 and 7 before you start on the parser?

[charmmmz]

Accepted for slice 1: I will implement and validate firmware with album folders of 32 images or fewer, matching the advertised max_frames: 32.

Recording one non-blocking follow-up for paging: the current server manifest emits every folder frame in one page and marks overflow entries cache: false. The firmware REST receive ceiling is currently 32 KiB, so a sufficiently large folder (roughly hundreds of entries, depending on JSON/path lengths) can truncate before parsing and prevent even the first 32 cached frames from syncing.

This does not block the agreed first slice or its <=32-image test matrix. Before large-album support, we should either page/cap the returned frame entries or move the firmware collection path to a bounded/streaming parser. I will keep this explicit in the firmware test boundary.

[charmmmz]

Thanks — confirmed from the firmware side:

1. We will keep the pinned interruption behavior for slice 1: a one-off Image/Dashboard push interrupts Album playback and it auto-resumes at the next interval. Reassigning or unbinding Rotation/Deck/Album stops it.
2. Quiet hours remain out of slice 1. Firmware will ignore server quiet hours for offline playback; we will track a device-local playback window as a later shared-cadence addition rather than introducing an Album-only shape.
3. Firmware implementation is now underway against the pinned contract, with the first validation scope limited to albums of 32 images or fewer.

No server-side changes are needed for these decisions. The large single-page manifest / 32 KiB receive ceiling noted below remains a non-blocking paging follow-up.

[dmellok]

Both decisions are recorded and the pinned doc already says what you confirmed, so nothing moved there: interruption stays "push interrupts, auto-resume at next interval; reassign or unbind stops", and quiet hours stay out of slice 1.

Two server-side changes since, both on edge v0.239.0:

The 32 KiB ceiling is closed rather than just noted. /collection is now paged: at most 64 frame entries per response, ?cursor=<next_cursor> continues, and next_cursor is bound to version exactly as pinned (a stale cursor returns a fresh first page). version and total_frames describe the whole collection on every page, so paging never reads as a content change. Because the page size exceeds the slice-1 max_frames, every cache: true frame still lands on page one: firmware that reads a single page sees the same cacheable set as before, and your <=32-image test matrix is unaffected. Server side of your follow-up is done; what remains for large albums is firmware walking the cursor chain when you get there.

The playback report is now consumed. A collection block on the /status body (id, version, cached/total, state) is validated, stored with its own received-at, carried across beats that omit it, and event-logged on state and version transitions (cached/total churn during a sync deliberately doesn't log). The Devices card shows it on the device's Status tab while the described album is still the bound one: album name, state, cached/total, and the report's age, never a claimed current frame, per the truthful-reporting section. Surfacing the same block through the Companion API is a device-view field, i.e. a contract addition, so I've left it out rather than shipping it unilaterally; happy to fold it into the next contract rev whenever you want it on the Display card.

docs/dev/frame-cache.md is updated to match both.