Proposal: native Gallery management in Tesserae Companion

[charmmmz]

Summary

I would like to propose a native Gallery surface in Tesserae Companion: browse the server's Gallery folders, upload photos from the iOS photo library, and hand an existing Gallery image into the app's current Send flow.

The phone is the natural place where new photos enter the system, while Tesserae should remain the storage, validation, rendering, and device-capability source of truth. This would complement the Web admin rather than embed it or duplicate its filesystem-oriented controls.

This is separate from native Lineups authoring (#203) and builds on the Companion contract direction from #147. Offline Album support from #177 would be a later Gallery-folder action, not the definition of Gallery itself.

Product boundary

There are three related but distinct concepts:

• Gallery is the reusable server-side photo library. It should be useful for every display: photos can be uploaded, browsed, used by Gallery widgets, or sent through the normal online render path.
• Offline Album is an optional playback mode for a Gallery folder on a device that currently advertises the frame_cache capability.
• Lineups schedule Dashboards/content. Gallery photos should not become one generated Dashboard step per image.

In particular, a device such as PicPak can still benefit from Gallery browsing, uploads, and online Send even though its current community firmware does not implement Offline Album playback.

Proposed native workflow

Folder list

A Gallery destination in Companion would show:

• folder name;
• cover thumbnail;
• image count;
• internal or external/read-only state;
• optional Offline Album summary when one exists.

Folder detail

• thumbnail grid;
• native multi-selection from Photos using PhotosPicker;
• per-file upload progress, retry, and clear partial-failure results;
• tap an image to preview it and reuse the existing Send destination/framing flow;
• create a new internal folder;
• external folders remain read-only and do not expose their host filesystem path.

This gives the first slice a focused job: browse, create, upload, and send. Delete, rename, move/copy, camera capture, and Share Sheet “Save to Gallery” can follow once the read/upload contract is proven.

Companion API direction

I do not think the app should call the existing /plugins/picture_gallery/... browser routes. Those routes use browser forms, redirects, filenames, and Web session behavior rather than a stable app contract.

A capability-gated /api/app/v1 surface could expose operations along these lines (names illustrative):

GET  /api/app/v1/gallery/folders
POST /api/app/v1/gallery/folders
GET  /api/app/v1/gallery/folders/{folder_id}
POST /api/app/v1/gallery/folders/{folder_id}/images
GET  /api/app/v1/gallery/images/{image_id}/thumbnail
GET  /api/app/v1/gallery/images/{image_id}/content

Suggested contract properties:

• opaque/stable folder and image identifiers rather than filesystem paths as identity;
• folder kind: internal | external and server-computed writable state;
• image dimensions/orientation, thumbnail URL, content URL, and an ETag/digest where useful;
• advertised upload limits: accepted formats, maximum bytes, and maximum batch size;
• explicit per-file results for a partially successful batch;
• idempotency for mutations so retrying an interrupted upload does not create accidental duplicates;
• capability and scope gating, for example gallery:read and gallery:write (exact names open to the contract owner);
• all filename normalization, decoding/validation, duplicate policy, metadata/privacy policy, thumbnail generation, and storage remain server-owned.

The iOS app would own Photos selection, upload queue/progress, retry UI, previews, and navigation. It would not send Photos-library identifiers or reason about server paths.

Offline Album as a follow-up

Once Gallery management is established, a folder can gain Play offline on a display, mapping to the existing Album model:

• target displays;
• Fit / Fill;
• sequential / shuffle;
• interval;
• loop / reshuffle / hold last;
• later, order and sync status.

Eligibility must be based on the device's runtime capability, not its model name and not a hard-coded assumption that storage must be an SD card. A future client with suitable internal flash could implement the same contract.

I suggest the server expose a computed eligibility state for each target rather than making Companion interpret raw heartbeat details:

• supported — the latest usable heartbeat advertises frame_cache, optionally with capacity and max_frames;
• unsupported — a current/recent heartbeat explicitly lacks it;
• unknown — there is no usable current capability report, for example a device that has not checked in.

The same rule should be reflected in the Web Gallery UI. Today the Offline Album form lists every registered device whose kind resolves, so an unsupported PicPak can be checked and saved even though it will never receive a collection. The UI should not present those devices as equally eligible; unsupported targets should be disabled with a reason, while unknown/offline targets should be clearly distinguished from confirmed support.

That Web correction can remain a small independent server PR; it does not need to wait for Companion Gallery implementation.

Proposed sequencing

1. Agree the native workflow, ownership boundary, capability/scopes, and first contract shape.
2. Land a contract-only Gallery addition with fixtures and compatibility tests.
3. Implement the server adapter over the existing Picture Gallery storage.
4. Add Companion folder browsing, creation, multi-photo upload, and Send hand-off.
5. Add destructive management operations and Share Sheet import separately.
6. Add Album API/configuration and truthful device eligibility/sync state.
7. Consider a later Gallery/Album adapter into Lineups without generating one Dashboard per photo.

Each step can be reviewed and shipped independently. The MVP does not require firmware changes and does not depend on Offline Album support.

Questions

1. Does a native Gallery surface fit Companion's scope as a high-value mobile workflow?
2. For the first slice, is browse/create/upload/Send the right boundary, leaving delete and reorganization out?
3. Would you prefer one upload request per image or a multipart batch with per-file outcomes?
4. Should upload normalize images and strip privacy-sensitive EXIF (especially GPS) by default while preserving orientation and colour profile?
5. Should Album target eligibility be returned as part of Gallery/Album responses, or reuse a more general server-computed device action/applicability model?

[dmellok]

Green light on the direction and on the first slice as scoped.

1. It fits. The phone is where photos originate, and the current alternative is
AirDropping to a machine that can reach the Web admin. Agreed that the app should
not drive /plugins/picture_gallery/...; those routes are browser forms with
redirects and filename identity, and they'll keep changing shape as the plugin does.

2. Browse, create, upload, Send is the right boundary. Delete, rename, and
move stay out of v1. Those are the operations where a mobile mis-tap is unrecoverable
and where I'd want an operator-visible grant first (see scopes below).

3. One request per image, not a multipart batch.
Per-file progress and retry come for free, a failed photo doesn't complicate the
success of the other nine, and there's no partial-batch result object to specify or
version. The app owns the queue and its own concurrency. Send an idempotency key per
image so an interrupted upload retried on reconnect resolves to the same record
instead of a duplicate. The batch limit then becomes advice the server advertises
rather than a request shape it has to parse.

4. Yes: strip GPS by default, and not as a toggle in v1.
All location EXIF removed server-side, orientation baked into the pixels rather than
preserved as a tag the renderers then disagree about, ICC profile preserved. If
someone later wants their coordinates kept, that's a considered addition with a
visible setting, not a default.

5. Reuse a general device-capability model, not a Gallery-local one.
The tri-state you describe (supported / unsupported / unknown, derived from the
latest usable heartbeat rather than the model name) is the same question Lineups and
the Web forms need to answer. I'd rather it be a device-level computed field that
Gallery and Album responses carry through than a Gallery-specific concept that
something else has to reimplement. The "not necessarily an SD card" point is correct
and I'll write the eligibility rule against the advertised capability only.

On the Web form: you're right and it's worse than described. _bindable_devices
in the Picture Gallery plugin filters on nothing but "the kind resolves", so every
registered device is offered and an unsupported one saves cleanly and silently never
receives anything. I'll take that as a separate small server PR now; it doesn't need
to wait for any of the above.

Scopes. gallery:read and gallery:write both go in the default pairing set:
uploading a photo is the same class of action as media:write, which paired clients
already have. The destructive second slice gets its own scope in the optional set,
so delete and reorganize arrive only when an operator grants them, the way
lineups:write does. That also gives the app a clean way to know whether to show
those controls at all, via GET /session.

Sequencing. Your list is the order I'd want. Contract-only PR with fixtures and
compatibility tests first, then the adapter over the existing Picture Gallery
storage, then the app surface. Agreed that a Gallery/Album adapter into Lineups
should not generate one Dashboard per photo; if that ever happens it's an album step
that owns its own playback, not N steps.

[charmmmz]

Thanks — I implemented the accepted contract-first slice and published it as a stacked pair in Companion:

1. #7 — Apple Health summary contract targets main and defines contract 0.10.
2. #8 — native Gallery management contract targets the Hardware compatibility reports #7 branch and advances the shared OpenAPI/docs to contract 0.11.

Please merge #7 first. Once it lands, I will retarget #8 to main; the Gallery diff will then remain the independent second PR.

#8 follows your decisions here: browse/create/upload/Send only, one idempotent request per image, server-owned GPS removal/orientation normalization/ICC preservation, default gallery:read + gallery:write, external folders read-only, and general device-level frame_cache tri-state support from the latest usable heartbeat. It remains contract-only: no production adapter, iOS Gallery surface, destructive actions, or Offline Album authoring yet.

[dmellok]

The server side of the first slice is built and on edge in v0.300.0. Nothing
is blocked on your stack; #7 and #8 are yours to merge in whatever order suits
you, and I'll re-vendor once they're on main.

What's implemented

Six routes under /api/app/v1/gallery/, matching the paths and shapes in
contract 0.11: list folders, create an internal folder, read a folder, upload
one image, thumbnail, content. Plus:

• gallery:read and gallery:write in the pairing set, labelled in Settings
  as "Browse your photo Gallery" and "Add photos to your Gallery" so an
  operator sees capabilities rather than scope strings. No destructive scope
  yet; it lands with the routes it guards.
• capability_support on every device in GET /devices, with frame_cache
  as the only key so far. Computed from the display's own heartbeats, never
  from its kind.
• The Web album form now disables displays that reported no frame cache and
  says why, marks ones that haven't checked in as unconfirmed, and drops an
  ineligible target on submit rather than saving a binding that can't be
  delivered.
• One image per request with a per-image Idempotency-Key, server-owned GPS
  removal, orientation baked into the pixels, ICC preserved, external folders
  read-only.

Uploads deliberately create no Job and no History entry. Nothing was sent to a
display, and an upload showing up in the activity feed as a push would be a
lie about what happened.

Four things worth a look before 0.12

1. GalleryImageContentType is doing two jobs. It types both
   limits.gallery_image_content_types (what upload accepts) and
   GalleryImage.content_type (what's stored). The server accepts HEIC/HEIF
   and transcodes to JPEG, because nothing else in Tesserae reads HEIF and a
   stored .heic would be a file the widgets, renderers and thumbnailer all
   skip. So image/heic is a legal upload type and can never be a legal
   stored type. Two enums would say that; one can't.

2. The stored-type enum is missing image/gif and image/bmp. The
   Gallery has accepted both through the Web UI since it was ported, and there
   are existing folders with them in. Rather than serve a media type your
   decoder doesn't know, I skip those files: they aren't listed and they don't
   count toward image_count. Happy either way, but the client should know
   that a folder can show fewer images than the Web UI shows for it until this
   is settled.

3. GalleryFolder.name is the normalized storage name. POST with
   {"name": "Summer 2026!"} creates summer-2026 and returns that, so the
   app and the Web UI name the same folder identically. Worth stating in the
   description, since a client could reasonably expect its requested display
   name back.

4. One extra reason_code: stale_heartbeat. Your fixture covers
   not_advertised and no_usable_heartbeat. I also need the case where a
   display did report, but long enough ago that a current-state capability
   read from it would be a guess. That's unknown, not unsupported, and it
   carries a non-null observed_at so an operator can see how old the
   evidence is. The threshold tracks each display's own poll cadence rather
   than a fixed window, so a panel that sleeps for half an hour isn't stale
   after five minutes. Generic-fallback handling covers it, but the code is
   worth adding to the localized set.

On sequencing: your list still looks right to me. Contract, adapter, app
surface, then destructive operations and Album authoring separately.

[charmmmz]

Update: #7 and #8 are now merged into Companion main in that order.

I also opened #9 — Gallery contract 0.12 refinements for all four implementation notes:

• separate accepted upload and returned stored-media schemas, while retaining the 0.11 schema name as a deprecated alias;
• HEIC/HEIF upload → stored JPEG;
• existing GIF/BMP files as stored/browseable types, without advertising them as accepted Companion uploads;
• authoritative normalized internal-folder names;
• stale_heartbeat fixture and poll-cadence-owned semantics.

Local contract, fixture-backed Swift, and simulator-build validation pass. I will merge #9 when its CI is green, after which the vendored contract can move directly to 0.12.

[charmmmz]

Final contract update: #9 is green and merged. Companion main now carries contract 0.12 at merge commit 8a9b492, so the server can re-vendor and remove the 0.11 transcription/errata. I am moving on to the capability-gated native Gallery app surface next; destructive actions and Offline Album authoring remain separate.

[dmellok]

Correcting point 2 in my last comment, since I've changed what the server
does rather than waiting on the enum.

I said BMP and GIF files were skipped, and that a folder could show fewer
images over the API than in the Web UI. That's no longer true as of v0.300.3.
Those files are now served as a full-size PNG rendition, generated on first
request and cached beside the thumbnails, with the stored file left alone.
content_type is image/png, name carries the .png extension so a saved
download isn't mislabelled, and bytes and etag describe the rendition
rather than the source. Dimensions are unchanged.

So there's nothing for the contract to do here after all: everything the API
lists is already inside GalleryImageContentType. Point 1 still stands, and
is the stronger version of the same observation: accepted-upload types and
stored-image types are different sets, and one enum can't say that. Points 3
and 4 are unchanged.

Worth knowing on the client side: upload stays limited to the five declared
types, so BMP is readable over the API but not writable to it.

[dmellok]

Vendored contract 0.12 at 8a9b492 and emptied the local errata file; all
twelve entries (Lineups features and scopes, the session read flag,
forbidden and precondition_failed, and the whole transcribed Gallery
block) are published now, so our responses validate against your shapes
rather than my copies of them. stale_heartbeat is bound to the fixture,
and the new clamp case in the framing fixture runs against the real
resolver. Edge v0.300.4.

Two notes on how the server lands against 0.12, neither needing a contract
change:

GIF and BMP arrive as image/png. After my last comment I changed the
behaviour rather than leaving those files hidden: a stored .gif or .bmp
is served as a cached PNG rendition, generated on first read beside the
thumbnails. The stored file is never rewritten, but content_type is
image/png, name gains .png so a saved download isn't mislabelled, and
bytes/etag describe the rendition. created_at stays the source file's.
Animated GIFs give their first frame. So GalleryStoredContentType's
image/gif and image/bmp are legal values this server never returns, and
a client shouldn't infer the stored type from the filename. Upload still
accepts only the five source types.

image_count on the folder list can exceed the folder detail's image
array. The list counts directory entries, because opening every image in
every folder to answer a list would be slow; the detail endpoint counts what
it actually decoded, so a truncated file is counted in one and dropped from
the other. Treat the detail response as authoritative for a grid.

Server side is ready for the native Gallery surface whenever you are.
Destructive operations and Album authoring stay where we left them, each
with its own scope when the routes exist.