Skip to content

Commit ccfb536

Browse files
committed
File viewer: render images and PDFs inline
The viewer now shows images and PDFs as themselves instead of the binary-warning banner over lossy UTF-8 bytes. The mode picker offers the concrete detected kind (Image / PDF / Text) with no umbrella term, and "View as text" still drops to the raw byte view for any file. How it works (macOS, no Rust image decoder, WKWebView decodes natively): - `viewer_open` classifies the file by magic bytes (`classify_viewer_content`): raster (JPEG/PNG/GIF/WebP/BMP/TIFF), HEIC, conservative SVG, and PDF, local files only. A media open skips the text backend entirely (a `Media` no-op backend), mints a token, and returns `kind` + `mediaToken` + best-effort image dimensions. - Bytes reach the webview through a new token-guarded `cmdr-media://` async URI scheme, not the asset protocol. The URL carries a 128-bit CSPRNG token, never a path, so a hostile file's content can't name an arbitrary file (unknown token → 404). The handler serves `Content-Type` from magic bytes (never the extension), honors `Range` (206 + inclusive `Content-Range`, 416), and runs its own `spawn_blocking` + timeout → 504 off the IPC thread. The token is dropped at the single `close_session` choke point both teardown paths funnel through. - Frontend renders `<img>` (fit/zoom/pan, transparency checkerboard, EXIF-upright, loading/error states) and `<embed>` for PDF; every text-only effect and control is data-gated on the backend `kind`. The binary-warning banner is suppressed for rendered media by that same `kind`, so unrendered formats (RAW, AVIF, ICO, Office docs) still warn. CSP adds the `cmdr-media:` scheme to `img-src` + `object-src` (verified live by the Playwright spec). Covered by Rust unit tests (classifier matrix, range math, token 404), Vitest, and two Playwright E2E specs rendering a real image and PDF under CSP. Plan and two Opus review rounds: docs/specs/viewer-media-plan.md.
1 parent 13b7246 commit ccfb536

42 files changed

Lines changed: 2618 additions & 146 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

apps/desktop/src-tauri/src/commands/file_viewer.rs

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -38,6 +38,23 @@ pub async fn viewer_open(path: String, window_label: String) -> Result<ViewerOpe
3838
.await
3939
}
4040

41+
/// Opens a fresh, full **text** session for `path`, ignoring media classification.
42+
///
43+
/// Backs the viewer's "View as text" override: a media (Image/PDF) session isn't
44+
/// upgraded in place; the FE calls this, swaps to the returned text session, and closes
45+
/// the old one. Re-registers the window -> session link so the window-destroyed handler
46+
/// frees the new session.
47+
#[tauri::command]
48+
#[specta::specta]
49+
pub async fn viewer_open_as_text(path: String, window_label: String) -> Result<ViewerOpenResult, IpcError> {
50+
blocking_result_with_timeout(VIEWER_TIMEOUT, move || {
51+
let result = file_viewer::open_session_as_text(&path).map_err(|e| e.to_string())?;
52+
file_viewer::register_window_session(&window_label, &result.session_id);
53+
Ok(result)
54+
})
55+
.await
56+
}
57+
4158
/// Fetches a range of lines from a viewer session.
4259
///
4360
/// # Arguments

apps/desktop/src-tauri/src/file_system/volume/manager.rs

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -129,6 +129,19 @@ impl VolumeManager {
129129
.unwrap_or_default()
130130
}
131131

132+
/// Returns all registered volumes as (id, handle) pairs. Unlike [`list_volumes`]
133+
/// (which returns display names), this hands back the `Volume` handles so callers
134+
/// can inspect capabilities (`root`, `supports_local_fs_access`,
135+
/// `smb_connection_state`). Used by the file viewer's locality check.
136+
///
137+
/// [`list_volumes`]: Self::list_volumes
138+
pub fn list_volumes_with_handles(&self) -> Vec<(String, Arc<dyn Volume>)> {
139+
self.volumes
140+
.read()
141+
.map(|volumes| volumes.iter().map(|(id, vol)| (id.clone(), vol.clone())).collect())
142+
.unwrap_or_default()
143+
}
144+
132145
/// Returns the number of registered volumes.
133146
pub fn count(&self) -> usize {
134147
self.volumes.read().map(|v| v.len()).unwrap_or(0)

apps/desktop/src-tauri/src/file_viewer/CLAUDE.md

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,9 @@ open-viewer helper and binary-warning classifier.
1616
`search_matcher.rs`: `Matcher`, huge-line chunking. `watcher.rs`: shared tail-mode watcher singleton.
1717
- Backend selection: `< 1MB` -> `FullLoad`; else `ByteSeek` (instant open) with a background `LineIndex` build that
1818
upgrades when ready. Full per-file roles, Tauri commands, and decision rationale are in [DETAILS.md](DETAILS.md).
19+
- Media (Image/PDF): `content_kind.rs` (pure `classify_viewer_content`), `media.rs` (the `cmdr-media://` token map),
20+
`media_protocol.rs` (the async scheme handler), `media_backend.rs` (no-op `MediaBackend`). See [DETAILS.md](DETAILS.md)
21+
§ "Media rendering".
1922

2023
## Must-knows
2124

@@ -35,7 +38,11 @@ open-viewer helper and binary-warning classifier.
3538
rebuilds replace the backend without blocking the `get_lines` read path. Each backend is immutable.
3639
- **`SESSIONS` is freed on BOTH close paths.** The titlebar-X path never fires `viewer_close`; it's covered by a
3740
`WindowEvent::Destroyed` branch in `lib.rs::on_window_event` for `viewer-*` labels (via the `WINDOW_TO_SESSION` map).
38-
Without it, titlebar-closed viewers leak their session until app quit.
41+
Without it, titlebar-closed viewers leak their session until app quit. The `cmdr-media://` token is dropped at this
42+
same choke point (`media::drop_token`); don't drop it elsewhere, or a closed viewer leaks a live token mapping a path.
43+
The scheme handler serves `Content-Type` from stored magic bytes (never the extension), does its OWN
44+
`spawn_blocking` + timeout (504, not `blocking_with_timeout`), and 404s an unknown token. See
45+
[DETAILS.md](DETAILS.md) § "Media rendering".
3946
- **`search_cancel` must not null `session.search`**: the cancel flag is what the search thread writes `Cancelled` into;
4047
nulling first lands the write in a dropped state and `search_poll` returns `Idle`. See [DETAILS.md](DETAILS.md).
4148
- **`SearchMatch.column` / `.length` are UTF-16 code units** (match JS `String.substring()`), avoiding a class of

apps/desktop/src-tauri/src/file_viewer/DETAILS.md

Lines changed: 46 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -30,6 +30,52 @@ Frontend counterpart: [`apps/desktop/src/routes/viewer/CLAUDE.md`](../../../src/
3030
- `*_test.rs`: unit tests for each backend: UTF-8 edge cases, search highlighting, checkpoint math, range reads,
3131
cancellation, encoding detection, UTF-16 newline scanning, encoding-switch rebuild + drain-and-swap
3232

33+
## Media rendering (Image / PDF)
34+
35+
The viewer renders images and PDFs inline instead of showing the binary warning. The backend half:
36+
37+
- `content_kind.rs`: pure `classify_viewer_content(head, ext, is_local) -> ViewerContentKind { Text, Image, Pdf }`.
38+
Magic bytes decide (JPEG/PNG/GIF/WebP/BMP/TIFF/HEIC/PDF); the extension is a tiebreaker only, and only for SVG (an
39+
`.svg` ext AND an `<svg` root after BOM/prolog/comments/DOCTYPE). Non-local files always classify `Text` (v1 scopes
40+
media to local POSIX volumes; MTP has no POSIX path, SMB can block). `media_mime(head, kind)` re-sniffs the magic to
41+
pick the served `Content-Type`.
42+
- `media.rs`: the `cmdr-media://` capability-token registry. `mint_token(entry)` stores
43+
`token -> { canonical_path, kind, mime }` (128-bit CSPRNG via `rand`, 32 hex chars) and returns the token;
44+
`resolve_token` / `drop_token` round it out. `read_image_dimensions` reads header-only pixel dimensions best-effort
45+
(the `image` crate; `None` for HEIC/SVG/errors).
46+
- `media_protocol.rs`: the async URI scheme handler. Pure helpers (`parse_token_from_uri`, `resolve_range`,
47+
`build_response`) are unit-tested; `handle_request` is the thin Tauri shell. It resolves the token (unknown -> 404),
48+
then runs its OWN `tauri::async_runtime::spawn_blocking` + `tokio::time::timeout` (expiry -> 504; deliberately NOT
49+
`blocking_with_timeout`, which returns an `IpcError`, not an HTTP response). Honors `Range`: 206 with inclusive
50+
`Content-Range`/`Accept-Ranges`, end clamped to `size-1`, 416 when unsatisfiable, 200 when no range. `Content-Type`
51+
comes from the token's stored magic-byte MIME, never the extension. Registered once in the `lib.rs` builder chain via
52+
`register_asynchronous_uri_scheme_protocol(media_protocol::SCHEME, ...)`, before any window exists (correct: `viewer-*`
53+
windows are lazy and inherit the app-wide scheme).
54+
- `media_backend.rs`: `MediaBackend`, a no-op `FileViewerBackend` so a media session can fill the non-optional
55+
`backend` field without a text backend. Every text-shaped call returns empty/zero.
56+
57+
Open flow: `open_session` reads the head, classifies, and for a media kind calls `open_media_session`, which mints the
58+
token, reads dimensions best-effort (header-only, must not extend the open past the metadata read), installs a
59+
`MediaBackend`, and returns a `ViewerOpenResult` with `kind` + `media_token` + `media_dimensions` and empty text
60+
fields. Media sessions spawn no watcher and no LineIndex upgrade. `open_session_as_text` (behind the
61+
`viewer_open_as_text` IPC) forces the text path for the "View as text" override; it returns a fresh full text session
62+
the FE swaps to (no in-place upgrade).
63+
64+
**Token lifetime == session lifetime.** `close_session` (the single choke point both teardown paths funnel through, the
65+
`viewer_close` IPC and the `WindowEvent::Destroyed` net via `close_session_for_window`) calls `media::drop_token`, so a
66+
closed-window viewer can't leave a live token mapping a real path. An unknown token is a 404, covering both
67+
never-existed and already-dropped.
68+
69+
CSP: `tauri.conf.json` adds `cmdr-media:` to `img-src` and `object-src`. On macOS WKWebView a Tauri custom scheme
70+
surfaces as `cmdr-media://localhost/...`, so the CSP token is the scheme `cmdr-media:` (verified by `viewer-media.spec.ts`
71+
E2E rendering a real image + PDF under this CSP with no violation, on macOS 26.5.1 / WKWebView 25F80, 2026-06-14;
72+
resource origin can differ by Tauri version / platform, e.g. `http://cmdr-media.localhost` on Windows). Access is gated
73+
by the per-open token in the handler, not by CSP, so the app-wide allowance is acceptable.
74+
75+
FDA: the handler `File::open`s a real path off an IPC-minted token, inheriting the viewer's existing assumption that a
76+
viewer only opens after the user picked the file (so FDA is already decided). This is NOT a new pre-gate read path
77+
(`fda_gate.rs`); a stray TCC denial reading bytes here is a real access failure, not a scheme bug.
78+
3379
## Backend selection logic
3480

3581
```rust
Lines changed: 205 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,205 @@
1+
//! Content classification for the file viewer: decide whether a file should render
2+
//! as inline media (an image, a PDF) or fall back to the text pipeline.
3+
//!
4+
//! The decision is driven by **magic bytes**, never the extension (a `.jpg` that is
5+
//! really a PDF/HTML polyglot must never be handed to an `<embed>`/`<iframe>` that
6+
//! could execute it). The extension is a tiebreaker only, and the SVG case is the
7+
//! sole place where the extension is load-bearing (see [`classify_viewer_content`]).
8+
//!
9+
//! Only **local** files are eligible for media rendering in v1. MTP has no POSIX path
10+
//! to serve, and SMB paths can block; both stay on the text pipeline. The caller
11+
//! decides locality and passes it in as `is_local`.
12+
13+
use serde::Serialize;
14+
15+
/// What the viewer should render a file as. The frontend branches on this:
16+
/// `Image` -> `<img>`, `Pdf` -> `<embed>`, `Text` -> the line pipeline.
17+
///
18+
/// "Media" survives only as an internal code word (the `cmdr-media://` scheme and
19+
/// this enum); users never see it. Future kinds (Markdown, Html) slot in here.
20+
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, specta::Type)]
21+
#[serde(rename_all = "camelCase")]
22+
pub enum ViewerContentKind {
23+
Text,
24+
Image,
25+
Pdf,
26+
}
27+
28+
/// Number of head bytes the caller should read and pass to [`classify_viewer_content`].
29+
/// Enough to cover every magic-byte signature plus the conservative SVG sniff (which
30+
/// skips a BOM, an XML prolog, comments, and a DOCTYPE before the `<svg` root).
31+
pub const CLASSIFY_HEAD_LEN: usize = 1024;
32+
33+
/// Classifies a file as [`ViewerContentKind`] from its head bytes, extension, and
34+
/// locality. Pure: no I/O.
35+
///
36+
/// - **Magic bytes decide** the kind (and, downstream, the served `Content-Type`):
37+
/// JPEG/PNG/GIF/WebP/BMP/TIFF/HEIC -> `Image`, `%PDF-` -> `Pdf`.
38+
/// - **SVG is the one extension-gated case**: classified `Image` only when `ext` is
39+
/// `svg` AND the first meaningful token (after a BOM, XML prolog, comments, and a
40+
/// DOCTYPE) is an `<svg` root. This avoids false-positiving an HTML file that merely
41+
/// embeds an inline `<svg>`.
42+
/// - **Non-local -> always `Text`.** v1 scopes media rendering to local files.
43+
pub fn classify_viewer_content(head: &[u8], ext: Option<&str>, is_local: bool) -> ViewerContentKind {
44+
if !is_local {
45+
return ViewerContentKind::Text;
46+
}
47+
48+
if let Some(kind) = classify_by_magic(head) {
49+
return kind;
50+
}
51+
52+
// SVG is text-shaped, so it has no binary magic. Only treat it as an image when
53+
// the extension claims SVG and the content actually opens with an `<svg` root.
54+
if ext.is_some_and(|e| e.eq_ignore_ascii_case("svg")) && looks_like_svg_root(head) {
55+
return ViewerContentKind::Image;
56+
}
57+
58+
ViewerContentKind::Text
59+
}
60+
61+
/// The MIME type to serve for a media kind, derived from the same magic bytes the
62+
/// classifier used. Returns `None` for `Text` (text never flows through the scheme).
63+
///
64+
/// For images the exact subtype matters for WKWebView decode hints, so we re-sniff
65+
/// the magic here rather than carry the subtype through the classifier. SVG has no
66+
/// magic, so it's the extension-driven fallback when no raster magic matches.
67+
pub fn media_mime(head: &[u8], kind: ViewerContentKind) -> Option<&'static str> {
68+
match kind {
69+
ViewerContentKind::Text => None,
70+
ViewerContentKind::Pdf => Some("application/pdf"),
71+
ViewerContentKind::Image => Some(image_mime(head)),
72+
}
73+
}
74+
75+
/// The image subtype for `head`, falling back to `image/svg+xml` when no raster
76+
/// magic matches (the classifier only ever reaches `Image` for raster magic or a
77+
/// confirmed SVG root, so the fallback is exactly the SVG case).
78+
fn image_mime(head: &[u8]) -> &'static str {
79+
match classify_by_magic(head) {
80+
Some(ViewerContentKind::Image) => raster_image_mime(head).unwrap_or("application/octet-stream"),
81+
_ => "image/svg+xml",
82+
}
83+
}
84+
85+
/// Magic-byte classification for the closed set of formats WKWebView decodes natively.
86+
/// Returns `None` for anything without a recognized binary signature (text, SVG, etc.).
87+
fn classify_by_magic(head: &[u8]) -> Option<ViewerContentKind> {
88+
if head.starts_with(b"%PDF-") {
89+
return Some(ViewerContentKind::Pdf);
90+
}
91+
if raster_image_mime(head).is_some() {
92+
return Some(ViewerContentKind::Image);
93+
}
94+
None
95+
}
96+
97+
/// Returns the MIME type if `head` starts with a known raster image signature, else
98+
/// `None`. The single source of truth for "is this raster image magic?".
99+
fn raster_image_mime(head: &[u8]) -> Option<&'static str> {
100+
// JPEG: FF D8 FF
101+
if head.starts_with(&[0xFF, 0xD8, 0xFF]) {
102+
return Some("image/jpeg");
103+
}
104+
// PNG: 89 50 4E 47 0D 0A 1A 0A
105+
if head.starts_with(&[0x89, 0x50, 0x4E, 0x47, 0x0D, 0x0A, 0x1A, 0x0A]) {
106+
return Some("image/png");
107+
}
108+
// GIF: "GIF8" (covers GIF87a and GIF89a)
109+
if head.starts_with(b"GIF8") {
110+
return Some("image/gif");
111+
}
112+
// WebP: "RIFF" .... "WEBP"
113+
if head.len() >= 12 && head.starts_with(b"RIFF") && &head[8..12] == b"WEBP" {
114+
return Some("image/webp");
115+
}
116+
// BMP: "BM"
117+
if head.starts_with(b"BM") {
118+
return Some("image/bmp");
119+
}
120+
// TIFF: "II*\0" (little-endian) or "MM\0*" (big-endian)
121+
if head.starts_with(&[0x49, 0x49, 0x2A, 0x00]) || head.starts_with(&[0x4D, 0x4D, 0x00, 0x2A]) {
122+
return Some("image/tiff");
123+
}
124+
// HEIC: ISO-BMFF `ftyp` box at offset 4, with a HEIF/HEIC brand.
125+
if is_heic(head) {
126+
return Some("image/heic");
127+
}
128+
None
129+
}
130+
131+
/// HEIC detection: an ISO base media file (`ftyp` box at offset 4) whose major brand
132+
/// is one of the HEIF/HEIC family. We check the major brand (bytes 8..12); the brand
133+
/// list mirrors what ImageIO treats as HEIC-family still images.
134+
fn is_heic(head: &[u8]) -> bool {
135+
if head.len() < 12 || &head[4..8] != b"ftyp" {
136+
return false;
137+
}
138+
matches!(
139+
&head[8..12],
140+
b"heic" | b"heix" | b"mif1" | b"msf1" | b"heim" | b"heis" | b"hevc" | b"hevx"
141+
)
142+
}
143+
144+
/// Conservative SVG sniff: returns true when the first meaningful token, after
145+
/// skipping a UTF-8 BOM, leading whitespace, an XML prolog (`<?xml ... ?>`), any
146+
/// number of comments (`<!-- ... -->`), and a DOCTYPE (`<!DOCTYPE ... >`), is an
147+
/// `<svg` element open tag.
148+
fn looks_like_svg_root(head: &[u8]) -> bool {
149+
let mut rest = head;
150+
// Strip a UTF-8 BOM if present.
151+
if let Some(stripped) = rest.strip_prefix(&[0xEF, 0xBB, 0xBF]) {
152+
rest = stripped;
153+
}
154+
loop {
155+
rest = skip_ascii_whitespace(rest);
156+
if rest.starts_with(b"<?") {
157+
// XML prolog / processing instruction: skip to the closing "?>".
158+
match find_subslice(rest, b"?>") {
159+
Some(i) => rest = &rest[i + 2..],
160+
None => return false,
161+
}
162+
} else if rest.starts_with(b"<!--") {
163+
// Comment: skip to the closing "-->".
164+
match find_subslice(&rest[4..], b"-->") {
165+
Some(i) => rest = &rest[4 + i + 3..],
166+
None => return false,
167+
}
168+
} else if rest.starts_with(b"<!") {
169+
// DOCTYPE (or other declaration): skip to the next ">".
170+
match find_subslice(rest, b">") {
171+
Some(i) => rest = &rest[i + 1..],
172+
None => return false,
173+
}
174+
} else {
175+
break;
176+
}
177+
}
178+
// The root element must be `<svg`, followed by whitespace, `>`, or `/`.
179+
let Some(after) = rest.strip_prefix(b"<svg") else {
180+
return false;
181+
};
182+
match after.first() {
183+
None => true, // truncated head right after "<svg": accept (the head cap clipped it)
184+
Some(b) => b.is_ascii_whitespace() || *b == b'>' || *b == b'/',
185+
}
186+
}
187+
188+
fn skip_ascii_whitespace(mut s: &[u8]) -> &[u8] {
189+
while let Some((first, rest)) = s.split_first() {
190+
if first.is_ascii_whitespace() {
191+
s = rest;
192+
} else {
193+
break;
194+
}
195+
}
196+
s
197+
}
198+
199+
/// Returns the index of the first occurrence of `needle` in `haystack`, or `None`.
200+
fn find_subslice(haystack: &[u8], needle: &[u8]) -> Option<usize> {
201+
if needle.is_empty() || haystack.len() < needle.len() {
202+
return None;
203+
}
204+
haystack.windows(needle.len()).position(|w| w == needle)
205+
}

0 commit comments

Comments
 (0)