Skip to content

Media and PDF

codingncaffeine edited this page Nov 20, 2026 · 2 revisions

Media and PDF

Video, sound and PDF are read by code in the repository, like everything else (The Pledge): the containers, the decoders and the PDF reader are Sashfold's own, and each is held to another implementation's reading of the same files. Where the graphics driver can decode a video's pictures it is asked to; where it cannot, a decoder of our own does.

Video and sound

Containers WebM; MP4, plain (sample tables, moov first or last, edit lists) and fragmented as Media Source Extensions and DASH send it; WAVE
Video VP9, by the graphics driver (Vulkan Video or VA-API). H.264, by the graphics driver (8-bit 4:2:0) or by a software decoder of our own: Baseline, Main and High, High 10, High 4:2:2 and High 4:4:4 Predictive with their intra profiles, progressive and interlaced (field pictures and MBAFF), lossless coding included
Sound Opus; AAC of our own: AAC-LC, spectral band replication (HE-AAC) and parametric stereo (HE-AAC v2)
Playing <video> and <audio> with a plain src (fetched in ranges while it plays) or fed by Media Source Extensions (the WebM and ISO BMFF byte streams); canPlayType and isTypeSupported answer by codec profile, claiming exactly what decodes
Protected media Encrypted Media Extensions with the Clear Key system, for MP4 (cenc, cbcs) and WebM

How each piece is checked:

  • MP4: packet for packet against ffprobe's reading of the same files.
  • H.264: the software decoder's pictures are identical, sample for sample, to ffmpeg's on 48 clips and to the reconstructed pictures of 127 ITU-T H.264.1 conformance streams. The driver's path was held to the same pictures. Pictures deeper than 8 bits are dithered down for the screen. A picture coded as several slices is decoded on several threads: 430 frames a second at 1080p on four, where one after another gave 301.
  • AAC: against ffmpeg's output and the ISO conformance streams. AAC-LC stands at 79 dB from ffmpeg; all 53 ISO streams for spectral band replication at 101 to 119 dB; parametric stereo at 113 dB.
  • Clear Key: 122 of the 123 Clear Key tests of the Web Platform Tests.

The main menu has a switch for hardware video decoding (kept in settings.json): with it off, every picture is decoded in software. A stream the driver cannot take (an interlaced one, or a driver fault met while playing) goes to the software decoder by itself, and playback carries on from where it was.

Sound is played by speaking the sound server's own protocol. No test of the engine can make a sound: each runs against a sound server that is not there.

What is still owed is on Not written yet: slice groups and HDR tone mapping, two parts of parametric stereo, and a content decryption module other than Clear Key, which the large subscription services require.

PDF

A PDF opens in the tab, in a reader of our own. It is held to what the readers built into the shipping browsers do, and each step was checked page by page against another reader's render of the same file.

  • The file: objects, cross-reference tables and streams (repaired when broken), every standard filter, and encrypted files (RC4, AES-128 and AES-256, with a password prompt).
  • The drawing: paths and clips; every color space, with ICC profiles applied (within one or two steps in 255 of a color-management library's result); pictures and masks, including CCITT fax, JBIG2 and JPEG 2000; shadings, patterns, transparency groups and soft masks.
  • The text: Type 1, TrueType, CFF, Type 3 and CID fonts with their encodings and CMaps; the standard 14 fonts from the system's own; ToUnicode maps, so copied text is the text; glyph edges computed by exact pixel area.
  • The tab: pages drawn lazily and off the page's thread, zoom to 6400% with sharp tiles, fit, rotate, the outline, thumbnails, find, selection, links, download and #page=. The same reader shows a PDF in an <iframe>, an <embed> and an <object>.
  • Forms: fields are filled in the tab, with text in any script saved in an embedded subset font, comb fields, and the form's own calculation and format scripts run in a sandbox. Signatures in the file are left intact. A filled form is saved and printed so that every other reader shows the values.
  • The rest: annotations and printing.

Owed: vertical forms of CJK glyphs, exact-area edges for fills and strokes, and shaped text (Hindi and the like) in saved fields.

Clone this wiki locally