yom-format

YOM (.yom) is a read-only, single-file package format for scanned books — AVIF page images, thumbnails, OCR layout, and full-text search data behind a front-mounted FlatBuffers index, so a reader can open a book and jump to any page by byte range without ever loading the whole file.

日本語版は README.ja.md をどうぞ。

┌─────────────────────────────────────────────┐
│ Header — 4096 B fixed (magic "YOM1",        │
│ version, index location, SHA-256 hashes)    │
├─────────────────────────────────────────────┤
│ Main index — FlatBuffers, uncompressed:     │
│ book metadata, page table, blob table,      │
│ search terms, PDF-export outline            │
├─────────────────────────────────────────────┤
│ Blobs — AVIF pages & thumbnails,            │
│ OCR layout, normalized text,                │
│ search dictionary & postings                │
├─────────────────────────────────────────────┤
│ Footer — 96 B mirror of the header          │
└─────────────────────────────────────────────┘

YOM was built for Yomu, an iOS reader for heavy scanned PDFs. Converting a 290-page, 61.4 MB scanned/searchable PDF produces a ~32.1 MB .yom with pages, thumbnails, OCR, and a search index included — at the original scan resolution, with no upscaling.

Why a custom format?

• First paint needs only FlatBuffers + AVIF. The header and index are never compressed; page images are plain AVIF blobs. zstd is required only for OCR/search/PDF-export data, which stays off the rendering path.
• Byte-range everything. Every blob has an absolute offset, length, and its own SHA-256 — a reader (including one behind HTTP Range requests) can fetch and verify exactly one page.
• No writer freedom where it doesn't pay. Compression is fixed per blob kind and enforced by validators on both ends. Two independent implementations stay byte-compatible because there is nothing to configure.
• Immutable by design. Reading position, bookmarks, and annotations live outside the file. Packages stay cacheable, hashable, and safe to share.

The full rationale, binary layout, and validation rules are in the format specification.

What's in this repository

Path	Contents
schema/	The two FlatBuffers schemas (yom_index.fbs, yom_ocr.fbs) — the format's source of truth. Generated bindings are committed, not rebuilt at build time.
docs/format-spec.md	The normative v1 specification.
swift/	Swift implementation: YOMFormat (read/write/validate/search-index) and YOMConverter (PDF → .yom on iOS).
web/	TypeScript implementation of the same format, plus a fully client-side PDF → .yom pipeline (pdf.js + AVIF + zstd-wasm). Published to npm as yom-format.

Both implementations are exercised by round-trip and validator tests in CI, and are kept in lockstep with the Yomu iOS app, which writes and reads the same format (the Swift sources here are the same format code the app ships).

Swift

Add the package (iOS 16+; the AVIF codec uses ImageIO/UIKit, so this is an iOS library):

.package(url: "https://github.com/juntaki/yom-format.git", branch: "main")

Read pages from a .yom file:

import YOMFormat

let reader = try YOMPackfileReader(fileURL: url)
print(reader.pageCount)
let avifData = try reader.pageImage(at: 0)   // AVIF bytes for page 0

try YOMValidator.validate(fileURL: url)      // full integrity check

Write a package with YOMPackfileWriter.write(_:), or convert a PDF on-device with the YOMConverter product. Building from a checkout:

xcodebuild build -scheme YOMFormat -destination 'generic/platform=iOS Simulator'
xcodebuild test  -scheme YOM-Package -destination 'platform=iOS Simulator,name=iPhone 17 Pro'

TypeScript

npm install yom-format

import { YOMPackfileReader, validateYOM } from "yom-format";

const reader = await YOMPackfileReader.open(bytes); // Uint8Array of a .yom file
console.log(reader.pageCount);
const avif = await reader.pageImage(0);             // AVIF bytes for page 0

await validateYOM(bytes);                           // full integrity check

The browser-side PDF → .yom conversion pipeline (pdf.js rendering, AVIF encoding, OCR layout, search indexing — no server involved) ships as the yom-format/pipeline subpath. See web/README.md for details and bundler notes.

Format at a glance

Extension / MIME	.yom / application/vnd.yomu.yom
Magic / version	YOM1 / 1
Index	FlatBuffers MainIndex, uncompressed, at byte 4096
Page images	AVIF, exactly one image + one thumbnail per page
Compression	Fixed per blob kind: images & postings none; OCR, text, dictionary zstd
Search	Character-bigram index over NFKC, whitespace-stripped text; delta+varint postings
Integrity	SHA-256 per blob + index hash + whole-file hash
Mutability	None — user state lives outside the file

Relationship to Yomu

The format was designed for and extracted from Yomu (iOS). The app vendors its own copy of this code; the schemas and format logic here are kept identical, and this repository is the canonical home of the specification. Format documentation also lives on the site: yomu.juntaki.com/yom-format.

Scope notes

• YOMConverter (Swift) and the web pipeline are the two official converters; both write through the same validator-enforced writer rules.
• In-app search UI adapters are not part of this repository — the format exposes the search blobs and their encoding; how an app queries them is app territory (decodePostings is provided in both implementations).
• v1 deliberately excludes tiles, alternate codecs, encryption, and embedded annotations. See Non-goals.

License

MIT, except swift/vendor-CZstd/ — a vendored subset of zstd 1.5.7, BSD-licensed (see its LICENSE).