A pure-Rust, memory-safe, CPU-only OCR engine for a small family of hand-ported vision-language models. Baidu Unlimited-OCR is the fast default for document OCR, GOT-OCR2 handles specialized structured formats, SmolVLM2 handles image description and VQA, OneChart extracts chart data, and Polyphonic-TrOMR turns full scanned sheet-music pages or staff crops into MusicXML through --task music when supplied a local or converted artifact. All of it runs through model-specific Rust kernels with no general ML framework, no Python, no CUDA, no FFI at inference, and no GPU. It parses document images and PDFs into Markdown, MusicXML, structured JSON, or versioned NDJSON from a single static binary that fits in about 5 MB.

curl -fsSL https://raw.githubusercontent.com/Dicklesworthstone/franken_ocr/main/install.sh | bash

The installer detects your platform, downloads the right prebuilt binary from the v0.3.0 release, verifies it by SHA256, and puts focr on your PATH. Then focr pull fetches the weights once and you run offline forever after.

The problem. Baidu Unlimited-OCR is a strong document-parsing model: Markdown, tables, LaTeX, reading order, many pages in one pass. The official stack is Python plus CUDA. Most machines that need OCR (laptops, CI runners, agent hosts, edge boxes) have no usable GPU, and a Python plus CUDA dependency is heavy to ship and awkward to embed.

The solution. franken_ocr is a library plus a single-binary CLI (focr) that runs the ready model set on CPU with nothing but a Rust binary. It transforms bf16 checkpoints into custom .focrq int8 artifacts and runs them through kernels written for each model's exact shapes. On a real Unlimited-OCR page measured against the Baidu reference, the end-to-end character-error-rate is 0.0094; the decode matched the reference to within a single token, and on one token it beat the reference. That is a measured result on the 6.67 GB model, not a target.

# 1. Fetch the int8 weights once (~3.9 GB) into the local cache, verified by SHA256.
focr pull

# 2. OCR a page into Markdown (the human default).
focr ocr page.png

# 3. Same page as structured JSON (markdown + every span's bounding boxes).
focr ocr page.png --json

# 4. Write the result to a file; format follows the extension (.md or .json).
focr ocr page.png -o page.md
focr ocr page.png -o page.json            # structured JSON with bounding boxes

# 5. Also save figures the model can't transcribe (charts/photos) next to the .md.
focr ocr page.png -o page.md --extract-figures   # -> page.md + page_figures/

# 6. Use a specialized model when the task is not plain document OCR.
focr pull got-ocr2
focr ocr --model got-ocr2.int8.focrq --task tables table.png

focr ocr --model /path/to/smolvlm2.int8.focrq --task describe photo.jpg
focr ocr --model /path/to/smolvlm2.int8.focrq --task describe --question "How many people are visible?" photo.jpg

focr ocr --model /path/to/onechart.int8.focrq --task chart-data chart.png

focr ocr --model /path/to/tromr.focrq --task music score-page-or-staff.png -o score.musicxml

# 7. Stream NDJSON pipeline events for an agent (run_start ... run_complete; full event set via `focr robot schema`).
focr ocr page.png --robot

# 8. Prove the int8 kernel on THIS CPU is bit-identical to the scalar oracle.
focr robot selftest

# 9. (optional) Convert your own bf16 safetensors into the int8 .focrq format.
focr convert model.safetensors -o unlimited-ocr.focrq --quant int8

# TrOMR conversion is available for local sheet-music artifacts.
focr convert /path/to/tromr/model.safetensors -o tromr.focrq --quant int8 --model-id tromr

After step 1 the weights live in ~/.cache/franken_ocr/models and every later command runs fully offline.

A few models, every dimension fixed. A general ML framework pays a generality tax on every operation: dynamic dtype dispatch, arbitrary shapes, autograd bookkeeping, broadcast machinery, and a device abstraction. franken_ocr runs a small set of hand-ported models whose important dimensions are known up front. For the default Unlimited-OCR path that means hidden 1280, 10 heads, head_dim 128, 64 experts, top-6 routing, MoE intermediate 896, R-SWA window 128, and vocab 129280. That buys shape-specialized kernels with no runtime shape branching in the hot loop. The scope is a few hand-tuned, certified models, not any model; there is no generic runtime underneath.

Model status is explicit. focr models is the source of truth for the runtime zoo. Ready models have a runtime forward arm and can be used with focr ocr; planned models are visible so agents and humans can see the roadmap without accidentally running the wrong architecture. OneChart is runtime-ready for --task chart-data when you supply a local or converted onechart.int8.focrq; its public focr pull onechart manifest entry is still pending. TrOMR is runtime-ready for --task music when you supply a local or converted tromr.focrq; it accepts either a staff crop or a full printed/scanned page. The page path runs pure-Rust staff detection with global deskew, orders crops top-to-bottom, recognizes each staff sequentially through the certified ResNetV2 plus ViT encoder and four-head decoder, merges the semantic streams, and emits partwise MusicXML. Remaining TrOMR work is narrower: public pull-manifest upload, a perf-ledger row, optional **kern export, camera-photo dewarp/barline splitting, and broader corpus-quality metrics.

Performance claims are ledgered. The A11 zoo gauntlet records native runs beside pinned Hugging Face CPU references for GOT-OCR2, SmolVLM2, and OneChart. Decode-per-token speedups are documented where the native path is ahead, and full end-to-end rows stay in the ledger even when artifact loading or preprocessing makes the total slower. The README summarizes measured rows; docs/PERF_LEDGER.md is the audit trail.

Offline at inference. The only network step is focr pull, which runs ahead of time. There is no Python, no CUDA, no FFI, and no GPU in the inference path. The async runtime that orchestrates I/O and cancellation is an owned internal detail; the library API is synchronous and blocking, so there is no async plumbing to thread through your code.

Correctness before speed (always). A parity gate comes first and a faster kernel that drifts the OCR output is reverted, no source landed, and recorded in the negative-evidence ledger. Speed is shipped on top of parity, never instead of it. The int8 expert/FFN quantization is validated against the f32 path; the vision tower, projector, embeddings, MoE router, and all norms stay high precision.

Runtime ISA dispatch, one binary per arch. There is no per-CPU-feature variant to choose. One x86_64 binary covers AVX2 / AVX-VNNI / AVX-512-VNNI; one aarch64 binary covers NEON / SDOT / SMMLA. At load, CPU feature detection picks the fastest available int8 kernel tier. Apple Silicon deliberately prefers SDOT over SMMLA because measured M-series i8mm throughput does not beat the dot-product path; non-Apple ARM64 can choose SMMLA when it is actually faster. On a Threadripper 5995WX, a Zen 3 part whose ceiling is AVX2, dispatch correctly selects AVX2.

Bounded generated-token KV. Every decoder attention layer is R-SWA (Reference Sliding Window Attention). Each generated token attends to all reference tokens (visual plus prompt prefix, kept as a frozen, never-evicted global KV) plus only the previous 128 generated tokens through a ring-buffer KV cache. Generated-token KV memory stays constant instead of growing with output length. That, not arbitrary input resolution, is what "Unlimited" means.

franken_ocr is the only one of these built for a fixed, hand-tuned set of models on CPU.

When to use franken_ocr:

• You need this model's output and your host has no usable GPU.
• You want to embed OCR in a Rust program with no Python or FFI.
• You want one binary you can drop on a CI runner, an agent host, or an edge box.

When franken_ocr might not be ideal:

• You need a generic inference runtime that loads arbitrary models. franken_ocr runs a few hand-ported, certified models by design.
• You need the OmniDocBench leader; Unlimited-OCR is strong but not the top of the board.

curl -fsSL https://raw.githubusercontent.com/Dicklesworthstone/franken_ocr/main/install.sh | bash

The script detects your OS and CPU architecture, downloads the matching binary from the v0.3.0 release, verifies the SHA256 sidecar, and installs focr. Under WSL it proceeds as Linux. Under native Git-Bash, MSYS, or Cygwin it points you at the PowerShell installer below and exits cleanly.

On native Windows, install from PowerShell:

irm https://raw.githubusercontent.com/Dicklesworthstone/franken_ocr/main/install.ps1 | iex

This downloads the focr-x86_64-pc-windows-msvc.exe binary, verifies it by SHA256, and puts focr on your PATH.

Release binaries are raw executables, not tar.gz archives. Each one is a single portable file that dispatches the ISA tier at runtime, so there is one binary per architecture (no per-CPU-feature variant). Sizes are roughly 4.7 to 5.9 MB. Linux binaries are gnu (glibc), not musl.

Each asset has a <asset>.sha256 sidecar in the standard "<hex> <asset>" format. Download the binary and its sidecar from the release base URL, verify, then install. Example for Apple Silicon:

base=https://github.com/Dicklesworthstone/franken_ocr/releases/download/v0.3.0
asset=focr-aarch64-apple-darwin-neon-sdot-i8mm

curl -fsSLO "$base/$asset"
curl -fsSLO "$base/$asset.sha256"

# macOS: shasum -a 256 -c   |   Linux: sha256sum -c
shasum -a 256 -c "$asset.sha256"

chmod +x "$asset"
mv "$asset" /usr/local/bin/focr

On Linux, swap the asset name and use sha256sum -c "$asset.sha256".

franken_ocr requires the nightly Rust toolchain pinned in rust-toolchain.toml. cargo build --release builds both the focr and franken_ocr binaries from one shared entrypoint.

The catch: franken_ocr path-depends on sibling repositories that are not published on crates.io (asupersync, patched to /dp/asupersync; ../frankentorch; ../frankensqlite). A fresh-clone cargo build or a cargo install --git will fail to resolve those dependencies. There is no working cargo install from crates.io. Prebuilt binaries are the supported path; build from source only if you have those sibling repositories laid out as the workspace expects.

cargo build --release
# produces target/release/focr and target/release/franken_ocr (identical behavior)

1. Install the binary with the curl one-liner, or download and verify a release asset by hand.
2. Fetch the weights once: focr pull. This downloads about 3.9 GB of int8 weights plus tokenizer.json into ~/.cache/franken_ocr/models, verifying every byte by SHA256 against a committed manifest.
3. Verify your CPU kernel (optional but reassuring): focr robot selftest. Exit 0 means the dispatched int8 GEMM is bit-identical to the scalar oracle on this host.
4. OCR a page: focr ocr page.png for Markdown, add --json for structured output (markdown + bounding boxes), -o out.md / -o out.json to write a file, or --robot for an NDJSON event stream.
5. Batch many pages in one process (model loaded once): focr ocr-batch page1.png page2.png page3.png --json.

franken_ocr is a library as well as a CLI. The public API is synchronous and blocking: OcrEngine owns the internal asupersync runtime, loads the model on first use, caches that model behind an Arc, and returns ordinary FocrResult<T> values. Callers do not need to build or pass an async runtime.

use std::path::Path;

use franken_ocr::{OcrEngine, FocrResult};

fn main() -> FocrResult<()> {
    let engine = OcrEngine::new()?;

    let markdown = engine.recognize(Path::new("page.png"))?;
    println!("{markdown}");

    let document = engine.recognize_with_layout(Path::new("page.png"))?;
    println!("{} layout spans", document.layout.len());

    Ok(())
}

The same engine exposes the surfaces the CLI uses:

Model resolution follows the CLI rules: FOCR_MODEL_PATH wins, then the default cache location is searched. For path-explicit calls, pass the model artifact directly. The engine keeps the one-live-forward discipline from the CLI, so a single process can reuse weights without fanning out concurrent model forwards.

Both focr ocr and focr robot run accept the same image plus inference-tuning flags. The default crop mode is base (the certified single 1024-pixel global view); --crop-mode gundam selects the reference dynamic-resolution tiling, whose end-to-end certification is still pending.

Parse a document image into Markdown (default), JSON, or an NDJSON stream.

focr ocr page.png                          # Markdown to stdout
focr ocr page.png --json                   # structured JSON to stdout
focr ocr page.png -o page.md               # write Markdown to a file
focr ocr page.png -o page.json             # write structured JSON (markdown + boxes) to a file
focr ocr page.png -o page.md --extract-figures   # also save figures, referenced from the .md
focr ocr page.png --robot                  # NDJSON pipeline events
focr ocr page.png --crop-mode gundam       # reference dynamic-resolution tiling (uncertified)
focr ocr page.png --max-length 4096 --temperature 0.0
focr ocr page.png --model /path/to/unlimited-ocr.focrq
focr ocr eq.png --task formula --model got-ocr2.int8.focrq   # specialized task routing (see below)
focr ocr chart.png --task chart-data --model /path/to/onechart.int8.focrq

Output (-o/--output FILE). Writes the result to a file instead of stdout; the format follows the extension: .json emits structured JSON, any other extension (e.g. .md) emits Markdown. --json forces JSON regardless of extension. The structured JSON carries the rendered markdown plus a layout array, one {label, boxes} entry per grounded span, where each box is [x1, y1, x2, y2] in source-image pixels (a PDF nests these under a per-page pages array). This is the same shape --json prints to stdout.

Figures (--extract-figures). The model sees figures/photos/diagrams it does not transcribe to text. With --extract-figures, those regions are cropped out of the source image and saved into a subfolder (default <output-stem>_figures/, or set --figures-dir DIR), then referenced from the output: the Markdown gets a real ![figure N](report_figures/page1_figure_1.jpg) in place of each figure, and the JSON gains a figures array of {label, page, bbox, path}. Each figure's format is chosen by content: JPG (quality 85) for photographic regions, lossless PNG for line-art / charts / screenshots. Requires -o (or --figures-dir for a stdout run). PDFs name figures per page (page{N}_figure_{M}).

Tasks (--task). Convenience routing over the model zoo (focr models). --task ocr (the default) is plain document OCR, unchanged. --task formula, tables, chart, molecular, and geometry are served by GOT-OCR2's OCR with format: mode, so each implies --format (an explicit --format composes idempotently) and needs the got-ocr2 model: focr pull got-ocr2, then --model got-ocr2.int8.focrq. --task music has two valid lanes: TrOMR is the native OMR path and returns MusicXML, while GOT-OCR2 can still run its sheet-music format mode. Use --model /path/to/tromr.focrq --task music for a full printed/scanned page or a staff crop, or use the GOT lane (--model got-ocr2.int8.focrq --task music). --task describe (photo description / VQA) is served by the smolvlm2 model: --model smolvlm2.int8.focrq --task describe, optionally with --question "What color is the car?". SmolVLM2 has no instruction modes; the task is the question (default: the model-card caption prompt). --task chart-data is the OneChart path: it runs chart-to-dict extraction with the number-head reliability check and needs a onechart artifact, for example --model /path/to/onechart.int8.focrq. Pointing a specialized task at the wrong model family fails with usage guidance before any weights load.

Tuning flags: --base-size and --image-size (preprocessing resolution), --crop-mode (gundam or base), --max-length (decode token cap), --temperature (sampling), --no-repeat-ngram and --ngram-window (the sliding no-repeat n-gram decode guard).

OCR many images in one process, loading the model once and reusing it across all pages.

focr ocr-batch a.png b.png c.png --json     # one load, many pages
focr ocr-batch *.png --f32                  # use the high-precision f32 decode path

--f32 runs the f32 decode path instead of int8 (see Limitations for when this matters).

Download the int8 weights and tokenizer into the cache, verifying every byte.

focr pull                                   # default int8, built-in manifest
focr pull --quant int8 --json               # explicit quant, machine-readable
focr pull --manifest ./manifest.json        # override the manifest source

Downloads run over asupersync's native HTTP stack (rustls + webpki-roots), reassemble GitHub split parts, verify against a committed SHA256 manifest, and cache to ~/.cache/franken_ocr/models. Verified in production against this repo's models-v1 GitHub release and a Hugging Face mirror.

List the model zoo with each id, status, task set, tokenizer family, decoder family, and default artifact name.

focr models                                 # human table
focr models --json                          # machine-readable list

Which model do I use?

unlimited-ocr is the fast default for ordinary text. Reach for got-ocr2 only when you need format extraction (formulas, tables, charts, etc.); it is a much larger decode and is not meant to replace the default for plain text. Install and run it with:

focr pull got-ocr2                                    # download the weights + tokenizer
focr ocr --model got-ocr2.int8.focrq page.png         # plain-text OCR
focr ocr --model got-ocr2.int8.focrq --format eq.png  # structured .mmd: LaTeX / tables / charts / music
focr ocr --model got-ocr2.int8.focrq --task tables page.png  # task shorthand (implies --format)

Run SmolVLM2 (photo description / VQA) with a local or converted artifact:

focr ocr --model /path/to/smolvlm2.int8.focrq --task describe photo.jpg
focr ocr --model /path/to/smolvlm2.int8.focrq --task describe --question "How many dogs are there?" photo.jpg

Run OneChart (chart-to-data extraction) with a local or converted artifact:

focr ocr --model /path/to/onechart.int8.focrq --task chart-data chart.png

The OneChart runtime path is ready, but the public pull manifest has not published the artifact yet. Use a local or converted onechart.int8.focrq until focr pull onechart appears in models/manifest.json.

Run TrOMR sheet-music OMR with a local or converted artifact:

focr convert /path/to/tromr/model.safetensors -o tromr.focrq --quant int8 --model-id tromr
focr ocr --model /path/to/tromr.focrq --task music score-page.png -o score.musicxml
scripts/tromr_convert_e2e.sh
scripts/tromr_music_e2e.sh

The TrOMR runtime accepts a single staff crop or a full printed/scanned page. If the detector finds multiple staves, it deskews the page globally, crops staves top-to-bottom, recognizes each staff through the same certified single-staff path, and emits one MusicXML part per staff; if it finds fewer than two staves, it treats the whole image as the staff input. The current tromr.focrq is an 82 MB all-f32 artifact with 260 tensors; int8 candidates remain gated until they are measured lossless. Public focr pull tromr support is still waiting on artifact publication.

Offline weight transformation: a bf16 safetensors checkpoint into a custom .focrq artifact. Decoder tensors are int8 where that policy is validated for the model; tiny models such as TrOMR can stay f32 by default while still using the same one-binary runtime.

focr convert model.safetensors -o unlimited-ocr.focrq --quant int8
focr convert got.safetensors -o got-ocr2.int8.focrq --quant int8 --model-id got-ocr2
focr convert smolvlm2.safetensors -o smolvlm2.int8.focrq --quant int8 --model-id smolvlm2
focr convert onechart.safetensors -o onechart.int8.focrq --quant int8 --model-id onechart
focr convert tromr/model.safetensors -o tromr.focrq --quant int8 --model-id tromr
focr convert model.safetensors -o out.focrq --quant int8 --json

The Unlimited-OCR output is proven byte-identical to the load-time int8 path on the real 6.67 GB model (6,672,547,120 bytes of bf16 become 3,914,093,440 bytes of int8, about 3.9 GB), with the source SHA256 stamped into the header. --model-id selects the architecture descriptor, tied-head policy, tensor-name prefixes, tokenizer expectation, and license notice for each zoo model. --arch <target> records architecture-specific prepacking intent (default: generic). Only int8 is validated for shipped artifacts today; int4 is still a gated optimization path.

Agent-facing surface: versioned NDJSON, self-describing, line-oriented, easy to pipe.

focr robot run page.png                      # stream OCR events as NDJSON (same flags as ocr)
focr robot schema                            # self-describing, versioned event/contract schema
focr robot health                            # model present? arch features? thread budget?
focr robot backends                          # detected SIMD tiers + core count
focr robot selftest                          # int8 kernel vs scalar oracle on this host

focr robot selftest runs the dispatched int8 GEMM against a bit-identical scalar oracle across a shape battery, including the worst-case K=6848 i32-accumulation overflow case, and emits a single JSON verdict; it exits 1 if any parity case diverges. It has passed 24/24 on Apple SDOT and on a real x86 AVX2 box. Set FOCR_FORCE_ARCH to verify a specific ISA tier.

focr ocr records each run in a local fsqlite store on a best-effort basis. A store failure prints a note to stderr and never fails the OCR run itself. The default store is ~/.cache/franken_ocr/runs.db; set FOCR_RUN_STORE to use a different path.

focr runs --limit 10                         # recent runs, plain table
focr runs --limit 10 --format json           # one JSON object with a runs array
focr runs --format ndjson                    # one run record per line
focr runs --id <run-id> --json               # inspect one run

focr sync export-jsonl --json                # export the canonical audit log
focr sync export-jsonl --file runs.jsonl     # choose the export path
focr sync import-jsonl --file runs.jsonl --json

Exports are canonical and atomic: the writer takes an exclusive lock, writes a temporary file beside the target, fsyncs it, then renames it into place. Imports are idempotent because records replace by run_id.

doctor still emits its planned JSON contract and returns a not-yet-implemented status (exit code 1).

focr doctor --json                          # idempotent self-check / repair

The hot path is not a generic tensor interpreter. Each model is converted into a .focrq artifact, then executed through fixed-shape Rust code that calls model-specific kernels and keeps allocations out of the decode loop.

The backend keeps handwritten SIMD narrow. Ordinary Rust loops stay in place where LLVM autovectorizes well; audited SIMD is reserved for int8 matmul kernels whose scalar oracle is bit-identical and whose speedup is measured. That is why robot backends reports the exact dispatched tier, and why AMX and int4 remain gated until real kernels and parity evidence land.

Apple Silicon / ARM64. The aarch64 backend detects NEON dot-product (SDOT) and matrix-multiply int8 (SMMLA / i8mm) at runtime. The hot decoder linears use packed int8 matmul kernels where the scalar fallback proves the exact result, while norms, softmax, preprocessing, and TrOMR's f32 vision/decoder glue stay as simple loops that LLVM can autovectorize. On macOS, dispatch prefers SDOT over SMMLA because measured M-series cores issue i8mm at a rate that does not beat the dot-product path once operand packing is included. On non-Apple ARM64, the order can favor SMMLA when the hardware makes it faster. Both tiers share the same scalar oracle, and FOCR_FORCE_ARCH=sdot|smmla|scalar focr robot selftest verifies the selected path on the current host.

Intel / AMD x86-64. The x86 backend detects AVX-512-VNNI, AVX-VNNI, and AVX2 in that order, then falls back to scalar. AVX-VNNI and AVX-512-VNNI take the native dot-product path for int8 decode; AVX2 uses an exact non-saturating implementation rather than a shortcut that could corrupt accumulation. AMX is not advertised until there is a real AMX backend; robot backends reports the tier this binary will actually dispatch. The same binary therefore runs correctly on older AVX2-only Zen/Intel hosts and selects wider VNNI kernels on newer CPUs.

Quantization policy. The validated int8 path targets decoder GEMMs: dense MLPs, MoE expert/FFN matrices, and the per-token decode matmuls. The vision tower, projector, embeddings, MoE router, and all norms stay high precision. That split is deliberate: quantizing the vision side breaks OCR quality, while decoder int8 delivers the bandwidth win where it is safe.

TrOMR-specific CPU primitives. The sheet-music lane has native staff preprocessing, global deskew and staff grouping, TF-SAME padding arithmetic, TF-SAME max-pool support, a Torch-parity GroupNorm kernel with optional fused ReLU, and a hybrid ResNetV2 plus ViT encoder checked at cosine 1.0 against oracle seams. Weight-standardized convolutions are folded during checkpoint export so the runtime can keep the conv path simple. The decoder includes self-attention, cross-attention, GEGLU, stream embeddings, and four output heads, then merges the rhythm/pitch/lift streams into semantic music tokens and partwise MusicXML. Opaque-alpha RGBA staff images take the RGB luma path rather than upstream's blanket inverted-alpha path, because the literal upstream rule blanks fully opaque demo staves; docs/DISCREPANCIES.md records the measured SER impact. The TrOMR closeout pins mean SER at 0.211 on committed single-staff examples and shows detection-lossless full-page reads at 0.125 / 0.040 SER for stacked staves. These pieces are scalar Rust loops designed for LLVM autovectorization across Apple Silicon and Intel/AMD CPUs; hand-written wide SIMD stays limited to int8 matmul kernels where the proof and measurement support it.

Zoo performance evidence. The latest zoo gauntlet keeps paired reference rows for GOT-OCR2, SmolVLM2, and OneChart. On an aarch64 host with NEON dotprod at eight threads, the native decode-per-token path measured 3.37x over Hugging Face CPU for GOT-OCR2, 2.58x for OneChart, and 1.67x for SmolVLM2. The ledger also records the full end-to-end rows, including the current artifact-load tax, so the project can improve throughput without hiding unfavorable totals.

Correctness gates. Every accelerated int8 GEMM has a bit-identical scalar fallback. focr robot selftest includes the doctrine worst case, K=6848, proving i32 accumulation stays in range. The batch scheduler and decode cache are guarded by byte-identity tests against the proven sequential path.

Unlimited-OCR is a roughly 3B-parameter Mixture-of-Experts vision-language model and a DeepSeek-OCR derivative. The forward path:

  page.png
     │
     ▼
┌──────────────────────────────────────────────────────────────────────┐
│  DeepEncoder (vision tower, kept high precision)                       │
│    SAM-ViT-B  ──►  16x conv token compressor  ──►  CLIP-L/14           │
│    (SAM + CLIP features concatenate to 2048 dims)                      │
└──────────────────────────────────────────────────────────────────────┘
     │
     ▼
┌──────────────────────────────────────────────────────────────────────┐
│  Linear projector   2048 ──► 1280                                      │
└──────────────────────────────────────────────────────────────────────┘
     │
     ▼
┌──────────────────────────────────────────────────────────────────────┐
│  DeepSeek-V2 MoE decoder   12 layers, hidden 1280, 10 heads, hd 128    │
│    layer 0 : dense MLP                                                 │
│    layers 1..11 : MoE  ── router 1280 ──► 64, top-6 of 64 experts      │
│                        (each expert 1280 ──► 896, SiLU-gated)          │
│                        + 2 always-on shared experts                    │
│    attention replaced by R-SWA (window 128)                           │
│    final RMSNorm  ──►  lm_head  1280 ──► 129280                        │
└──────────────────────────────────────────────────────────────────────┘
     │
     ▼
  autoregressive text  ──►  Markdown / JSON / NDJSON

  ── ISA dispatch (chosen at load, one binary per arch) ──
     aarch64 : NEON / SDOT / SMMLA (i8mm)
     x86_64  : AVX2 / AVX-VNNI / AVX-512-VNNI

• DeepEncoder (vision tower). SAM-ViT-B, then 16x conv token compression, then a CLIP-L/14 cascade. SAM and CLIP features concatenate to 2048 dims. Kept high precision; quantizing the vision tower hurts OCR, a result both community quants confirm.
• Linear projector. A single 2048 to 1280 map bridges the vision tower to the decoder hidden size.
• DeepSeek-V2 MoE decoder. 12 layers, hidden 1280, 10 heads, head_dim 128. Layer 0 is a dense MLP; layers 1 to 11 are MoE, with a router (1280 to 64) selecting the top 6 of 64 experts (each a 1280 to 896 SiLU-gated MLP) plus 2 always-on shared experts. Final RMSNorm and lm_head (1280 to 129280) feed autoregressive sampling.
• R-SWA. The decoder's attention novelty; generated-token KV is bounded to a 128-token window while the reference block stays as a frozen, never-evicted global KV.

The upstream checkpoint is a single 6.67 GB bf16 safetensors shard under the MIT license. focr convert turns the decoder FFN/expert GEMMs into int8 inside the .focrq format and leaves the vision tower, projector, embeddings, router, and norms high precision.

The weights are not in the cache yet. Fetch them once:

focr pull

On an interactive terminal, focr ocr with no model present offers to download. In robot mode and any non-TTY context it never auto-downloads; it returns a model-not-found result plus a focr pull hint, so automated callers stay predictable.

Check what the binary detected on this host, then confirm the int8 kernel is correct:

focr robot backends     # selected SIMD tier + available exact backends
focr robot health       # model present, arch features, thread budget
focr robot selftest     # int8 GEMM bit-identical to scalar oracle (exit 1 on divergence)

To force a specific tier for verification, set FOCR_FORCE_ARCH (for example FOCR_FORCE_ARCH=scalar focr robot selftest).

focr pull verifies every byte automatically, so prefer it. If you downloaded an asset by hand and shasum -a 256 -c (or sha256sum -c) fails, the download is corrupt or truncated; re-download the binary and its .sha256 sidecar from the v0.3.0 release. A format or version mismatch on a model artifact surfaces as exit code 7.

After focr pull, nothing else touches the network. To pin the artifact explicitly or to use weights you converted yourself:

export FOCR_MODEL_PATH=/path/to/unlimited-ocr.focrq
focr ocr page.png

FOCR_MODEL_PATH accepts a .focrq blob or a safetensors directory.

On most pages int8 is byte-identical to f32, but a few hard, dense tables can send int8 decode into a repetition run. Two mitigations:

# Tighten the no-repeat n-gram guard.
FOCR_NO_REPEAT_NGRAM=20 focr ocr hard_table.png

# Or fall back to the high-precision f32 decode path.
focr ocr-batch hard_table.png --f32

For the single-page ocr command, point FOCR_MODEL_PATH at the bf16 safetensors directory to run f32 end to end.

What this is and is not:

• int8 can repeat on hard tables. int8 decode is roughly 2.5x faster and byte-identical to f32 on typical pages, but some dense tables (for example page_0590) can trigger repetition runs. The no-repeat n-gram guard and the f32 fallback are the documented kill-switches. The vision tower stays high precision because quantizing it breaks OCR.
• Image and PDF input. PNG, JPG, and similar, plus native PDF: focr ocr file.pdf rasterizes each page (pure-Rust, no FFI, no out-of-band pdftoppm) and OCRs the document. The fast path covers common scanned-PDF codecs: JPEG (DCTDecode), CCITT Group 4 fax, and FlateDecode/LZW raw rasters. Two image codecs with no production-quality pure-Rust decoder, JPXDecode (JPEG 2000) and JBIG2Decode, plus born-digital vector/text pages, are reported with a precise error naming what was unsupported. Rasterize that PDF out of band and retry.
• Native Windows (x86_64) is supported and proven end-to-end; ARM64 is not yet. The x86_64-pc-windows-msvc binary runs full OCR on real Windows 10: the same 3.9 GB int8 weights, vision tower, and DeepSeek-V2 decoder produce the same markdown a Mac or Linux host does. focr.exe robot selftest passes 24/24 (int8 GEMM bit-identical to the scalar oracle, including the K=6848 overflow case). focr pull works on Windows too; the full 3.9 GB multi-part download, reassembly, and SHA-256 verify complete over the native async HTTP/TLS stack. The earlier send-path bug that surfaced as WSAENOTCONN / os error 10057 (bd-15ow) is fixed. The model cache resolves to %LOCALAPPDATA%\franken_ocr\models, falling back to %USERPROFILE%\.cache\franken_ocr\models; on macOS and Linux it stays at ~/.cache/franken_ocr/models. The one remaining gap, tracked under epic bd-3u97, is that ARM64 Windows is not published.
• A few models, not any model. Generality is a deliberate non-goal. franken_ocr runs a small family of hand-ported models: Unlimited-OCR by default, GOT-OCR2 for structured formats, SmolVLM2 for image description/VQA, OneChart for chart-to-data extraction, and TrOMR for sheet-music OMR on full printed/scanned pages or staff crops. TrOCR and pix2tex remain descriptor-only roadmap items. Each ready model is transformed offline and certified against its reference before it ships. It will not become a generic inference runtime that loads arbitrary checkpoints.
• Not benchmark SOTA. Unlimited-OCR is strong but not the OmniDocBench leader. The aim is fidelity to this model, bounded generated-token KV for long-document parsing on CPU, and speed on commodity hardware, not topping a benchmark.
• CPU only. No GPU. CUDA is a deferred stretch goal; CPU stays the product.

Is this affiliated with Baidu? No. It is an independent pure-Rust reimplementation that runs Baidu's openly-licensed (MIT) model weights. The weights and any quantized derivative this project distributes carry the model notice surfaced by the binary and .focrq metadata: Baidu Unlimited-OCR - Copyright (c) 2026 Baidu, MIT License.

Why use this instead of llama.cpp or ONNX? Both are excellent general runtimes. franken_ocr is a focused build: a small fixed set of hand-ported models lets the kernels specialize to each model's exact shapes and skip the generality tax. The whole thing ships as one Rust binary with no FFI and is portable to targets where ort or CUDA cannot build.

Why Rust, and why forbid unsafe? Memory safety for a multi-gigabyte weight loader and a tight decode loop, with unsafe confined to small, audited SIMD modules that each carry a bit-identical scalar fallback.

Does int8 hurt accuracy? On a real page the end-to-end character-error-rate is 0.0094 versus the Baidu reference, matching the reference decode to within a single token. The known failure mode is repetition on a few dense tables, mitigated by the no-repeat n-gram guard or the f32 fallback. The vision tower is never quantized.

Can I embed it in my Rust program? Yes. The library API is synchronous and blocking, and the engine owns its runtime internally, so there is no async plumbing to thread through your code.

Where do the weights live, and do they ever download at inference? They cache to ~/.cache/franken_ocr/models after focr pull. Weights are never bundled and never downloaded during focr ocr; inference is offline.

Which binary do I download? One per architecture. The x86_64 binary covers AVX2/AVX-VNNI/AVX-512-VNNI; the aarch64 binary covers NEON/SDOT/SMMLA. The right tier is chosen at load. Or just use the curl installer and let it pick.

Please don't take this the wrong way, but I do not accept outside contributions for any of my projects. I simply don't have the mental bandwidth to review anything, and it's my name on the thing, so I'm responsible for any problems it causes; thus, the risk-reward is highly asymmetric from my perspective. I'd also have to worry about other "stakeholders," which seems unwise for tools I mostly make for myself for free. Feel free to submit issues, and even PRs if you want to illustrate a proposed fix, but know I won't merge them directly. Instead, I'll have Claude or Codex review submissions via gh and independently decide whether and how to address them. Bug reports in particular are welcome. Sorry if this offends, but I want to avoid wasted time and hurt feelings. I understand this isn't in sync with the prevailing open-source ethos that seeks community contributions, but it's the only way I can move at this velocity and keep my sanity.

The franken_ocr source code is licensed under the MIT License with an OpenAI/Anthropic Rider, Copyright (c) 2026 Jeffrey Emanuel.

The model weights are a separate matter. The Baidu Unlimited-OCR weights, and any quantized derivative this project distributes, are under the MIT License, Copyright (c) 2026 Baidu, reproduced in full in LICENSE under "THIRD-PARTY MODEL WEIGHTS, NOTICE". That notice travels with any distributed weight artifact.

• COMPREHENSIVE_PLAN_FOR_FRANKEN_OCR.md, the master plan: architecture census, quantization format, the kernel-optimization catalog, the alien-artifact math families, the verification gauntlet, and the phased roadmap.
• AGENTS.md, conventions for human and agent contributors, including the engineering doctrine.
• CHANGELOG.md, the project history.
• docs/PERF_LEDGER.md, the honest measured perf-ratio log.
• docs/NEGATIVE_EVIDENCE.md, what did not work, including results inherited from sibling projects.
• docs/DISCREPANCIES.md, known measured divergences from the reference model.