Getting Started

Getting Started

This page walks through an OCR Foundry dataset and shows you how to inspect the output. Examples use ocr-foundry; existing khocr-gen commands run the same CLI. The v0.2 distribution ocr-foundry installs both executables. The old khocr-gen PyPI distribution is separate and is not automatically updated by this release.

0. Before you start

You need:

• Python 3.12 or newer
• A Khmer font and an English font (.ttf / .otf)
• A UTF-8 text corpus, one string per line

1. Install

pip install ocr-foundry

For development installs, Rust acceleration, or offline/uv setups see Installation.

Confirm the CLI is on your PATH:

ocr-foundry --version
ocr-foundry --help

2. Lay out fonts

Create two subdirectories. The names matter:

fonts/
├── khmer/     <- .ttf / .otf Khmer fonts
└── english/   <- .ttf / .otf English fonts

Font files dropped directly in fonts/ (not in a subdirectory) are added to both pools as fallbacks. See Fonts for variant (bold/italic) handling and coverage checks.

3. Prepare a corpus

A plain UTF-8 text file with one string per line:

សួស្តី
Hello World
ស្វាគមន៍ Welcome

Mixed Khmer/English on one line is the point of the tool. See Corpus and Normalization for filtering, split control, and rare-character oversampling.

4. Generate a dataset

ocr-foundry generate \
  --corpus corpus/corpus.txt \
  --fonts fonts \
  --output data \
  --copies 3 \
  --storage lmdb

What this does:

• splits the corpus into train / val / test (default 80/10/10),
• renders --copies augmented variants of every training line,
• writes images + labels.txt + vocab.json,
• packs everything into LMDB and deletes the raw images (--storage lmdb).

Resulting layout:

data/
├── train/
│   ├── lmdb/          <- data.mdb, lock.mdb
│   └── labels.txt
├── val/
│   └── ...
├── test/
│   └── ...
└── vocab.json

Use --storage both to keep the raw image files as well. Full layout details are in Output and Storage.

One-liner with a config file

Instead of long flag lists, put the recipe in YAML:

ocr-foundry generate --config configs/generate.yml

ocr-foundry generate with no --config auto-detects configs/generate.yml when it exists. See Configuration.

For licensed multilingual source text and fonts, use the calibrated recipe from the repository root:

ocr-foundry generate --config configs/multilingual-realistic.yml \
  --corpus /path/to/licensed-lines.txt --fonts /path/to/fonts --output /tmp/multilingual

Its seed: 42 seeds splitting and each sample's rendering/augmentation, independent of worker count for fixed inputs, fonts and runtime. See Configuration for the recipe's height guard, script factors and augmentation weights. For evaluation on real documents, see Evaluation; the synthetic test split is not a real OCR benchmark.

5. Verify the augmentations look right

Render every augmentation method at fixed MIN and MAX intensity on clean canvases:

ocr-foundry verify --fonts fonts --output verify_output

Output is one PNG per method with the MIN column on the left and the MAX column on the right. You can also verify individual text decorations and effects:

ocr-foundry verify --fonts fonts --method text_deco_bold text_effect_glow

6. Inspect a generated dataset

# Summary of an LMDB database
ocr-foundry view --lmdb data/train/lmdb --summary

# Extract images (and labels) to a directory
ocr-foundry view --lmdb data/train/lmdb --output-dir extracted

# Print labels only
ocr-foundry view --lmdb data/train/lmdb --labels-only

7. Merge datasets

Multiple generation runs can be merged into one LMDB dataset:

ocr-foundry combine data_run1 data_run2 data_run3 --output data_merged

A merged vocab.json covering every character across all merged splits is written too.

Where to go next

I want to…	Read
Understand every flag	CLI Reference
Write a reproducible recipe	Configuration
Tune augmentations for my domain	Augmentation
Vary image heights	Variable Line Height
Add color, underlines, glow, shadows	Text Decorations and Effects
Speed up generation	Rust Acceleration
Score OCR on heldout documents	Evaluation
Compose pages with geometry	Page Layout
Something broke	Troubleshooting

Minimal end-to-end script

set -euo pipefail

pip install ocr-foundry

mkdir -p fonts/khmer fonts/english corpus
# ... copy your .ttf fonts and write corpus/corpus.txt ...

ocr-foundry generate \
  --corpus corpus/corpus.txt \
  --fonts fonts \
  --output data \
  --copies 3 \
  --storage both \
  --line-height-mode variable --min-line-height 32 --max-line-height 96 --line-height-step 8 \
  --font-size-mode proportional \
  --blur-prob 0.4 --rotation-prob 0.2 --jpeg-compression-prob 0.4

ocr-foundry view --lmdb data/train/lmdb --summary
ocr-foundry verify --fonts fonts --output verify_output