Card Identification and OCR

Card Identification and OCR

English · Español

A user's own card image (a custom proxy, a scan, a fan-made card) is just a picture: the app does not know which real card it represents. "Link with a card" answers that. Rust proposes candidates from the file name and, optionally, from an OCR reading of the whole card image matched against a local catalogue of Scryfall names; the user reviews and confirms; the card keeps its own image but gains the real card's data, so it counts in statistics, card info and token detection.

Boundaries: Scryfall access is Scryfall-Integration; the Cockatrice/other-language fallbacks used here are External-Sources; the card model is Deck-Model.

Files

File	Responsibility
src-tauri/src/commands/identify.rs	Commands ocr_ready, card_identify, identify_batch, card_link, card_link_externa; the pipeline identificar(); scoring (puntuar, parecido, normalizar); ordering of typed searches.
src-tauri/src/ocr.rs	OCR engine (ocrs + rten), model download, line reading, zone split (repartir), footer/type/token heuristics.
src-tauri/src/originales.rs	Finding the original English name anywhere on the card (buscar) and reading set / number / language from the footer (pie_leido).
src-tauri/src/scryfall/nombres.rs	The local name and set-code indexes these two use (Indice, codigos_de_sets).
src-tauri/src/scryfall/fichas.rs	Spanish token type -> English token name (candidatas).
src/features/variants/vincular.ts	vincular(), desvincular(), propiasPorImagen(), estaVinculada(): the deck mutation.
src/features/variants/vincular-dialog.tsx	Single-card dialog "Link with a card".
src/features/variants/vincular-lote.tsx	Bulk dialog "Link in bulk" (phases: choose, read, review, apply).

What "linked" means

CardSource::Custom (src-tauri/crates/proxy-core/src/model.rs) has two optional fields:

Field	Meaning
scryfall_id (JSON scryfallId)	Linked to a real Scryfall printing.
externa (RefExterna { fuente, id })	Linked to a card from another source (a Cockatrice token that Scryfall lacks).

Both absent = a plain custom image. Design decisions:

• The image stays the user's. Linking never replaces the picture. The linked card provides identity: name, set, collector number, type line, layout, language, which feed the info panel, the statistics (id_scryfall() in commands/stats.rs returns a custom card's scryfall_id, so cached oracle data, colours, mana value, prices and function tags apply) and "Detect tokens" (tokens created by linked cards).
• externa is an optional field, not a new enum variant, so an older app version still opens such a deck and sees a normal custom card.
• CardMeta.image_lang records the language of the image, which may differ from the linked printing's lang (a Spanish proxy linked to the English printing). vincular() sets it from the footer reading when OCR provides it; it is also editable in the card menu; the library uses it for deck language details (src-tauri/src/biblioteca.rs).
• Copies share the link. vincular() applies to the chosen card and to every custom copy with the same front.blob, so one decision links all copies of that image. The bulk dialog also shows one row per distinct image (propiasPorImagen). Relinking a designed card to a different card drops its design: vincular() (via cambiaDeCarta) runs quitarDiseno on it first (back to its pre-design image, back and edits) and then writes the new source and meta. Otherwise the designer kept showing the design built from the previous card, and "Remove design" restored before.source (the old link) over the new one, leaving source and meta pointing at different cards. The same link keeps the design.
• Linking keeps the image-specific state. The new meta comes from the linked card but imageStatus is cleared, highres is kept from the custom card, and langFallback is reset (the Scryfall image state does not describe the user's picture).
• Unlinking (desvincular) only resets source to { kind: "custom" } on the card and its same-image copies; the metadata that linking wrote is left in place (verified in vincular.ts: only source changes).

What card_link returns

card_link(scryfall_id) -> CardMeta: from the local cache (cache::leer) if present, else from Scryfall, saving the card into scryfall_cards so statistics work offline; NotFound if Scryfall does not have it. card_link_externa(fuente, id) -> CardMeta reads the Cockatrice catalogue (meta_cockatrice). The front end then writes the result into the deck (vincular() in vincular.ts), so the Rust side does not mutate the deck.

Identification pipeline (identificar)

Input: the image blob hash, an optional hint (the file name, or what the user typed), ocr: bool, and escrito: bool ("the hint was typed by a person"). card_identify (single dialog) always passes escrito = true; identify_batch passes false.

Output CardIdentify { lines, candidates, best, confidence, pie }.

flowchart TD
    A["hint, blob, ocr, escrito"] --> B["pista = ocr::limpiar(hint), len >= 3"]
    B --> C{"ocr?"}
    C -- yes --> D["models present? else descargar_modelos<br/>decode blob, spawn_blocking leer_carta"]
    C -- no --> E
    D --> E["Lectura: titulo, pie, resto, tipo, ficha, lineas"]
    E --> F["name index: seguros / aproximados<br/>(pista exact, originales::buscar)"]
    E --> G["pie_leido(lineas, set codes)"]
    G --> H["card_lang(set, number, en) = del_pie"]
    E --> I{"ficha?"}
    I -- yes --> J["fichas::candidatas then<br/>search ! name t:token include:extras"]
    F --> K["search up to 6 distinct texts<br/>exact name else multilingual in-name"]
    H --> K
    J --> K
    K --> L{"no candidates?"}
    L -- yes --> M["fuentes::nombres_ingleses on pista/titulo"]
    K --> N{"escrito?"}
    N -- yes --> O["multilingual search up to 700<br/>ordenar_por_lo_escrito"]
    K --> P["+ Cockatrice tokens"]
    P --> Q["puntuar(candidates, seguros + principales)"]
    Q --> R["fallbacks: del_pie or first token = Probable"]
    R --> S["best moved to index 0"]

Loading

Step by step:

1. Hint. ocr::limpiar strips symbols, quotes and trailing single letters (mana symbols read as letters); fewer than 3 characters means no hint.
2. OCR (only if requested). If the model files are missing they are downloaded first (descargar_modelos). The blob is read with AppState::leer_blob, decoded to RGBA at full resolution, and leer_carta runs in spawn_blocking (it is CPU-heavy and synchronous). Details below.
3. Priority of texts. principales = hint, then footer names (leido.pie), then title lines. The footer comes before the title because custom cards often invent their title, while the footer carries the original English name.
4. Local name index (nombres::indice, no network after the first download): for the hint, exacto then al_principio; for the OCR lines, originales::buscar returns real card names found anywhere, split into seguros (exact hits) and aproximados (found by similarity). If nothing was found at all, the closest name to the hint becomes an approximate hit (one wrongly typed letter).
5. Footer (pie_leido): set code, collector number and language, using the set-code index (codigos_de_sets). If both set and number exist, card_lang(set, number, "en") fetches that exact printing as a candidate (del_pie).
6. Tokens. If the reading says the card is a token (Lectura.ficha), fichas::candidatas turns the type line ("Ficha de criatura - Guerrero") or a one-word title into an English token name from the table in scryfall/fichas.rs, and Scryfall is searched with !"Warrior" t:token include:extras. Exact-name results go first (so "Dinosaur // Treasure" does not precede "Treasure"). The first token found is remembered as primera_ficha.
7. Search. seguros, aproximados, then principales are searched; at most MAX_TEXTOS = 6 distinct normalised texts, stopping at MAX_CANDIDATAS = 12 cards. Each text is looked up as an exact English name (limit 4, with include:extras -layout:art_series -layout:front_card), and if that is empty, as a name fragment in any language (search_multilingual, limit 6, one result per card).
8. No candidates at all. The text may be a name printed in another language for a card Scryfall has no printing in that language (old Spanish cards). The first two texts (hint, title) go through fuentes::nombres_ingleses using the footer's language (default es), and those English names are searched.
9. Typed searches (escrito): for each hint, every card with the text anywhere in its name (any language), up to MAX_ESCRITO = 700 (four Scryfall pages), ordered by ordenar_por_lo_escrito: equal name, then name starting with it, then a word starting with it, then the rest; whole-name matches beat single-face matches ("Treasure" before "Dinosaur // Treasure"). So typing "waste" lists Wastes, Wasteland, Lay Waste...
10. Cockatrice tokens are added as candidates when their name equals something read (seguros, hint, title), and, for typed searches, when the name merely contains the hint.
11. Scoring (puntuar): see below. Approximate hits do not score on their own; the proposal is measured against what was actually read, so a misread letter comes out as Probable and gets reviewed.
12. Fallbacks. If confidence is None, the exact printing from the footer (del_pie) becomes the proposal as Probable (a footer can be copied from a template, so never Exact); failing that, the first token found becomes Probable (several versions of every token exist).
13. The proposal is moved to the front of candidates. lines = footer names followed by title lines (shown to the user as "Read on the card"); pie is the parsed footer.

Scoring and confidence (puntuar)

For each candidate it builds the names to compare: name and printed_name ("whole" names) and, for A // B cards, each face (not "whole"). Each is normalizard (lower case, accents and punctuation removed) and compared with every normalised text of 3+ characters using parecido: 1 - Damerau-Levenshtein / max(len), where swapping two adjacent letters counts as one error ("Menmarch" for "Memnarch", a typical mistake when writing a name by hand). The best similarity wins; on a tie a whole-name match beats a single-face match ("Lightning Bolt" beats "Emeritus of Conflict // Lightning Bolt").

Best similarity	Confidence	Meaning
>= 0.999	Exact	Normalised name equals a text.
>= 0.85 (UMBRAL_PROBABLE)	Probable	One misread letter, accent, small difference.
lower	None	Candidates exist but none is close; best = null.

OCR (ocr.rs)

• Engine. The pure-Rust ocrs crate on the rten runtime: no system libraries. Two models, text-detection.rten and text-recognition.rten (about 12 MB together), are not bundled. They are downloaded on first use to app_data_dir/modelos/ (descargar_modelos, written with escribir_atomico) and stay there; modelos_listos checks both exist and are non-empty. The app needs them for nothing else. ocr_ready (command) lets the UI warn about the first-time download ("about 12 MB, works offline afterwards").
• Gotcha: the models come from a third-party storage host with plain reqwest, not through fuentes::Externo, and without a hash check (External-Sources).
• One engine per process. MOTOR: OnceLock<OcrEngine>; two threads racing initialise it twice and one result is dropped.
• Reading. leer_carta(datos, img) reads the whole card at original resolution (not just a title strip: small footer text needs the pixels). lineas() runs detect words, find text lines, recognise text, and returns LineaLeida { texto, y, alto } where y is the line's vertical centre and alto its glyph height, both as fractions of the card height (0 = top).
• Zones (repartir). Constants FRANJA_TITULO = 0.16 and FRANJA_PIE = 0.94:
  • y < 0.16: title candidates (cleaned, kept if parece_un_nombre: at least 3 letters, a vowel, not a short all-caps string, not a short mixed-case blob like "OOofC", which filters mana-symbol and set-symbol noise).
  • y > 0.94, or the line looks like a footer (parece_pie: a collector number with 2+ digits followed by a single capital rarity letter, e.g. 0057 S Menmarch): footer. nombre_del_pie strips the number and rarity, ignores lines with no lowercase letters (set code and language such as RTS ES) and copyright lines (wizards), and ignores a lone "Token".
  • everything else: resto (rules text, type line...).
  • The type line (linea_de_tipo) is the line between the two bands made only of type words (Spanish and English, PALABRAS_DE_TIPO), closest to mid-card (y = 0.55). ficha is set when the type line, a footer-like line, or the start of a line says "Token"/"Ficha"; the word "ficha" in rules text ("non-token") is deliberately ignored.
• Lectura.lineas keeps every line unfiltered, top to bottom, for originales.

Finding the original name (originales.rs)

Authors put the English original name in many places: under the title, at the bottom left or right, after the artist (NEO • SP ➤ ARTIST | Mirror Box), in parentheses on the type line, in small caps, or split over two lines. So nothing is assumed: all pieces of all lines are tried against the local name index and each hit is scored by where and how it appears.

buscar(lineas, indice) -> Vec<Acierto { nombre, puntos, aproximado }>:

1. Each line gets a zone: Titulo, Pie, Tipo or Reglas (same thresholds as above). Adjacent lines are also merged into one candidate text when they are in the same non-rules zone, have glyph heights within 25 % and are close (gap up to 1.8 line heights): this catches names split over two lines.
2. Each text is cut at separators (SEPARADORES: | • · — / ( ) [ ] « » : ; ★ © ➤ ... and triple spaces) because a name never spans them; chunks containing "wizards" (copyright) are dropped.
3. For every chunk, every window of 1..=6 consecutive words is checked with Indice::exacto (case/accents/punctuation-insensitive, OCR-confusable letters folded). A single-word hit counts only if it is the whole chunk and not in the rules text (so "Fog" in a sentence is not the card name).
4. Points: multi-word hit 3 + 0.5 * words; single word 1; +1 if the window is the whole chunk; zone bonus +2 title or footer, +1 type line, -2 rules text; +0.5 if the line's glyphs are under 70 % of the title height (small print). Only hits with >= PUNTOS_SEGURO (3.0) are kept; per name the best score is kept.
5. If no multi-word name was found, a fuzzy pass (Indice::parecido, similarity >= 0.85) tries up to 6 multi-word chunks from title/footer zones; these hits carry 3.0 points and aproximado = true.
6. Results are sorted by points, best first.

pie_leido(lineas, sets) -> Option<PieLeido { set, numero, idioma }> reads the footer: it scans lines with y > 0.85 or footer-looking text; the first token that is a collector number (up to 4 digits, optionally 250/302; leading zeros removed) becomes numero; the first all-uppercase 2-7 character token that is a known set code becomes set, and the following word is the language (idioma_del_pie: EN, ES or SP, FR, DE, IT, PT, JP or JA, KO or KR, RU, CS->zhs, CT->zht, PH). A glued form like NEOSP (the OCR swallowed the bullet) is split. Both set and number are required; one alone does not identify a printing, so the result is None.

Commands and events

Command	Signature	Notes
ocr_ready	() -> bool	Are the OCR models on disk?
card_identify	(blob, hint?, ocr) -> CardIdentify	One card; escrito = true. Downloads models first if ocr and missing.
identify_batch	(items: [{blob, hint?}], ocr) -> job_id	Starts a job. Events: identify://progress (IdentifyProgressEvent { jobId, done, total, blob, result? }, one per card) and identify://done (IdentifyDoneEvent { jobId, error? }). Cancelled with job_cancel.
card_link	(scryfall_id) -> CardMeta	See above.
card_link_externa	(fuente, id) -> CardMeta	For Cockatrice tokens.

Batch behaviour (lote): models are downloaded once before the loop; for each item identificar(..., escrito = false) runs; a failure on one item emits a progress event with result = None and the job continues; a Network error aborts the whole job (no point continuing offline); the cancel flag is checked before each item. The command holds a state.trabajo() guard like other long jobs.

Front-end flows

Single card: VincularDialog

Opened from the card menu ("Link with a card..." / "Change the link...", src/features/preview/slot-menu.tsx). It shows the card image, a search box pre-filled with the card's name, and a grid of candidates (RejillaCartas). On open it runs a name-only identification when the name has 3+ characters. Buttons: "Search" (name-only card_identify, ocr = false) and "Detect with OCR" (ocr = true, shows a note about the 12 MB first-time download when ocr_ready is false). After OCR, the "Read on the card" list shows the lines; clicking one searches for it. The proposal (best, else the first candidate) is preselected; double-clicking a candidate links immediately. The footer button "Link with " calls vincular(card.id, chosen, idiomaPie); "Remove the link" appears when scryfallId is set and calls desvincular. It closes with an announcement "Linked with ".

Bulk: VincularLoteDialog

Entry points: the decklist panel button "Link with a card" (whole deck, or the selection when several are selected), and "Link the selected..." in the multi-card menu (soloSeleccion). Phases:

1. Choose (elegir). If a selection contains own images, a toggle picks "Whole deck (n)" or "Only the selected (n)". Scope: "Only those missing" (not yet linked) or "All again". Checkbox "Read the whole card with OCR (more reliable, about 2 s per card)"; without OCR only the file name is used.
2. Read (leyendo). identify_batch runs; each result arrives as a row. A progress bar with a remaining-time estimate. "Stop and review what was read" cancels the job but keeps the rows already read.
3. Review (revisar). Counters (Exact / Probable / No proposal). Each row has: a checkbox, the own image thumbnail, the text read, a selector of candidates ("Do not link" plus name · SET number · lang), a confidence badge, and the candidate's thumbnail; either thumbnail opens a large comparison (AmpliarCartas). Rows with exact or probable confidence and a best start checked; the rest start unchecked and can be fixed by choosing another candidate (choosing one checks the row, "Do not link" unchecks it). A filter shows all rows or only unchecked ones.
4. Apply (aplicando). For each checked row it calls vincular(card.id, candidate, resultado.pie?.idioma) sequentially (each updates the deck store, so it lands in undo history like other edits; Inferred from useDeck.setCartas), with a "Stop" button. While reading or applying, the dialog is marked busy and closing it interrupts the work; with unsaved review results it asks for confirmation (cambiosSinGuardar).

The review step is the safety net: Rust only proposes; nothing is linked until the user confirms.

Screenshots (English UI):

Failure modes

Situation	Behaviour
First OCR use offline	descargar_modelos fails with Network; the single dialog shows the error toast, the batch ends with an error.
Engine/model load failure	ErrorCode::Internal with ocr param.
Blob missing	NotFound (blob).
Scryfall unreachable	Network; in a batch it aborts the job, in the single dialog it shows an error toast.
Nothing read, nothing matches	candidates empty or confidence = None; the UI shows "No results" / "No proposal"; the user can type a name.
Footer misread	pie = None; no exact-printing candidate; the language of the image is not auto-set.

Tests

• src-tauri/src/commands/identify.rs: parecido, exact/probable/none scoring, double-faced matching, whole-card preference, typed-order (lo_escrito_ordena_por_parecido), normalizar; an #[ignore]d live test over a folder.
• src-tauri/src/ocr.rs: noise rejection, footer names, token detection only in type line or footer, type lines, limpiar; a live test (en_vivo, #[ignore]d, needs the models via PB_OCR_DATOS) that reads a real image.
• src-tauri/src/originales.rs: real-case reading of a custom card, original name in each position (footer, under the title, type-line parentheses, small caps, split lines, one wrong letter), rules-text words not counted, footer set/number/language, a speed guard (many lines against 30 000 names in under 500 ms).
• src-tauri/src/scryfall/nombres.rs, fichas.rs: the index and token-name table.
• src/features/variants/__tests__/vincular.test.ts: linking all same-image copies, not taking the Scryfall image state, unlink, error leaves the deck unchanged, propiasPorImagen.

Where to start when extending

• Better recognition of a new card layout: add a case to originales.rs tests with the lines the OCR really produced, then adjust scoring; keep the rule "a single word counts only as a whole chunk outside the rules text".
• A new source of candidates: add them in identificar() before puntuar, as CardSummary rows (set fuente if not from Scryfall).
• Changing thresholds: UMBRAL_PROBABLE (identify), UMBRAL (name index fuzzy match), PUNTOS_SEGURO (originales); the bulk dialog pre-checks both Exact and Probable, so loosening them affects automatic linking.

See also

• Scryfall-Integration - search, caches and the name catalogue.
• External-Sources - Cockatrice tokens and other-language name sources.
• Deck-Model - CardSource, CardMeta.
• Statistics - why linked cards matter.
• Add-Card-and-Variants - the sibling dialogs in features/variants.
• Settings-and-Shell, IPC-and-Error-Handling