Repository navigation
Identify From a Photo
English · Español
«Identify from a photo» recognises real Magic cards from a picture: which card it is and which printing. It is reached from the Card Search screen (the recognised card opens its details) and from the Add card dialog (the recognised cards go into the open deck with their copies). On desktop the person drops or picks photos; on Android a live scanner recognises cards without pressing anything and chains them, one after another, the way dedicated collection apps do.
It is a different job from linking an own image: there the picture is a proxy whose title may be invented; here it is a photo of a printed card, with a background, perspective and light, and the printing matters. Both share the OCR and the name proposal.
| File | Responsibility |
|---|---|
src-tauri/crates/proxy-core/src/foto/ |
Pure image work: detectar.rs (the card's four corners in a photo), enderezar.rs (homography and resampling to a flat 630 × 880 card), firma.rs (fingerprint of the illustration to compare printings), mod.rs (Punto, Esquinas, nitidez, shared helpers). |
src-tauri/src/reconocer/mod.rs |
From a photo to a Reconocida: reconocer (detect + straighten) and reconocer_enderezada (OCR, name, printing). |
src-tauri/src/reconocer/impresion.rs |
Which printing: the footer, or the closest illustration among the printings of the name (ordenar_por_imagen). |
src-tauri/src/reconocer/vivo.rs |
The live scanner: one-frame slot, worker thread, Seguimiento (when to read, when a card is new), JNI entry points. |
src-tauri/src/commands/reconocer.rs |
Commands reconocer_foto, reconocer_vivo_abrir, reconocer_vivo_cerrar, reconocer_vivo_linterna. |
src-tauri/src/commands/identify.rs |
leer_imagen (OCR of a card) and proponer (the name proposal), shared with linking. |
src-tauri/plugins/android-extra/android/…/EscanerCartas.kt |
The Android camera: CameraX preview behind the transparent WebView and frames for Rust. |
src-tauri/plugins/android-extra/android/…/Nativo.kt |
The JNI bridge object (comprobar, quiere, fotograma). |
src/stores/reconocer-store.ts |
useReconocer: open dialog or scanner, mode (buscar or deck), the tray, the queue of photos. |
src/features/reconocer/cuentas.ts |
Pure tray logic: start, arrive (same card adds a copy), choose, quantity, what to add. With tests. |
src/features/reconocer/reconocer-dialog.tsx |
The dialog (desktop and gallery), drop zone, elegirFotos. |
src/features/reconocer/escaner-vivo.tsx |
The live scanner screen (Android). |
src/features/reconocer/visor.ts |
Pure geometry of the scanner overlay (marcoGuia, encajar). With tests. |
src/features/reconocer/bandeja.tsx |
One row per photo: straightened photo next to the proposed printing, how it was known, a selector to correct it, copies or «View card details». |
src/features/reconocer/anadir.ts |
anadirAlDeck: inserts the chosen cards like «Add card» does. |
src/features/reconocer/boton-reconocer.tsx |
The «Identify from a photo» button (scanner on a phone, dialog on desktop). |
flowchart TD
F["photo (file, content:// or camera frame)"] --> D["foto::detectar<br/>four corners or none"]
D -- corners --> E["foto::enderezar → 630 × 880"]
D -- none --> W["whole photo stretched to 630 × 880"]
E --> O["leer_imagen: OCR (title, footer, rest)"]
W --> O
O --> P["proponer: name index, footer, Scryfall"]
P --> Q{"anything?"}
Q -- no --> R["rotate 180° and read again"]
R --> P
Q -- yes --> I["impresion::elegir"]
I --> A{"footer set + number<br/>of the same card?"}
A -- yes --> X["exact printing (por = pie)"]
A -- no --> B["printings of the name<br/>compare illustrations (por = imagen)"]
B -- no images / no network --> N["name only (por = nombre)"]
Like a document scanner, on a 480 px grey copy of the photo (block average, so a 12 MP photo costs a few milliseconds):
- Gaussian blur (
imaging::blur::gauss, σ 1.2) and Sobel gradient magnitude. - The strongest 14 % of edges (but never weaker than 24, so a smooth photo does not invent edges), dilated one pixel to close gaps.
-
rten_imageproc::find_contours(the same crate the OCR already brings). - Each contour covering 8–99.5 % of the photo: convex hull, then Douglas-Peucker with a growing tolerance until four vertices remain. Douglas-Peucker always keeps its first point, so the hull is rotated to start at the point farthest from the centre (a corner); otherwise a point mid-side survives and a corner is cut. If no four-vertex polygon appears (a finger over a corner, very round sleeves), the minimum-area rectangle is used when the hull fills at least 85 % of it.
- The quadrilateral must have the proportion of a card (63 / 88 = 0.716) within 20 %, measured without the band the blur and dilation add around the contour (
BANDA, 2 px of the reduced image per side: with it every rectangle looks squarer). The tolerance is wide on purpose: a card seen from below in perspective reaches 0.84 (18 %); with a tighter 12 % the outer edge was rejected and the inner text box won. A 16:9 screen (a TV, a phone: 21 %) and a 2:1 box are rejected; on the emulator's virtual camera the TV used to pass as a card. - The largest that passes wins.
Corners are ordered clockwise starting at the card's top-left: the first side is one of the short ones, the one higher in the photo (or the left one when the card lies sideways). An upside-down card comes out rotated 180°; the reader fixes that by reading again rotated (girar180).
A four-point homography (8 × 8 system, Gaussian elimination with partial pivoting) maps the flat card to the photo; every output pixel is sampled bilinearly, with 2 × 2 or 3 × 3 samples per pixel when the card in the photo is much larger than 880 px tall (small footer text survives). Output: ANCHO × ALTO = 630 × 880. With no corners, the whole photo is stretched to that size (it may already be a scan).
leer_imagen and proponer are the two halves of the old identificar in commands/identify.rs, split so that linking and photos share them (linking behaves exactly as before). proponer runs with no hint and escrito = false. A real card's English name is found through the local name index (originales::buscar), a printed name in another language through Scryfall's multilingual search, and the footer (pie_leido) gives set, number and, on recent cards, the language.
If the first reading gives nothing (no confident name, no footer) the card was probably upside down: it is rotated 180° and read again, keeping whichever reading gave more.
-
Footer. If the footer gives set and number,
card_lang(set, number, language or en)fetches that exact printing; it is used only if it is the same card as the proposed name (a misread footer can point elsewhere).por = pie, confidenceExact. -
Illustration. Otherwise the printings of the name are fetched (
consulta_impresiones, the query ofcard_prints, in the language of the proposal, or English if that language has none). They are grouped byillustration_id(a new optional field ofScryCard; old cached cards without it simply count each printing as its own group) and the firstMAX_ILUSTRACIONES = 40groups are compared: thesmallimage of one printing per group is downloaded (four at a time) intoapp_cache_dir/reconocer/<id>.jpg, itsFirmais computed and compared with the photo's. Printings are sorted by distance; those not compared go last, in Scryfall's order.-
Threshold. The best illustration only counts if its distance is at most
UMBRAL_IMAGEN = 0.30. With real cards the right art lands around 0.11 and the others around 0.5; a proxy, or an art Scryfall does not have, gives about 0.5 for everything. Above the threshold the result ispor = nombre(the list stays sorted, but the person chooses). -
The frame, within the winning art. Many printings share one illustration (Lightning Bolt by Christopher Moeller is in M10, M11, A25, 2X2, Jumpstart…) and their art fingerprints are identical.
afinar_marcoreorders that block: printings in the card's language first (the language of the proposal or of the footer: the frames of one printing are the same in every language), and among them the one whose whole card (Firma::de_zona(carta, ENTERA)) is closest to the photo, comparing one printing per set and number, at mostMAX_MARCOS = 12.por = imagen.
-
Threshold. The best illustration only counts if its distance is at most
-
Name only. Without network or images, or when no illustration passes the threshold, the proposal is the proposed card itself.
por = nombre.
A card from another source (a Cockatrice token) has no printings: it is returned as is.
The illustration window, in fractions of the card (ARTE: x 0.08–0.92, y 0.11–0.55, valid for modern and old frames and also for borderless art), gives two things:
- a dHash of 16 × 16 (each zone brighter than its right neighbour), robust to brightness, contrast, JPEG and slight blur;
- the mean colour of a 6 × 4 grid in CIE Lab, with the mean lightness subtracted (room light changes brightness more than hue).
distancia = 0.5 · Hamming/256 + 0.5 · min(mean ΔE / 40, 1). The size does not matter: Scryfall's 146 × 204 small and the 630 × 880 photo give the same fingerprint within 0.12.
| Command | Signature | Notes |
|---|---|---|
reconocer_foto |
(ruta) -> Reconocida |
Reads the file through fichero.rs (an Android content:// too, up to 80 MB), decodes it (EXIF orientation applied) and runs the pipeline. |
reconocer_vivo_abrir |
(canal: Channel<EstadoEscaner>) -> () |
Android only (Unsupported, motivo: camara elsewhere). Downloads the OCR models if missing, starts the worker, opens the camera (asking for permission) and keeps the screen on (UsoPantalla::Escaner). |
reconocer_vivo_cerrar |
() -> () |
Stops the worker, closes the camera, lets the screen sleep. Safe when nothing is open. |
reconocer_vivo_linterna |
(si) -> bool |
Torch on or off; false if there is no flash. |
Reconocida { esquinas?, miniatura, lectura, candidatas, mejor?, confianza, impresiones, por }: esquinas in fractions of the photo (null = whole photo used), miniatura a ~300 px JPEG data: URL of the straightened card, impresiones a list of { carta, distancia? } best first, por one of pie, imagen, nombre, nada.
Errors: camera without permission is Unsupported with motivo: permiso_camara, no usable camera is Unsupported with motivo: camara (both translated in lib/errores.ts); image problems come through the usual ImagingError conversion (ErrorCode::Image).
sequenceDiagram
participant K as EscanerCartas.kt
participant R as reconocer::vivo (Rust)
participant P as escaner-vivo.tsx
P->>R: reconocer_vivo_abrir(canal)
R->>K: escanerAbrir (permission, CameraX)
loop every frame (≥ 120 ms apart)
K->>R: Nativo.quiere()?
K->>R: Nativo.fotograma(rgba, w, h, giro)
R->>R: rotate, detectar
R-->>P: viendo { esquinas, ancho, alto, leyendo }
alt card still for 2 frames, sharp, not read yet
R->>R: enderezar, OCR, proponer, impresion
R-->>P: reconocida { reconocida } (only if new)
end
end
P->>R: reconocer_vivo_cerrar
R->>K: escanerCerrar
-
The camera behind the page.
EscanerCartasadds a CameraXPreviewView(FILL_CENTER) to the WebView's parent, brings the WebView to the front and makes it transparent, like thewindowedmode of the QR scanner, with the same CameraX version (1.5.1) so Gradle does not mix two. The scanner UI is React: while it is openhtml[data-escaner]makeshtmlandbodytransparent and hides#root(without unmounting it,index.css); the scanner lives in a portal outside#root, so dialogs (a card's details) still appear above it. CameraX is bound to the activity's lifecycle and stops by itself in the background. -
Frames without waste.
ImageAnalysisinRGBA_8888, keep-only-latest, about 1280 × 720. Before copying a frame Kotlin asksNativo.quiere(); Rust says yes only when its single slot is empty and the worker is idle. Rows are copied without their stride padding and handed over withNativo.fotograma(rgba, ancho, alto, giro), at most one every 120 ms. A newer frame overwrites an untaken one: there is never a queue. -
Why JNI and not IPC. A frame is about 3.7 MB; through the WebView IPC (JSON or base64) that would be unworkable, and
getUserMediais not wired in Tauri's Android WebView (RustWebChromeClientdoes not answeronPermissionRequest). JNI hands the bytes straight to the worker; the screen only gets small messages through aChannel. -
When to read (
Seguimiento). Pure and tested with corner sequences:- a card counts as still when its corners move less than 2 % of the frame diagonal (
QUIETA) for 2 frames (ESTABLE); - a jump above 15 % (
SALTO) is another card (or a big move): it may be read again; - 6 frames with no card (
SALIDA) means it left: the same card counts again if it comes back; - a read that gives nothing waits 3 frames (
REINTENTO); a blurry straightened card (foto::nitidezunder 20) waits 1; - a card already read is not read again while it stays in view, and a reading that gives the card already in view is not sent.
- a card counts as still when its corners move less than 2 % of the frame diagonal (
-
The screen. Top bar with close, torch and gallery; the guide frame (a card shape, 75 % of the space) and the detected outline (amber while reading); a status pill («Point at a card…», «Hold still…», «Reading…», «Recognised: X»); the tray at the bottom (on the side in landscape) with the newest first and «Add to the deck» in deck mode. A recognised card vibrates briefly. Android's back button leaves the scanner: with something in the tray it goes to the dialog (nothing is lost), otherwise it closes. Without permission it explains and offers Android settings (
abrirAjustesDeLaCamara) or the gallery. -
Overlay geometry (
visor.ts): the preview fills the screen cropping what does not fit, soencajarscales the frame bymax(W / ancho, H / alto), centres it and subtracts where the overlay box starts.
The dialog and the scanner share useReconocer and the same tray, so switching between them keeps it.
- A photo enters as «reading» (
empezar), then arrives (llega). If another row already has the same chosen card it adds a copy there and the new row disappears (scanning the same card twice means two copies). A row with no proposal does not count until a card is chosen. - Each row shows the straightened photo beside the proposed printing's thumbnail (to compare), how it was known (exact by footer, by illustration, name only, «check it» for
Probable, «choose the card» forNone) and a selector with the printings first (up to 40) and then the other proposed cards. - Search mode: «View card details» opens the card details above (the scanner stays open).
-
Deck mode (from «Add card», which closes first): copies per row and «Add to the deck».
anadirAlDeckfetches each card withtraerCartaonce, clones the extra copies with a fresh id, inserts them at the position the dialog was opened from (and in place of the blank slot, if any) in one undo step and announces «N cards added to the deck». - Photos are read one at a time (a promise queue in the store); closing empties the tray and stops reading the rest. The dialog asks before closing while reading, and in deck mode with cards not yet added.
-
Dropping files: the dialog listens to
onDragDropEventwhile open; the global drop handler of the preview (useArrastrarFicheros) does nothing while it is open, so dropped photos are not also added as own cards.
On a desktop CPU (release build), for a Spanish card rotated 12° on a noisy background in a 3000 × 2600 photo: detect and straighten 25 ms; OCR of the whole straightened card 185 ms median (title strip alone 82 ms, footer strip 108 ms). The title and the printed English name are read correctly. The whole chain against Scryfall (first run, with the downloads) takes 2–4 s: a Lightning Bolt photographed from M10 gives exactly M10 146 by its illustration and frame; a custom proxy whose art Scryfall does not have stays «name only». A phone is several times slower; reconocer::tests::mide_el_ocr (#[ignore], PB_DATOS and PB_FOTO) measures it on a device.
-
proxy-core/src/foto/: detection straight, rotated (±15°, −25°), sideways (90°), upside down (170°), in perspective, in a 3000 px photo, no card, a 2:1 box and a 16:9 screen; homography round trip and degenerate points; straightening a perspective photo back within a mean error of 6/255; fingerprint: the same art as a worse photo is closer than another art, size does not matter, the whole card tells two frames of the same art apart; sharpness. All on synthetic photos built from a fixed generator (foto::pruebas). -
src-tauri/src/reconocer/: whole photo when no card is found, corners as fractions, thumbnail as JPEG data URL; printing ranking with a fakeCardApi(the photo's illustration wins, one download per illustration, cache on the second run, cards without image last, only 40 illustrations compared, the frame and the language decide within one art, nothing more is downloaded when no art is close); safe cache file names;Seguimientosequences (read when still, no re-read nor repeat while in view, counts again after leaving, another card in its place, retry wait); frame rotation; no frames wanted with the scanner closed. Two#[ignore]tests:mide_el_ocrandreconoce_una_foto_de_verdad(the whole chain against Scryfall in a temporary folder:PB_MODELOS,PB_FOTO). -
src/features/reconocer/__tests__/: the tray (cuentas.test.ts) and the overlay geometry (visor.test.ts);features/preview/__tests__/arrastrar-ficheros.test.tschecks the global drop does nothing while the dialog is open.
- Better detection on difficult backgrounds: add a synthetic case to
detectar.rstests, then tunePERCENTIL_BORDE,AREA_MINIMAorTOLERANCIA_PROPORCION. - Better printing choice:
ARTEand the weights inFirma::distancia;MAX_ILUSTRACIONESbounds downloads for cards with hundreds of printings (basic lands, Sol Ring). - Scanner feel: the constants at the top of
reconocer/vivo.rsandEscanerCartas.INTERVALO_MS. - Not done yet: moving the corners by hand when detection fails, a webcam on desktop, telling foil apart.
- Card Identification and OCR - the OCR and the name proposal it shares.
- Card Search and Add Card and Variants - where the button lives.
-
Mobile - the
android-extraplugin, JNI and release checks. - Scryfall Integration - printings and images.
-
Storage and Blobs -
app_cache_dir/reconocer/.
Use and build · Uso y desarrollo
- User-Guide · Guía de usuario
- Development-Setup · Preparación del entorno de desarrollo
- Testing-and-CI · Tests y CI
- Releases-and-Auto-Updates · Versiones y actualizaciones automáticas
- Conventions · Convenciones
- Rename-to-Tapete · Cambio de nombre a Tapete
- License · Licencia
Architecture · Arquitectura
- Architecture · Arquitectura
- IPC-and-Error-Handling · IPC y manejo de errores
- Rust-App-Layer · Capa de aplicación Rust
Data · Datos
- Deck-Model · Modelo del mazo
- Storage-and-Blobs · Almacenamiento y blobs
- Deck-Package-Format · Formato del paquete de mazo
- Deck-Library · Biblioteca de mazos
Getting cards · Conseguir cartas
- Decklist-Parsing · Parseo de la lista del mazo
- Scryfall-Integration · Integración con Scryfall
- External-Sources · Fuentes externas
- Card-Identification-and-OCR · Identificación de cartas y OCR
- Identify-From-a-Photo · Identificar por foto
Layout, images, PDF · Maquetación, imagen y PDF
- Layout-Engine · Motor de maquetación
- Imaging-and-PDF · Imagen y PDF
- Export · Exportación
- The-Card-Stamp · El sello de las cartas
- Live-Preview · Vista previa en vivo
- Statistics · Estadísticas
Card designer · Diseñador de cartas
- Designer-Overview · Visión general del diseñador
- Designer-Engine · Motor del diseñador
- Designer-Composition · Composición del diseñador
- Frame-Packs · Packs de marcos
- Designer-App-Layer · Capa de aplicación del diseñador
- Designer-UI · Interfaz del diseñador
- Card-Editor-Dialog · Diálogo Editar carta
Game table · Mesa de juego
- Game-Engine · Motor de la mesa
- Game-Table · Mesa de juego
- Online-Play · Juego en línea
- Friends-and-Messages · Amigos y mensajes
Mobile · Móvil
Frontend · Frontend
- Frontend-Architecture · Arquitectura del frontend
- State-Stores · Stores de estado
- Internationalization · Internacionalización
- Themes-and-Skins · Temas y skins
- Icons · Iconos
- Settings-and-Shell · Ajustes y shell
- Tutorials · Tutoriales
- Decklist-Panel · Panel de la lista del mazo
- Add-Card-and-Variants · Añadir carta y variantes
- Card-Search · Búsqueda de cartas
- Set-Browser · Explorador de sets
Reference · Referencia