When people ask how accurate a card scanner is, they usually mean one thing, but the question actually has two different answers depending on what you care about. Understanding the difference matters if you're trying to evaluate whether a system is good enough for your use case.
Card accuracy vs. edition accuracy
Card accuracy measures whether the system identified the correct card — meaning the same illustration, the same name. "That's a Sol Ring" is a card-level identification.
Edition accuracy measures whether the system identified the correct printing — the exact combination of card, set, and frame treatment. "That's the Commander 2018 Sol Ring" is an edition-level identification. This is what Scryfall's UUID distinguishes.
For hobbyist use — cataloging a collection, identifying what you pulled from a pack — card accuracy is often enough. You want to know what the card is and roughly what it's worth, and most printings of the same card are worth similar amounts.
Edition accuracy starts mattering for pricing. A card with dozens of printings can vary from $0.25 for a recent precon version to $30+ for an original set printing, even sharing the same artwork. Getting the edition wrong means getting the price wrong, sometimes significantly.
It also matters for collection tracking. If you have three copies of Sol Ring and want to know which sets they're from, card accuracy doesn't help you — you already know they're Sol Rings. You need edition accuracy to distinguish them.
Oracle accuracy
Scryfall has a concept called the oracle_id, which groups all printings of the same card regardless of set or art. A card search returning the correct oracle ID is what I'm calling "card accuracy" above — it means you got the right card but possibly the wrong printing.
The eval script in the repo reports both:
Edition 93.4% 94.8% 95.6% (top-1, top-3, top-5)
Card 97.2% 98.0% 98.4%
Card accuracy is always higher than edition accuracy, because getting the right card but wrong printing counts as a card hit but not an edition hit.
Where these numbers come from
The examples/eval_accuracy.py script runs the full pipeline against a directory of labeled images. Images are labeled by Scryfall UUID in the filename:
7286819f-6c57-4503-898c-528786ad86e9_sample.jpg
The script detects corners, dewarps, embeds, and searches, then checks whether the top-k results include the correct UUID (edition accuracy) or the correct oracle ID (card accuracy).
python examples/eval_accuracy.py my_cards/ --catalog hf://HanClinto/milo/scryfall-mtg
Why both numbers matter
A system with high card accuracy but low edition accuracy is useful for "what is this card" but not for pricing or inventory management. The gap between the two tells you something about how well the system distinguishes between printings of the same card.
For cards with very similar-looking printings — same artwork, similar frame treatment, only minor visual differences — that gap can be large. The most extreme case in Magic is Sol Ring; Scryfall currently shows 43 Sol Ring prints sharing Mike Bierek's artwork. That's covered in the next post.
When people ask how accurate a card scanner is, they usually mean one thing, but the question actually has two different answers depending on what you care about. Understanding the difference matters if you're trying to evaluate whether a system is good enough for your use case.
Card accuracy vs. edition accuracy
Card accuracy measures whether the system identified the correct card — meaning the same illustration, the same name. "That's a Sol Ring" is a card-level identification.
Edition accuracy measures whether the system identified the correct printing — the exact combination of card, set, and frame treatment. "That's the Commander 2018 Sol Ring" is an edition-level identification. This is what Scryfall's UUID distinguishes.
For hobbyist use — cataloging a collection, identifying what you pulled from a pack — card accuracy is often enough. You want to know what the card is and roughly what it's worth, and most printings of the same card are worth similar amounts.
Edition accuracy starts mattering for pricing. A card with dozens of printings can vary from $0.25 for a recent precon version to $30+ for an original set printing, even sharing the same artwork. Getting the edition wrong means getting the price wrong, sometimes significantly.
It also matters for collection tracking. If you have three copies of Sol Ring and want to know which sets they're from, card accuracy doesn't help you — you already know they're Sol Rings. You need edition accuracy to distinguish them.
Oracle accuracy
Scryfall has a concept called the
oracle_id, which groups all printings of the same card regardless of set or art. A card search returning the correct oracle ID is what I'm calling "card accuracy" above — it means you got the right card but possibly the wrong printing.The eval script in the repo reports both:
Card accuracy is always higher than edition accuracy, because getting the right card but wrong printing counts as a card hit but not an edition hit.
Where these numbers come from
The
examples/eval_accuracy.pyscript runs the full pipeline against a directory of labeled images. Images are labeled by Scryfall UUID in the filename:The script detects corners, dewarps, embeds, and searches, then checks whether the top-k results include the correct UUID (edition accuracy) or the correct oracle ID (card accuracy).
Why both numbers matter
A system with high card accuracy but low edition accuracy is useful for "what is this card" but not for pricing or inventory management. The gap between the two tells you something about how well the system distinguishes between printings of the same card.
For cards with very similar-looking printings — same artwork, similar frame treatment, only minor visual differences — that gap can be large. The most extreme case in Magic is Sol Ring; Scryfall currently shows 43 Sol Ring prints sharing Mike Bierek's artwork. That's covered in the next post.