A CC0 point-in-time reference set, and three XBRL traps that cost me weeks #946
christianpichichero-max
started this conversation in
Show and tell
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
edgartools does the thing most EDGAR wrappers skip — resolving duplicate facts and precision rather than taking the first match that parses. I ran into the same class of problem from the data side and figured the artifact and the scar tissue might both be useful here.
I built a point-in-time sample off
companyfactswhere every row carries the value as first filed next to the current value, plus the date it became public. Public domain, no signup:https://github.com/christianpichichero-max/pit-fundamentals
Two things that were genuinely painful, in case they save someone else the week:
1. Unit selection is a silent killer.
unitsis a dict, and iterating it gives you whatever comes first in the JSON. ForEarningsPerShareDilutedthat can hand you aUSDentry when you wantedUSD/shares. What comes back is a plausible dollar figure, so nothing looks wrong — I shipped Costco's FY2010 EPS as the current-year number and it passed every range check I had. The fix was declaring the expected unit per concept up front and failing loudly instead of falling back tonext(iter(units)).2. Anchoring instants to the fiscal close is inverted from what you'd guess. For balance-sheet concepts you want the instant matching the fiscal year-end the filing itself declares. The trap:
fpis"FY"on every instant in a 10-K, including interim ones, and the actual year-end instant usually has no frame at all. So the obvious "filter to the FY frame" is precisely backwards — it drops the value you want and keeps interim balances, which then get served as annual figures. That one shipped too.3. The one that cost me most, added today. Static tag priority cannot be right for every filer.
Revenuesis the consolidated total for MetLife ($71.0B, against $2.2B of ASC-606 contract revenue in the same 10-K) and a minor slice for General Mills ($2.038B, against $19.857B of contract revenue in the same 10-K). RankingRevenuesfirst fixed one and silently broke the other by 10x — and the wrong number still looked like money, so nothing caught it. Resolving by magnitude within the filing gets both right, since every candidate element is meant to be a consolidated total.Also worth knowing: the largest apparent "restatements" aren't accounting errors, they're retroactive split adjustments. Amazon's FY2020 diluted share count reads 510,000,000 as originally filed and 10,198,000,000 in companyfacts today. Both correct, different denominators — and careful filing-date handling doesn't catch it, because the date is right.
The sample is 40 large caps, 7 concepts, annual 10-K only — a reference set, not a universe. If a fixture of "value as first filed vs value today, with real restatements in it" is useful for testing the facts layer, take it. CC0, no attribution needed.
Disclosure: I sell a commercial version of this at tradevodata.com. Nothing here is gated and I'm not asking for anything — posting because the bugs above are the kind that pass every sanity check, and this repo's users are exactly who'd hit them.
All reactions