data(cpu): fill process node, L3 cache and boost clock for 581 CPUs - #132
Conversation
581 CPU records gain 982 previously-null fields (process_node 400, l3_cache_mb 580, boost_clock_ghz 2), sourced from the Wikipedia "List of Intel Core desktop/mobile processors" and AMD list pages. Only empty fields are written; nothing existing is overwritten and `verified` is untouched. Each parsed row was accepted only when the data already held for that SKU agreed with it -- base clock within 0.05 GHz and TDP within 1 W -- because these tables do not share a column layout and a positional read silently shifts values (an early pass produced "L3 = 95 MB" from a TDP column and "TDP = 2011" from a release-year column). Rows that disagreed were dropped rather than guessed. MSRP was deliberately not imported: Wikipedia lists launch price, which is not what msrp_usd holds. Refs #1
🔎 Data verification — Tier 0 (offline existence/trust)Scored by Changed records in this PR581 record(s) scored. %%{init: {"theme":"base","themeVariables":{"pie1":"#3fb950","pie2":"#d29922","pie3":"#f85149","pieStrokeWidth":"0px","pieOpacity":"1"}}}%%
pie showData
title Verification bands — all records
"Green" : 581
"Yellow" : 0
"Red" : 0
Full-dataset baseline105742 record(s) scored. %%{init: {"theme":"base","themeVariables":{"pie1":"#3fb950","pie2":"#d29922","pie3":"#f85149","pieStrokeWidth":"0px","pieOpacity":"1"}}}%%
pie showData
title Verification bands — all records
"Green" : 13773
"Yellow" : 90057
"Red" : 1912
green = authoritative source + complete + consistent · yellow = plausible, needs confirmation · red = sparse/weak source or a hard contradiction. Promotion to |
Regenerates site/public/v1/cpus for the 581 records changed in the previous commit, using the engine at the pinned submodule commit so scoring output is unchanged. Only pages whose content actually differs are committed -- app.dump rewrites created_at/updated_at on every record, and those 7,374 timestamp-only pages are left alone. Refs #1
TechEngine change review: PASS
Changed data
Changed record examples
Heuristic review
|
TechEngine validation stats: PASS
Data summary
Warning Tracked verified coverage is below 50% for watch 2.5% (11/433), brand 4.8% (10/207), tablet 5.0% (174/3455), soc 5.8% (123/2104), smartphone 9.1% (8453/93396), all 9.5% (10028/105742), gpu 12.0% (244/2030), pda 19.3% (27/140), and 1 more. Validation notes
Key output:
|
…kipped Recovers records that the merged batches (#132-#135) left untouched because of three separate mismatches in how the pages name things, not because the data was unavailable: - model codes are not always digit-first (Core 2 uses E6600, Atom uses N270), and the table parser required a leading digit; - the Atom page is built entirely from {{cpulist}} templates and had never been run through that parser; - the Opteron page lists a bare model-number column with no branding column, so the family name now comes from the page itself. Same confirm-before-write rule as before (base clock within 0.05 GHz, TDP within 1 W, L3 must be a real capacity and must not equal the row's TDP): 1,610 rows parsed, 1,053 matched a record, 885 passed, 210 still had a gap to fill. Verified against known parts: Clarkdale i5-6xx/i3-5xx 32 nm, Haswell i7-4770R 22 nm, Broadwell i7-5775R 14 nm, Atom 330/D410 45 nm, Atom D2500 32 nm, Opteron Shanghai 6 MB L3. Older families whose rows failed the cross-check (Core 2 E-series, Atom N270, early Opteron) were left empty rather than guessed. Refs #1
What
581 CPU records gain 982 previously-null fields —
process_node400,l3_cache_mb580,boost_clock_ghz2 — from the Wikipedia CPU list pages (Intel Core desktop/mobile, AMD lists).Coverage moves:
process_node24% → 34%,l3_cache_mb19% → 34%.Why this source
3,001 of 3,977 CPUs are bulk stubs: schema-complete, values null, and their
architectureholds a brand family ("Xeon", "Core") rather than a microarchitecture — so the node is not derivable inside the dataset (Xeon alone spans 250 nm to 5 nm). An external, citable source was required. The Wikipedia list pages carry the codename + lithography in the section heading and L3/clocks per row, and are not bot-blocked.How correctness was enforced
These tables do not share a column layout (some carry an extra iGPU / GPU-clock column), so a positional read silently shifts values. An early pass produced
L3 = 95 MB(actually the TDP column),TDP = 2011(the release year), andL3 = 4600(the "HD 4600" iGPU). Guards:msrp_usdholds (120 of 129 disagreed; e.g. our i7-860 = $44). Different semantics, not an error to "fix".Spot-checks against known specs pass: i7-2600K → 32 nm / 8 MB, i3-2100 → 32 nm / 3 MB, i7-4790K → 22 nm / 8 MB / 88 W.
Only empty fields are written; nothing existing is overwritten,
verifiedis untouched, and the sourcing Wikipedia page is appended tosource_urls.python -m app.validate→ Data validation passed.Closes #1