Skip to content

data(cpu): fill process node and L3 cache for 540 Xeon CPUs - #133

Merged
Seungpyo1007 merged 2 commits into
mainfrom
data/cpu-xeon-specs
Jul 29, 2026
Merged

data(cpu): fill process node and L3 cache for 540 Xeon CPUs#133
Seungpyo1007 merged 2 commits into
mainfrom
data/cpu-xeon-specs

Conversation

@Seungpyo1007

@Seungpyo1007 Seungpyo1007 commented Jul 29, 2026

Copy link
Copy Markdown
Member

What

540 Xeon records gain 1,037 previously-null fieldsprocess_node 493, l3_cache_mb 540, boost_clock_ghz 4 — from the per-generation List of Intel Xeon processors (<uarch>-based) pages.

Combined with #132, CPU process_node coverage moves 24% → 46% and l3_cache_mb 19% → 48%.

Why these pages needed different handling

The Xeon hub page is only an index of links, which is why the earlier run found 0 rows there. The real data lives on 16 per-generation sub-pages, and those do not use wikitext tables at all — each CPU is a template call:

{{cpulist|...|model=Xeon D-2141I|cores=8|threads=16|l3=11|turbo=2.7/3.0 GHz|tdp=65|date=February 2018}}

Values are read by parameter name, so the column-shift hazard that required so much guarding in #132 cannot occur here.

Correctness

  • A row is used only when the data already held agrees: core count exactly, and TDP within 1 W when both sides have it. 20 of 656 matches were dropped on those checks; 378 parsed rows had no counterpart and were ignored.
  • process_node comes from the page's microarchitecture, which is unambiguous per generation (Skylake 14 nm, Haswell 22 nm, Ice Lake 10 nm, Sandy Bridge 32 nm). Pages that mix generations — Core-based, Nehalem, P6 — were left without a node rather than guessed.
  • Spot-checks against known specs all pass: E5-2699 v4 → 14 nm / 55 MB, Platinum 8180 → 14 nm / 38.5 MB, E5-2670 → 32 nm / 20 MB, Gold 6248 → 14 nm / 27.5 MB, E3-1230 v5 → 14 nm / 8 MB.

Only empty fields are written, nothing existing is overwritten, verified is untouched, and the specific generation page is recorded in source_urls.

python -m app.validateData validation passed. Dump refreshed for the 540 changed pages only.

Closes #1

540 Xeon records gain 1,037 previously-null fields (process_node 493,
l3_cache_mb 540, boost_clock_ghz 4), sourced from the per-generation
"List of Intel Xeon processors (<uarch>-based)" pages.

Those pages do not use wikitext tables: every CPU is a {{cpulist|...}}
template call with named parameters (model=, cores=, l3=, tdp=, turbo=),
so values are read by name and the column-shift hazard of the table-based
pages cannot occur here. A row is still only used when the data already
held agrees -- core count exactly, and TDP within 1 W when both sides have
it (20 of 656 matches were dropped on those checks).

The process node comes from the page's microarchitecture, which is
unambiguous per generation (Skylake 14 nm, Haswell 22 nm, Ice Lake 10 nm,
Sandy Bridge 32 nm). Pages that mix generations -- Core-based, Nehalem,
P6 -- were left without a node rather than guessed.

Refs #1
Regenerates site/public/v1/cpus for the 540 records changed in the previous
commit, with the engine at the pinned submodule commit so scoring output is
unchanged. Only pages whose content actually differs are committed; the 7,415
pages that differed solely by created_at/updated_at are left alone.

Refs #1
@TechEngineBot

Copy link
Copy Markdown
Member

🔎 Data verification — Tier 0 (offline existence/trust)

Scored by app.verify; posted by TechEngineBot. Informational only — the structural gate (app.validate) is separate and authoritative for merge.

Changed records in this PR

540 record(s) scored.

%%{init: {"theme":"base","themeVariables":{"pie1":"#3fb950","pie2":"#d29922","pie3":"#f85149","pieStrokeWidth":"0px","pieOpacity":"1"}}}%%
pie showData
    title Verification bands — all records
    "Green" : 539
    "Yellow" : 0
    "Red" : 1
Loading
Category Distribution Total 🟢 🟡 🔴 🟢 %
cpu 🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟥 540 539 0 1 99.8%
All 🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟥 540 539 0 1 99.8%

Hard violations (forced red):

Count Check
1 !boost_ge_base

Full-dataset baseline

105742 record(s) scored.

%%{init: {"theme":"base","themeVariables":{"pie1":"#3fb950","pie2":"#d29922","pie3":"#f85149","pieStrokeWidth":"0px","pieOpacity":"1"}}}%%
pie showData
    title Verification bands — all records
    "Green" : 14312
    "Yellow" : 89517
    "Red" : 1913
Loading
Category Distribution Total 🟢 🟡 🔴 🟢 %
brand 🟩🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨 207 24 183 0 11.6%
soc 🟩🟨🟨🟨🟨🟥🟥🟥🟥🟥🟥🟥 2104 123 680 1301 5.8%
smartphone 🟩🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟥 93396 11602 81206 588 12.4%
tablet 🟩🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟥 3455 174 3266 15 5.0%
watch 🟩🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟥 433 11 415 7 2.5%
pda 🟩🟩🟨🟨🟨🟨🟨🟨🟨🟨🟨🟥 140 27 112 1 19.3%
gpu 🟩🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨 2030 245 1785 0 12.1%
cpu 🟩🟩🟩🟩🟩🟨🟨🟨🟨🟨🟨🟥 3977 2106 1870 1 53.0%
All 🟩🟩🟨🟨🟨🟨🟨🟨🟨🟨🟨🟥 105742 14312 89517 1913 13.5%

Hard violations (forced red):

Count Check
1 !boost_ge_base

green = authoritative source + complete + consistent · yellow = plausible, needs confirmation · red = sparse/weak source or a hard contradiction. Promotion to verified runs in the scheduled verify-network workflow.

@Seungpyo1007
Seungpyo1007 merged commit fd3b642 into main Jul 29, 2026
5 checks passed
@github-project-automation github-project-automation Bot moved this from In Progress to Done in TechAPI-Project Jul 29, 2026
@Seungpyo1007
Seungpyo1007 deleted the data/cpu-xeon-specs branch July 29, 2026 06:50
@TechEngineBot

Copy link
Copy Markdown
Member

TechEngine change review: PASS

Check Result
python -m app.validate PASS
python integrity_check.py TechAPI/data --strict PASS

Changed data

Category Added Modified Deleted Added verified Added unverified Added Kaggle-sourced
brand 0 0 0 0 0 0
soc 0 0 0 0 0 0
smartphone 0 0 0 0 0 0
tablet 0 0 0 0 0 0
watch 0 0 0 0 0 0
pda 0 0 0 0 0 0
gpu 0 0 0 0 0 0
cpu 0 0 0 0 0 0

Changed record examples

  • No data file changes detected.

Heuristic review

  • Heuristic warnings: none found.

@TechEngineBot

Copy link
Copy Markdown
Member

TechEngine validation stats: PASS

Data summary

Category Total Verified Unverified Missing verified Tracked Verified % of tracked
brand 207 10 197 0 207 4.8%
soc 2104 123 1981 0 2104 5.8%
smartphone 93396 8453 84943 0 93396 9.1%
tablet 3455 174 3281 0 3455 5.0%
watch 433 11 422 0 433 2.5%
pda 140 27 113 0 140 19.3%
gpu 2030 244 1786 0 2030 12.0%
cpu 3977 986 2991 0 3977 24.8%
all 105742 10028 95714 0 105742 9.5%

Warning

Tracked verified coverage is below 50% for watch 2.5% (11/433), brand 4.8% (10/207), tablet 5.0% (174/3455), soc 5.8% (123/2104), smartphone 9.1% (8453/93396), all 9.5% (10028/105742), gpu 12.0% (244/2030), pda 19.3% (27/140), and 1 more.
Tracked coverage excludes records missing the verified field; see the Missing verified column for those records.
This does not fail validation. Keep imported records verified: false until manual audit, but treat this as follow-up verification work before relying on the affected categories as curated data.

Validation notes

  • Full advisory outlier listings are suppressed on successful runs because they are dataset-wide and mostly stable between PRs.
  • Failure runs still include a detailed log excerpt for debugging.

Key output:

## app.validate
## integrity_check.py --strict
loaded CPU=3977 GPU=2030
✅ integrity gate: no hard anomalies.
Integrity section Flagged lines
structural 0
CPU name/tier consistency (desktop mainstream only) 0
CPU single>multi (cinebench/geekbench — should be multi>=single) 0
CPU era-vs-score outliers 8
CPU cross-source ratio outliers (possible wrong-variant) 152
GPU cross-source ratio outliers + sanity 18

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

data Dataset changes enhancement New feature or request

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

Massive dataset rebuild: CPU + brand + GPU + smartphone + SoC (1989-2026)

2 participants