Skip to content

fix(site): show full dataset history, not just the last 7 commits#85

Merged
Seungpyo1007 merged 1 commit into
mainfrom
fix/history-timeline
Jun 23, 2026
Merged

fix(site): show full dataset history, not just the last 7 commits#85
Seungpyo1007 merged 1 commit into
mainfrom
fix/history-timeline

Conversation

@Seungpyo1007

@Seungpyo1007 Seungpyo1007 commented Jun 23, 2026

Copy link
Copy Markdown
Member

Problem

The homepage History section showed only the last 7 commits — older syncs were invisible. It called the GitHub commits API live (per_page=8slice(7)) and re-fetched the dump at every SHA, which also risked the unauthenticated rate limit (60/h).

Fix

Precompute the whole timeline at deploy time:

  • site/scripts/build-history.mjs walks the full git history of site/public/v1/index.json and writes site/public/v1/history.json (build-only, gitignored — regenerated every Pages deploy, never churns data PRs).
  • The page reads that one static file; it only falls back to the GitHub API for local astro dev.
  • deploy-pages checks out at fetch-depth: 0 so the generator can see old commits.

Verified locally: generator emits 27 points (1,870 records on 2026-06-01 → 101,879 on 2026-06-21) and the file ships to dist/v1/history.json. Full npm run build passes.

Refs #1

Closes #19

The homepage History section called the GitHub commits API live (per_page=8,
sliced to 7) and re-fetched the dump at each SHA, so older syncs were invisible
and the unauthenticated rate limit (60/h) made it flaky.

Precompute the whole timeline at deploy time instead: build-history.mjs walks the
full git history of site/public/v1/index.json and writes site/public/v1/history.json
(a build-only, gitignored artifact). The page reads that one static file and only
falls back to the GitHub API for local `astro dev`. deploy-pages now checks out at
fetch-depth: 0 so the generator can see old commits.

Refs #1
@github-actions github-actions Bot added bug Something isn't working ci CI and workflow changes enhancement New feature or request site Homepage and public site changes labels Jun 23, 2026
@Seungpyo1007 Seungpyo1007 moved this from Todo to In Progress in TechAPI-Project Jun 23, 2026
@Seungpyo1007 Seungpyo1007 merged commit 1fde014 into main Jun 23, 2026
2 checks passed
@Seungpyo1007 Seungpyo1007 deleted the fix/history-timeline branch June 23, 2026 10:48
@github-project-automation github-project-automation Bot moved this from In Progress to Done in TechAPI-Project Jun 23, 2026
@TechEngineBot

Copy link
Copy Markdown
Member

TechEngine change review: PASS

Check Result
python -m app.validate PASS
python integrity_check.py TechAPI/data --strict PASS
cd TechAPI/site && npm ci && npm run build PASS

Changed data

Category Added Modified Deleted Added verified Added unverified Added Kaggle-sourced
brand 0 0 0 0 0 0
soc 0 0 0 0 0 0
smartphone 0 0 0 0 0 0
tablet 0 0 0 0 0 0
watch 0 0 0 0 0 0
pda 0 0 0 0 0 0
gpu 0 0 0 0 0 0
cpu 0 0 0 0 0 0

Changed record examples

  • No data file changes detected.

Heuristic review

  • Heuristic warnings: none found.

Changed site

Area Added Modified Deleted
homepage/site 0 3 0

Modified site files

  • site/package.json
  • site/src/pages/index.astro
  • site/src/scripts/techapi.js

@TechEngineBot

Copy link
Copy Markdown
Member

TechEngine validation stats: PASS

Data summary

Category Total Verified Unverified Missing verified Tracked Verified % of tracked
brand 189 10 179 0 189 5.3%
soc 2104 123 1981 0 2104 5.8%
smartphone 90118 8453 81665 0 90118 9.4%
tablet 3048 174 2874 0 3048 5.7%
watch 378 11 367 0 378 2.9%
pda 110 27 83 0 110 24.5%
gpu 2030 244 1786 0 2030 12.0%
cpu 3977 986 2991 0 3977 24.8%
all 101954 10028 91926 0 101954 9.8%

Warning

Tracked verified coverage is below 50% for watch 2.9% (11/378), brand 5.3% (10/189), tablet 5.7% (174/3048), soc 5.8% (123/2104), smartphone 9.4% (8453/90118), all 9.8% (10028/101954), gpu 12.0% (244/2030), pda 24.5% (27/110), and 1 more.
Tracked coverage excludes records missing the verified field; see the Missing verified column for those records.
This does not fail validation. Keep imported records verified: false until manual audit, but treat this as follow-up verification work before relying on the affected categories as curated data.

Validation notes

  • Full advisory outlier listings are suppressed on successful runs because they are dataset-wide and mostly stable between PRs.
  • Failure runs still include a detailed log excerpt for debugging.

Key output:

## app.validate
## integrity_check.py --strict
loaded CPU=3977 GPU=2030
✅ integrity gate: no hard anomalies.
Integrity section Flagged lines
structural 0
CPU name/tier consistency (desktop mainstream only) 0
CPU single>multi (cinebench/geekbench — should be multi>=single) 0
CPU era-vs-score outliers 8
CPU cross-source ratio outliers (possible wrong-variant) 152
GPU cross-source ratio outliers + sanity 18

Homepage build:

10:54:29 [build] 2 page(s) built in 44.64s
10:54:29 [build] Complete!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working ci CI and workflow changes enhancement New feature or request site Homepage and public site changes

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

Track homepage and site improvements

2 participants