Daily popular paths delta tracking — per-day page views from rolling-window snapshots

[djdarcy]

Summary

Reconstruct per-day popular page view counts from the GitHub API's 14-day rolling-window snapshots — the same delta technique used for referrer tracking (#46), applied to the /traffic/popular/paths endpoint. Enables "which pages got the most traffic TODAY" analysis and referrer × page correlation on the dashboard.

Problem

The popular paths endpoint (GET /repos/{owner}/{repo}/traffic/popular/paths) returns only a 14-day aggregate with no per-day breakdown:

[
  { "path": "/DazzleTools/Windows-No-Internet-Secured-BUGFIX", "title": "...", "count": 153, "uniques": 125 },
  { "path": "/DazzleTools/Windows-No-Internet-Secured-BUGFIX/releases", "title": "...", "count": 11, "uniques": 7 }
]

Same constraints as referrers: top 10 only, no date parameter, no timestamps. GTT currently overwrites this snapshot daily with no history preserved (workflow line 268–279, state.popularPaths).

Users can see WHAT pages are popular over 14 days, but not WHICH pages spiked on a given day. Combined with #46's daily referrer deltas, this would enable the referrer × page correlation matrix: "When reddit.com spiked on Tuesday, which pages did those visitors land on?"

Proposed Solution

Apply the identical recurrence relation from #46:

S(d) = 14-day rolling total from API on day d
R(d) = actual page view count on day d

R(d) = [S(d) - S(d-1)] + R(d-14)

Same certainty model: first 14 days are estimates (deltaVerified: false), then exact (deltaVerified: true). Same gap tracking for pages rotating out of the top 10.

State.json Schema

Add two new top-level fields (alongside #46's referrerHistory/referrerGaps):

{
  "popularPaths": [...],           // Existing — current snapshot (unchanged)
  "pathHistory": [                 // NEW — daily snapshots with computed deltas
    {
      "date": "2026-02-27T00:00:00Z",
      "capturedAt": "2026-02-28T04:11:29Z",
      "pages": [
        { "path": "/owner/repo", "title": "...", "count": 153, "uniques": 125, "delta": 18, "deltaVerified": true },
        { "path": "/owner/repo/releases", "title": "...", "count": 11, "uniques": 7, "delta": 2, "deltaVerified": true }
      ]
    }
  ],
  "pathGaps": [                    // NEW — tracks pages dropping off / returning to top 10
    { "path": "/owner/repo/wiki", "lastSeen": "2026-02-20T00:00:00Z", "lastCount": 5, "returnedAt": null }
  ]
}

Dashboard Display

Add a "24h" column to the Popular Pages table (currently on the Dev tab):

# | Page                    | Views | Unique | 24h
1 | /owner/repo             | 153   | 125    | ⬆ +18
2 | /owner/repo/releases    | 11    | 7      | ⬆ +2
3 | /owner/repo/issues      | 9     | 7      | ⬇ -1
4 | /owner/repo/wiki        | 4     | 3      | 🆕 new

Same indicators as #46: ⬆/⬇ for verified, ~⬆/~⬇ for estimated, 🆕 for new pages, ↩️ for returned from gap.

Referrer × Page Correlation

With both #46 (daily referrer deltas) and this issue (daily page deltas), a future "Day Detail" drill-down view could show both side-by-side for a selected date:

February 27, 2026
─────────────────
Top Referrers Today         Top Pages Today
reddit.com      +40         /owner/repo             +18
Google          +3          /owner/repo/releases    +12
github.com      +2          /owner/repo/issues      +8

This correlation is the referrer × page matrix that the API can't give us directly, but daily deltas make meaningful.

Implementation

Nearly identical to #46 — the workflow delta logic, dashboard rendering, and gap tracking are structurally the same with different field names:

#46 (Referrers)	This issue (Paths)
referrerHistory	pathHistory
referrerGaps	pathGaps
sources[].source	pages[].path + pages[].title
Views tab referrer table	Dev tab popular pages table

Workflow Changes (traffic-badges.yml)

After fetching state.popularPaths (current line 268–279):

1. Store today's snapshot in pathHistory with date + capturedAt
2. Compare with yesterday's snapshot to compute deltas (match by path)
3. Track gap events in pathGaps
4. Trim pathHistory to 45 days

Dashboard Changes (docs/stats/index.html)

1. Add "24h" column to the Popular Pages table on the Dev tab
2. Read pathHistory for today's deltas
3. Render ⬆/⬇ indicators

Python Changes (src/ghtraf/gist.py)

Add pathHistory: [] and pathGaps: [] to build_initial_state().

Design Considerations

• Same schema v4 bump as Daily referrer delta tracking — per-day counts from rolling-window snapshots #46 — both referrerHistory/referrerGaps AND pathHistory/pathGaps should land in the same schema version. No need for separate v4 and v5.
• Path normalization — Unlike referrer names (which can be inconsistent: "Google" vs "google.com"), paths are URL paths and should be stable. However, GitHub occasionally changes page titles (the title field) — match on path, not title.
• Uniques not delta-computed — Same as Daily referrer delta tracking — per-day counts from rolling-window snapshots #46: uniques are not additive over the rolling window. Store raw, don't compute deltas.
• Page paths can be long — Release tag paths like /owner/repo/releases/tag/v0.7.4-alpha are verbose. Dashboard may need truncation in the 24h column layout.
• Correlation view is a stretch goal — The referrer × page drill-down is valuable but separate from the core delta tracking. Could be its own issue when both Daily referrer delta tracking — per-day counts from rolling-window snapshots #46 and this land.

Acceptance Criteria

• Workflow stores daily popular paths snapshots in pathHistory
• Workflow computes deltas using recurrence relation, with deltaVerified flag
• Workflow tracks page gap events in pathGaps
• pathHistory trimmed to 45 days
• Dashboard shows 24h column with ⬆/⬇ indicators on Popular Pages table
• Bootstrap period (< 14 days) shows estimated deltas with distinct styling
• build_initial_state() includes pathHistory: [] and pathGaps: []
• Test data updated with sample pathHistory
• Shares schema v4 bump with Daily referrer delta tracking — per-day counts from rolling-window snapshots #46 (single migration)

Related Issues

• Refs Daily referrer delta tracking — per-day counts from rolling-window snapshots #46 — Daily referrer delta tracking (same technique, sister issue)
• Refs ghtraf upgrade — run schema migrations on gist state.json #30 — ghtraf upgrade (schema v4 migration)
• Refs Shared JS data layer — extract duplicated logic from workflow and dashboard #42 — Shared JS data layer (delta computation functions are extraction candidates)
• Refs ghtraf verify — state.json validation and consistency checks #38 — ghtraf verify (will need pathHistory validation)

Analysis

See 2026-02-28__06-07-45__dev-workflow-process_daily-referrer-delta-tracking.md (Addendum section) for the initial discussion of extending the delta technique to popular paths.