Skip to content

fix(usps): support USPS v2 progress-bar tracking page#30

Merged
WolffRuoff merged 1 commit into
mainfrom
fix/usps-new-tracking-layout
Jul 9, 2026
Merged

fix(usps): support USPS v2 progress-bar tracking page#30
WolffRuoff merged 1 commit into
mainfrom
fix/usps-new-tracking-layout

Conversation

@WolffRuoff

Copy link
Copy Markdown
Owner

Problem

USPS scrapes started failing with a wait_for_selector timeout on .track-statusbar:

Page load failed ... selector='.track-statusbar'
  redirected_to=https://tools.usps.com/tracking/9434637902900031932314

USPS migrated its tracking page — …/go/TrackConfirmAction now 302-redirects to a new …/tracking/ page. The new DOM dropped the .track-statusbar wait selector and the .current-tracking-status-wrapper status banner, so the scrape timed out before parsing and returned no data. (The bot_service=Akamai Bot Manager line in the log is a separate IP-dependent variant; the deployment hitting this bug reaches real content fine.)

Fix

Support both layouts instead of swapping one for the other:

  • v1 — legacy widget (.track-statusbar / .current-tracking-status-wrapper)
  • v2 — progress bar (.tracking-progress-bar-status-container)

Robustness against future CSS churn

The redesign renamed the wrapper classes but kept the leaf classes (.tb-step, .tb-status, .tb-status-detail, .tb-date, .tb-location, .expected_delivery, .eta_snip). So parsing now anchors on the stable leaves and treats wrappers as optional — a future "v3" wrapper rename should need no code changes.

  • WAIT_SELECTOR waits on leaf classes + the known wrappers
  • status derives from leaf status text, with the v2 <state>-status container modifier class as a fallback
  • events select .tb-step directly (wrapper-independent), with dedup
  • _parse_date strips the clock time now on event dates ("July 8, 2026 9:49 AM") and no longer corrupts the weekday "Monday" (a bare "on" match); event description now prefers .tb-status-detail

Note: an embedded-JSON approach (à la SpeedX) was considered for maximum durability, but the new USPS page is server-rendered with no JSON/state blob (grep found no ld+json / __INITIAL_STATE__), so leaf-anchoring is the durable ceiling here.

Testing

  • New v2 fixtures (v2_in_transit.html, v2_delivered.html) built from a real captured page, plus a TestParseV2ProgressBarLayout suite and _parse_date regression tests.
  • Full scraper suite: 187 passed.

🤖 Generated with Claude Code

USPS migrated its tracking page: /go/TrackConfirmAction now 302-redirects
to a new /tracking/ page whose DOM dropped the .track-statusbar wait
selector and the .current-tracking-status-wrapper status banner, so scrapes
timed out on wait_for_selector and returned no data.

Rather than replace the old selectors, support both layouts (v1 legacy
widget, v2 progress bar). The key robustness insight from the redesign is
that USPS renamed the *wrapper* classes but kept the *leaf* classes
(.tb-step / .tb-status / .tb-status-detail / .tb-date / .tb-location /
.expected_delivery / .eta_snip), so parsing now anchors on those leaves and
treats wrappers as optional — this should survive a future "v3" wrapper
rename with no code changes.

- WAIT_SELECTOR waits on stable leaf classes plus the known wrappers
- status derives from leaf status text, with the v2 "<state>-status"
  container modifier class as a fallback
- events select .tb-step directly (wrapper-independent) with dedup
- _parse_date strips the clock time now present on event dates
  ("July 8, 2026 9:49 AM") and no longer corrupts weekday "Monday" (the
  bare "on" match); event description prefers .tb-status-detail
- new v2 fixtures + tests, built from a real captured page

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@WolffRuoff
WolffRuoff merged commit 8e9f2b2 into main Jul 9, 2026
5 checks passed
@WolffRuoff
WolffRuoff deleted the fix/usps-new-tracking-layout branch July 9, 2026 02:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant