fix(usps): support USPS v2 progress-bar tracking page#30
Merged
Conversation
USPS migrated its tracking page: /go/TrackConfirmAction now 302-redirects
to a new /tracking/ page whose DOM dropped the .track-statusbar wait
selector and the .current-tracking-status-wrapper status banner, so scrapes
timed out on wait_for_selector and returned no data.
Rather than replace the old selectors, support both layouts (v1 legacy
widget, v2 progress bar). The key robustness insight from the redesign is
that USPS renamed the *wrapper* classes but kept the *leaf* classes
(.tb-step / .tb-status / .tb-status-detail / .tb-date / .tb-location /
.expected_delivery / .eta_snip), so parsing now anchors on those leaves and
treats wrappers as optional — this should survive a future "v3" wrapper
rename with no code changes.
- WAIT_SELECTOR waits on stable leaf classes plus the known wrappers
- status derives from leaf status text, with the v2 "<state>-status"
container modifier class as a fallback
- events select .tb-step directly (wrapper-independent) with dedup
- _parse_date strips the clock time now present on event dates
("July 8, 2026 9:49 AM") and no longer corrupts weekday "Monday" (the
bare "on" match); event description prefers .tb-status-detail
- new v2 fixtures + tests, built from a real captured page
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
USPS scrapes started failing with a
wait_for_selectortimeout on.track-statusbar:USPS migrated its tracking page —
…/go/TrackConfirmActionnow 302-redirects to a new…/tracking/page. The new DOM dropped the.track-statusbarwait selector and the.current-tracking-status-wrapperstatus banner, so the scrape timed out before parsing and returned no data. (Thebot_service=Akamai Bot Managerline in the log is a separate IP-dependent variant; the deployment hitting this bug reaches real content fine.)Fix
Support both layouts instead of swapping one for the other:
.track-statusbar/.current-tracking-status-wrapper).tracking-progress-bar-status-container)Robustness against future CSS churn
The redesign renamed the wrapper classes but kept the leaf classes (
.tb-step,.tb-status,.tb-status-detail,.tb-date,.tb-location,.expected_delivery,.eta_snip). So parsing now anchors on the stable leaves and treats wrappers as optional — a future "v3" wrapper rename should need no code changes.WAIT_SELECTORwaits on leaf classes + the known wrappers<state>-statuscontainer modifier class as a fallback.tb-stepdirectly (wrapper-independent), with dedup_parse_datestrips the clock time now on event dates ("July 8, 2026 9:49 AM") and no longer corrupts the weekday"Monday"(a bare"on"match); event description now prefers.tb-status-detailTesting
v2_in_transit.html,v2_delivered.html) built from a real captured page, plus aTestParseV2ProgressBarLayoutsuite and_parse_dateregression tests.🤖 Generated with Claude Code