v0.1.1

Behaviour change: a keyword the site redirects to a category (for example "chicken feed") used to end with exit 4, "the listing is empty". It now reads that category and exits 0, with searched_keyword and search_redirected_to in the sidecar and the rows marked mode: category. A pipeline that branched on exit 4 for such a keyword was branching on a false statement about the catalogue.

Works through a third-party audit (2026-10-08); every finding was reproduced before it was fixed, and the declined ones are argued in #1.

Fixed

• A failed rewrite no longer destroys the previous good output: JSON, CSV and the sidecar are written beside their targets and renamed over them (reproduced before: a 4,518-byte good file became 13 bytes of invalid JSON). New files get the mode open() would give; an existing file keeps its own.
• A search the site redirects is followed, not reported as empty.
• A landing that fails with ERR_HTTP2_PROTOCOL_ERROR or a reset now says it is how Akamai refused local Chromium in testing, and a run without --cdp-endpoint warns up front. Still exit 5.

Added

• The sidecar lists each output file with its size and sha256 (outputs), so a reader can tell a complete set from one caught mid-update.
• duplicates_dropped, and rows_missing_vs_total when a run read every page (two full live categories: 0).
• CSV formula neutralisation (csv_cells_escaped), CSV only.

Changed

• The canary runs weekly. The "credentials last about a day" figure it was built on was copied from other profiles; measured here, an endpoint set on 2026-09-28 was still served on 2026-10-08.

See CHANGELOG.md.

v0.1.0

What you need: the 2Captcha Scraping Browser API with a US exit (--cdp-endpoint, country-us). On 2026-09-28 it was the only client tractorsupply.com served: plain HTTP, local Chromium from a datacentre or a US residential proxy, and the Scraper API were all refused by Akamai. The README's access table has the measurements.

First release. Category, department and search listings from tractorsupply.com's own search endpoint, priced per store (--zip, default 37027), over three browser engines that share one fetch loop.

Measured on the canary's first dispatch, from a GitHub runner through the Scraping Browser: 144 of 273 category products and 144 of 1,076 search results, both complete.

Highlights

• 35-column Product rows: price range, list price and computed discount, unit price, MAP handling, rating, reviews, stock and pickup at the chosen store, brand, model number, badges, promotion, variants.
• Traps handled and pinned by tests: the endpoint's price is often a variant's maximum; prices differ per store; a department's visible id returns nothing; an unknown category is HTTP 500; a wrong pricing zone fails under HTTP 200; a backend circuit breaker is retried rather than reported as a bad parameter; a nonsense query returns unrelated products (warned, and query_relevance in the sidecar).
• Selenium is not served on this site: chromedriver cannot use a credentialled CDP endpoint. It reports the refusal correctly (exit 3).

Not included, on purpose: no captcha solver (no refusal carried a widget), no Scraper API client (refused), no product-detail mode (the page carries no price).

See CHANGELOG.md.