Problem
The scraper currently discovers and extracts every link it finds on a roaster's listing page. Unfortunately, many roasters sell non-coffee items alongside their coffee:
- cajik / čaj - herbal teas, fruit infusions, black/green tea
- Coffee equipment - brewers, grinders, scales, towels
- Gift cards / voucher codes
- Chocolate / confectionery - some roasters sell chocolate bars
- Flavorings / syrups - vanilla, caramel syrups for coffee
These clutter the table and make the data harder to navigate. 'cajik' entries appear in scrape results because the roaster's product grid mixes them with coffee - they have a valid origin and packaging but aren't coffee.
Proposed
1. Tighten product-link detection
Add a negative keyword list to the product-link discovery filter to block obviously non-coffee pages from being sent to the LLM:
- Czech/Slovak:
cajík, cajíky, čaj, doplnok, darceká karta, zdravotencky, pytel, mlynček
- English:
tea, gift card, voucher, syrup, equipment, brewer, towel, chocolate
Match against the URL slug/path segment AND the listing page link text when available.
2. LLM decline fallback (already partially working)
The extract_product tool is optional - the model can decline. But this wastes an LLM call.
3. Post-extraction sanity check
If the extracted product name strongly suggests non-coffee (e.g. 'green tea matcha', 'ceramic pour-over'), mark as not_a_product.
Implementation notes
- Start with a simple blocklist of keywords in a new filter function near the top of the discovery loop.
- Track how many items are filtered - should reduce unnecessary LLM calls and clean up product listings.
Problem
The scraper currently discovers and extracts every link it finds on a roaster's listing page. Unfortunately, many roasters sell non-coffee items alongside their coffee:
These clutter the table and make the data harder to navigate. 'cajik' entries appear in scrape results because the roaster's product grid mixes them with coffee - they have a valid
originandpackagingbut aren't coffee.Proposed
1. Tighten product-link detection
Add a negative keyword list to the product-link discovery filter to block obviously non-coffee pages from being sent to the LLM:
cajík,cajíky,čaj,doplnok,darceká karta,zdravotencky,pytel,mlynčektea,gift card,voucher,syrup,equipment,brewer,towel,chocolateMatch against the URL slug/path segment AND the listing page link text when available.
2. LLM decline fallback (already partially working)
The
extract_producttool is optional - the model can decline. But this wastes an LLM call.3. Post-extraction sanity check
If the extracted product name strongly suggests non-coffee (e.g. 'green tea matcha', 'ceramic pour-over'), mark as
not_a_product.Implementation notes