feat: subcategory auto-discovery, fyndhörna filter, CSV price drop toggle#103
Merged
feat: subcategory auto-discovery, fyndhörna filter, CSV price drop toggle#103
Conversation
…ggle - Revert URL input from textarea back to single field — subcategory pagination now handles category discovery automatically - Add "Auto-discover subcategories" checkbox to add form; inet.se/komplett/ webhallen templates default to subcategory mode with pagination_selector - Add exclude_link_pattern per config: product URLs containing the pattern are silently skipped (inet.se template defaults to /produkt/x to exclude open-box fyndhörna items that caused false -50% drop readings) - Fix urljoin in extract_product: use page.url instead of config base_url (base_url may contain multiple lines after multi-URL feature) - CSV export: add include_drops query param; when enabled adds Was/Drop% columns using price_history; UI shows "Price drops" toggle next to Export Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
Replace partial SQL string concatenation with a static lookup table of four fully-prewritten queries keyed on (include_drops, has_site_name). No user input ever touches the SQL string itself; site_name is still passed as a parameterized %s value. Fixes CodeQL alert py/sql-injection (alert #37). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Previously the pagination_selector was silently set to '' for any site not matching a known template name, making subcategory auto-discovery non-functional for custom sites. - Show/hide 'Category link selector' input when subcategory checkbox is toggled - loadTemplate populates the field from the template definition - addConfig reads directly from the input instead of template lookup - Move csv/StringIO/Response imports to top-level (style fix) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
site_name (URL param) no longer influences which query is selected — export_site_csv always uses the site-specific variant keyed on (include_drops, True), while export_all_csv uses (include_drops, False). site_name only appears in the parameterized tuple passed to cur.execute. Removes _build_export_query helper (no longer needed). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
exclude_link_pattern-fält per config. Produktlänkar som matchar strängen ignoreras helt. Inet.se-templaten sätter/produkt/xsom default — filtrerar bort alla open-box-artiklar som orsakade falska prisraset (t.ex. WD Black visade -50% när det egentligen var 18%)urljoiniextract_productanvändeconfig['base_url']som bas — fel efter multi-URL-stödet. Ändrat tillpage.urlVad händer med befintliga inet.se-kategori-configs?
De fungerar fortfarande. Men efter merge kan du:
Test plan
dcl scraperloggar kategorier automatiskt utan att du listat dem/produkt/x-artiklar inte dyker upp i produktlistan🤖 Generated with Claude Code