Releases: pixlie/SmartCrawler
Releases · pixlie/SmartCrawler
Release list
SmartCrawler 0.4.1
- chore: bump version to 0.4.1
- Merge pull request #65 from pixlie/chore/update-readme-docs
- docs: move development and contributing info to separate guide
- chore: restructure documentation with docs/ folder and OS-specific guides
- Merge pull request #63 from pixlie/feature/template-detection
- fix: resolve clippy warnings for uninlined format args
- chore: apply code formatting with cargo fmt
- feat: disable domain duplicate filtering in template mode
- feat: apply template detection before domain duplicate analysis
- feat: implement template detection for variable and text concatenation patterns
- Merge pull request #60 from pixlie/feature/real-world-tests
- fix: resolve clippy warnings in real-world tests
- style: apply cargo fmt formatting
- feat: enhance real-world tests with complete SmartCrawler pipeline
- fix: configure real-world tests to run serially
- feat: add real-world integration tests (#56)
- Merge pull request #59 from pixlie/fix/duplicate-root-url-loading
- fix: prevent duplicate root URL loading (#58)
- Merge pull request #57 from pixlie/feature/filter-domain-duplicate-html-nodes
- feat: prioritize root URLs and improve element ID handling
- fix: remove overly aggressive page-level duplicate filtering
- feat: improve duplicate detection to include complete element structure
- fix: improve domain duplicate filtering logic
- feat: add domain-level duplicate HTML node filtering
- Merge pull request #55 from pixlie/feature/project-restart-step-one
- Formatting fixes
- chore: fix formatting and linting issues
- fix: change default WebDriver port from 9515 to 4444
- fix: initialize rustls crypto provider to prevent runtime error
- feat: project restart with WebDriver-based crawler
- feat: include single-item groups in grouped data detection
- Merge pull request #50 from pixlie/feature/grouped-data-extraction
- fix: show complete path including grouped elements in path display
- refactor: improve grouped data path display with full CSS selectors
- fix: eliminate duplicates in grouped data detection
- feat: implement grouped data detection with --grouped CLI option
- Merge pull request #48 from brainless/feature/data-extraction
- fix: resolve clippy warning in extractor module
- feat: implement data extraction mode with --extract CLI option
- Merge pull request #47 from brainless/chore/remove-html-cleaning
- chore: remove HTML cleaning functionality
- Merge pull request #46 from brainless/chore/remove-structured-content
- Formatting fixes
- chore: remove StructuredContent and extract_structured_data
- Minor changes to Claude.md
- Merge pull request #40 from brainless/feature/improve-html-cleaner
- feat: extend image filtering to handle SVG elements (issue #39)
- feat: improve HTML cleaner with image filtering and comment removal (fixes #39)
- Merge pull request #38 from brainless/feature/clean-html-url-support
- fix: initialize crypto provider in browser URL test
- feat: use browser/webdriver for URL fetching in --clean-html mode
- feat: add URL support to --clean-html mode (fixes #37)
SmartCrawler 0.3.2
- chore: bump version to 0.3.2
- Merge pull request #36 from brainless/feature/links-argument
- feat: implement --links argument for URL-based crawling (issue #32)
- Merge pull request #35 from brainless/feature/homepage-first-url
- feat: treat homepage as first URL in crawl analysis (fixes #34)
- Merge pull request #33 from brainless/feature/url-domain-parsing
- Formatting fixes
- fix: resolve clippy warnings for uninlined format args
- Formatting fixes; updated Claude.md
- feat: allow full URLs as domain arguments (fixes #31)
- Merge pull request #30 from brainless/feature/product-website
- feat: create one-pager product website for SmartCrawler
- docs: make CLAUDE.md more concise while preserving development workflow
- Merge pull request #28 from brainless/feature/improve-url-selection
- Formatting fixes
- feat: implement improved URL selection with keyword-based ranking - issue #27
- Merge pull request #26 from brainless/feature/typescript-schema-prompts
- fix: update TypeScript generation test assertions
- Formatting fixes
- feat: implement structured JSON responses with ts-rs macro generation
- Formatting fixes
- Formatting fixes
- fix: resolve JSON parsing errors in entity extraction
- Merge pull request #25 from brainless/feature/typed-entity-extraction
- fix: update analyze_content method to use TypeScript schemas
- feat: enhance entity extraction with TypeScript schema prompts
- feat: add typed entity extraction system for structured data
SmartCrawler 0.3.1
- chore: bump version to 0.3.1
- Crawler now asks if user wants to continue if objective is met; scroll made faster
- Merge pull request #23 from brainless/docs/user-friendly-readme
- docs: rewrite README.md for general users with comprehensive setup guide
SmartCrawler 0.3.0
- chore: bump version to 0.3.0
- Merge pull request #22 from brainless/feature/browser-scroll-enhancement
- feat: implement browser scrolling enhancement from GitHub issue #21
- Merge pull request #20 from brainless/feature/crawler-improvements
- Shorter and clearer response in CLI after crawling
- feat: implement crawler improvements from GitHub issue #19
SmartCrawler 0.2.8
- chore: bump version to 0.2.8
- Merge pull request #18 from brainless/refactor/llm-prompts-in-trait
- cargo fmt
- Merge pull request #17 from brainless/fix/cargo-rpm-file-path
- Refactor: Move LLM prompts to default trait implementations
- fix: resolve cargo-rpm file path issue in GitHub Actions workflow
- rustls fix
SmartCrawler 0.2.7
SmartCrawler 0.2.6
- chore: bump version to 0.2.6
- fix: resolve PowerShell syntax error in GitHub Actions workflow
SmartCrawler 0.2.5
- chore: bump version to 0.2.5
- Merge pull request #11 from brainless/fix/missing-installer-packages
- fix: resolve missing installer packages in GitHub releases
SmartCrawler 0.2.4
- chore: bump version to 0.2.4
- Updated CLAUDE.md about development workflow
- Merge pull request #10 from brainless/fix/github-actions-cross-compilation-strip
- fix: resolve cross-compilation strip failure in GitHub Actions
SmartCrawler 0.2.3
- chore: bump version to 0.2.3
- Removed unnecessary crates