Skip to content

Releases: pixlie/SmartCrawler

SmartCrawler 0.4.1

Choose a tag to compare

@github-actions github-actions released this 09 Jul 03:38
  • chore: bump version to 0.4.1
  • Merge pull request #65 from pixlie/chore/update-readme-docs
  • docs: move development and contributing info to separate guide
  • chore: restructure documentation with docs/ folder and OS-specific guides
  • Merge pull request #63 from pixlie/feature/template-detection
  • fix: resolve clippy warnings for uninlined format args
  • chore: apply code formatting with cargo fmt
  • feat: disable domain duplicate filtering in template mode
  • feat: apply template detection before domain duplicate analysis
  • feat: implement template detection for variable and text concatenation patterns
  • Merge pull request #60 from pixlie/feature/real-world-tests
  • fix: resolve clippy warnings in real-world tests
  • style: apply cargo fmt formatting
  • feat: enhance real-world tests with complete SmartCrawler pipeline
  • fix: configure real-world tests to run serially
  • feat: add real-world integration tests (#56)
  • Merge pull request #59 from pixlie/fix/duplicate-root-url-loading
  • fix: prevent duplicate root URL loading (#58)
  • Merge pull request #57 from pixlie/feature/filter-domain-duplicate-html-nodes
  • feat: prioritize root URLs and improve element ID handling
  • fix: remove overly aggressive page-level duplicate filtering
  • feat: improve duplicate detection to include complete element structure
  • fix: improve domain duplicate filtering logic
  • feat: add domain-level duplicate HTML node filtering
  • Merge pull request #55 from pixlie/feature/project-restart-step-one
  • Formatting fixes
  • chore: fix formatting and linting issues
  • fix: change default WebDriver port from 9515 to 4444
  • fix: initialize rustls crypto provider to prevent runtime error
  • feat: project restart with WebDriver-based crawler
  • feat: include single-item groups in grouped data detection
  • Merge pull request #50 from pixlie/feature/grouped-data-extraction
  • fix: show complete path including grouped elements in path display
  • refactor: improve grouped data path display with full CSS selectors
  • fix: eliminate duplicates in grouped data detection
  • feat: implement grouped data detection with --grouped CLI option
  • Merge pull request #48 from brainless/feature/data-extraction
  • fix: resolve clippy warning in extractor module
  • feat: implement data extraction mode with --extract CLI option
  • Merge pull request #47 from brainless/chore/remove-html-cleaning
  • chore: remove HTML cleaning functionality
  • Merge pull request #46 from brainless/chore/remove-structured-content
  • Formatting fixes
  • chore: remove StructuredContent and extract_structured_data
  • Minor changes to Claude.md
  • Merge pull request #40 from brainless/feature/improve-html-cleaner
  • feat: extend image filtering to handle SVG elements (issue #39)
  • feat: improve HTML cleaner with image filtering and comment removal (fixes #39)
  • Merge pull request #38 from brainless/feature/clean-html-url-support
  • fix: initialize crypto provider in browser URL test
  • feat: use browser/webdriver for URL fetching in --clean-html mode
  • feat: add URL support to --clean-html mode (fixes #37)

SmartCrawler 0.3.2

Choose a tag to compare

@github-actions github-actions released this 30 Jun 13:10
  • chore: bump version to 0.3.2
  • Merge pull request #36 from brainless/feature/links-argument
  • feat: implement --links argument for URL-based crawling (issue #32)
  • Merge pull request #35 from brainless/feature/homepage-first-url
  • feat: treat homepage as first URL in crawl analysis (fixes #34)
  • Merge pull request #33 from brainless/feature/url-domain-parsing
  • Formatting fixes
  • fix: resolve clippy warnings for uninlined format args
  • Formatting fixes; updated Claude.md
  • feat: allow full URLs as domain arguments (fixes #31)
  • Merge pull request #30 from brainless/feature/product-website
  • feat: create one-pager product website for SmartCrawler
  • docs: make CLAUDE.md more concise while preserving development workflow
  • Merge pull request #28 from brainless/feature/improve-url-selection
  • Formatting fixes
  • feat: implement improved URL selection with keyword-based ranking - issue #27
  • Merge pull request #26 from brainless/feature/typescript-schema-prompts
  • fix: update TypeScript generation test assertions
  • Formatting fixes
  • feat: implement structured JSON responses with ts-rs macro generation
  • Formatting fixes
  • Formatting fixes
  • fix: resolve JSON parsing errors in entity extraction
  • Merge pull request #25 from brainless/feature/typed-entity-extraction
  • fix: update analyze_content method to use TypeScript schemas
  • feat: enhance entity extraction with TypeScript schema prompts
  • feat: add typed entity extraction system for structured data

SmartCrawler 0.3.1

Choose a tag to compare

@github-actions github-actions released this 25 Jun 15:40
  • chore: bump version to 0.3.1
  • Crawler now asks if user wants to continue if objective is met; scroll made faster
  • Merge pull request #23 from brainless/docs/user-friendly-readme
  • docs: rewrite README.md for general users with comprehensive setup guide

SmartCrawler 0.3.0

Choose a tag to compare

@github-actions github-actions released this 24 Jun 15:28
  • chore: bump version to 0.3.0
  • Merge pull request #22 from brainless/feature/browser-scroll-enhancement
  • feat: implement browser scrolling enhancement from GitHub issue #21
  • Merge pull request #20 from brainless/feature/crawler-improvements
  • Shorter and clearer response in CLI after crawling
  • feat: implement crawler improvements from GitHub issue #19

SmartCrawler 0.2.8

Choose a tag to compare

@github-actions github-actions released this 24 Jun 12:31
  • chore: bump version to 0.2.8
  • Merge pull request #18 from brainless/refactor/llm-prompts-in-trait
  • cargo fmt
  • Merge pull request #17 from brainless/fix/cargo-rpm-file-path
  • Refactor: Move LLM prompts to default trait implementations
  • fix: resolve cargo-rpm file path issue in GitHub Actions workflow
  • rustls fix

SmartCrawler 0.2.7

Choose a tag to compare

@github-actions github-actions released this 24 Jun 11:57
  • chore: bump version to 0.2.7
  • Merge pull request #15 from brainless/fix/cargo-deb-asset-path
  • Undo test code
  • fix: resolve cargo-deb TOML parse error for DEB package building
  • Merge pull request #13 from brainless/fix/github-actions-powershell-syntax

SmartCrawler 0.2.6

Choose a tag to compare

@github-actions github-actions released this 24 Jun 11:06
  • chore: bump version to 0.2.6
  • fix: resolve PowerShell syntax error in GitHub Actions workflow

SmartCrawler 0.2.5

Choose a tag to compare

@github-actions github-actions released this 24 Jun 10:24
  • chore: bump version to 0.2.5
  • Merge pull request #11 from brainless/fix/missing-installer-packages
  • fix: resolve missing installer packages in GitHub releases

SmartCrawler 0.2.4

Choose a tag to compare

@github-actions github-actions released this 24 Jun 09:45
  • chore: bump version to 0.2.4
  • Updated CLAUDE.md about development workflow
  • Merge pull request #10 from brainless/fix/github-actions-cross-compilation-strip
  • fix: resolve cross-compilation strip failure in GitHub Actions

SmartCrawler 0.2.3

Choose a tag to compare

@github-actions github-actions released this 24 Jun 06:49
  • chore: bump version to 0.2.3
  • Removed unnecessary crates