Skip to content

v0.9.1

Choose a tag to compare

@github-actions github-actions released this 23 May 02:05
· 79 commits to main since this release

Changelog

  • 82af3c9 Fix broken example URLs in README.md
  • 0cd7da1 Merge pull request #8 from abhmul/main
  • 93fa9a3 chore(release): v0.9.1
  • b002496 docs(crawl): document idempotency requirement for URL rewrite rules
  • 395224f feat(cmd/scrape): construct scraper with rewriter; key cache by rewritten URL
  • 112ea63 feat(cmd/search): apply URL rewrites in search --scrape via shared Scraper
  • 439eb74 feat(config): add URLRewrites field
  • 67af60a feat(config): support url_rewrites in config set and discovery
  • ac1e645 feat(crawl): receive scraper from caller; rewrite URLs in enqueue and cache key
  • f922254 feat(output): emit fetched_url in scrape/search frontmatter when set
  • c71a61b feat(scrape): add optional URL rewriter to Scraper
  • 790e94c feat(scrape): apply URL rewrites in Scrape/ScrapeConditional/BrowserScrape
  • 0b34aa3 feat(search): expose FetchedURL on Result and populate from scraped Page
  • 3dc5396 feat(urlrewrite): add regex-based URL rewriter package
  • 2361cfc refactor(cmd): extract newScraper helper for rewriter+browser construction
  • 376093c refactor(crawl): drop redundant Rewrite in processItem cache key