Skip to content
This repository has been archived by the owner on Mar 7, 2021. It is now read-only.

simplecrawler 1.1.3
Compare
Choose a tag to compare
@fredrikekelund fredrikekelund released this 21 Jun 11:47
· 111 commits to master since this release
v1.1.3
cc22c5c

New features

  • #376 - added Crawler#sortQueryParameters option. This option will sort the query parameters in a URL, making it simple to avoid fetching duplicate pages on sites that use a lot of query parameters. Thanks @HaroldPutman!
Assets 2
Loading