Releases: wintermi/get-linked-data
Releases · wintermi/get-linked-data
v0.3.1
- Now shuffles the URL list, changing the order which Colly will scrape the data
- Change the User Agent
- Change the defaults for Parallelism and Wait Time
v0.3.0
- Switch to use asynchronous processing within Colly
- Add additional arguments allowing you to specify the parallelism and maximum delay wait time
- Replace the automatic retries with outputting the Failed Request URLs to a file
v0.2.0
- Added a new selector using
jq
to allow you to reduce a JSON object to just the elements you want, works in conjunction to the element selector.
v0.1.0
A command line application designed to crawl a given set of URLs and scrape the JSON Linked Data (JSON-LD) contained within the webpage before writing the data entries out to a CSV file.