Skip to content

0.60.0

Compare
Choose a tag to compare
@jqnatividad jqnatividad released this 25 Jul 03:58
· 6348 commits to master since this release

Added

  • fetch: add redis --flushdb option #387
  • fetch: add --report & --cache-error options. --report creates a separate report file, detailing the URL used,
    the response, the HTTP status code, and if its a cache hit.
    --cache-error is used to also cache errors - i.e. identical fetches will return the cached error. Otherwise, fetch will
    request the URL again. #393

Changed

  • fetch: fast defaults. Now tries to go as fast as possible, leveraging dynamic throttling (using RateLimit and Rety-After headers)
    but aborting after 100 errors. Also added a separate error progress bar. #388
  • Smarter tojsonl. Now scans CSV file and infers data types and uses the appropriate JSON data type #389
  • tojsonl is also multithreaded #392
  • stats: use unwrap_unchecked for even more performance #390
  • fetch: refactor dynamic throttling #391
  • Bump sysinfo from 0.24.6 to 0.24.7 by @dependabot in #384
  • cargo bump update several dependencies
  • pin Rust nightly to 2022-07-23

Fixed

  • fetch: fix --http-header parsing bug #386

Full Changelog: 0.59.0...0.60.0