Version 5.10.0
Features and Improvements
- Added
ia download --rangefor partial (byte-range) downloads. It requires
--stdoutand is repeatable, taking[FILE:]START-ENDvalues: a bare
range binds to the named file (vary the range or the file, not both at once),
orFILE:START-ENDbinds each range to its own file. Ranges may be given as
START-END, open-endedSTART-, suffix-N(the lastNbytes), or
bytes=..., and a single value may carry several comma-separated ranges
(0-9,50-99), fetched in order. Segments are streamed back-to-back with no separator, so e.g.
WARC records selected via a CDX index's compressed offset/length can be piped
straight tozcat. Useful for partial fetches of private items (configured
credentials are used).Item.download(),File.download(), and the
top-levelinternetarchive.download()gained aheadersargument, and
Item.download()arange_jobsargument; passing aRangeheader is
treated as an intentional partial fetch and disables resume and full-file
checksum validation. An unsatisfiable range (HTTP416) fails fast with a
clear message instead of being retried; a range covering the whole file
returns the full contents (HTTP200). If any segment fails,
ia downloadexits non-zero, so a downstream pipe consumer can tell the
output is incomplete. - Added
ia tasks --follow-task-log <task_id>to follow a task log live
as the task runs (tail -fstyle), stopping automatically when the task
finishes. Combine with-p lines=-Nto seed the lastNlines first
(Tasks APIlinessemantics, as with--get-task-log); any other
-pparams are forwarded to the Tasks API. A new
ArchiveSession.follow_task_log()method exposes the same behavior to the
library.
Bugfixes
- Fixed
File.download(stdout=True)consulting the local filesystem: a
same-named local file could cause the stream to be skipped (length/date or
checksum match) or trigger the auto-resume code path, which seeks the output
and fails on a pipe. A stdout download now ignores any on-disk file. - Fixed a retried
stdoutdownload falling back to writing a local disk file
instead of the pipe (leaving the pipe empty). Astdoutdownload now always
writes tostdout, even across retries. - Fixed auto-resume corrupting a file when a resumed transfer was itself retried:
the internalRangeheader was not recomputed for the retry, so it no longer
matched the (grown) local file and the seek offset, re-fetching already-written
bytes. The resumeRangeis now recomputed from the current file size on
every attempt. - Fixed
ia tasks --parametercrashing when combined with
--get-task-log. Parameters such aslinesare now merged into the
task log request's query string, allowingia tasks -G <task_id> -p lines=100to fetch a truncated log.get_task_log()gained aparams
argument;paramsandrequest_kwargsare now keyword-only and kept
distinct, so request kwargs (e.g.timeout,headers) are no longer
serialized into the URL as query parameters
(#764 <https://github.com/jjjake/internetarchive/pull/764>_).