Feature Request: Download web site(s) + Use existing assets (Action)

[mpheyse]

I would like to see a new action that is that same as "Download web site(s)" but will use (and not re-download) any non-cache files existing locally.

I'm working with a particularity slow server, and all the pages are in a single CGI and I'm getting lots of errors and problems etc.

The non HTML 'files' are the bulk of the data, take the most time to download (days) and are not my major error source.

The thing I'm noticing is that repairing (continue) an existing site download is more error prone (breaks pages that were working) than a fresh new download as I work through the mostly undocumented errors.

So What I would like to do is make a run fresh "Download web site(s)" on a clean directory, but prepopulate it with all the .jpg .zip .dat files I've already downloaded 4 times already. And as the new run hits an existing on disk file, in the right place with the correct name, it accepts it as-is and doesn't re-download it.

[mpheyse]

New action

Download web site(s) + Use existing assets

Functionally the same as the current "Download web site(s)" Action, but any file found existing on disk (that would have been a collision), will not be (re)downloaded but will be treated/recorded as if, it had been.

Discover Local Assets

Checks current local disk, for any files not appearing in new.lst and addes entries for them to mark them as downloaded.

This allows for manual later addition of assets. It also could replace "Download web site(s) + Use existing assets" if you run "Discover Local Assets" and then "Continue Interrupted Download".