On make dl

[fmap]

Reposting mail verbatim, because lazy:

> I'm going to call that done, at least until the second volume is
> converted upstream (at which point we shouldn't need to do anything but
> add a line to the Makefile.) Do the I and III build okay on each of your
> systems?

I get fail on 'make all'

Converting css/28796_002.css... nothing to do.
Converting css/polytexnic.css... nothing to do.
Converting css/custom.css... nothing to do.
Converted 95 files in 0.1 seconds.
Makefile:8: recipe for target 'dl' failed
make: *** [dl] Error 8

clean is a bit extreme as it's now dowloading everything again.

> make: *** [dl] Error 8

From the Make documentation:
> ‘[foo] Error NN’
> ‘[foo] signal description’
> These errors are not really make errors at all. They mean that a program that make invoked as part of a recipe returned a
non-0 error code (‘Error NN’), which make interprets as failure, or it exited in some other abnormal fashion (with a signal
of some type). See Errors in Recipes.
> If no *** is attached to the message, then the sub-process failed but the rule in the makefile was prefixed with the -
special character, so make ignored the error.

From the wget manpage:
> C<8>
>     Server issued an error response.

I'm re-running dl to see if I get the same error / which resource that
response's about.

> I'm re-running dl to see if I get the same error / which resource that
> response's about.

Eww, so:

  1. There are links to unpublished chapters somewhere on that site,
     e.g. http://www.feynmanlectures.caltech.edu/II_15.html         

  2. When wget tries to fetch them, it gets a 404 back, and sets its
     return code to 8                                               

        % wget http://www.feynmanlectures.caltech.edu/II_15.html
        --2013-12-06 01:49:31--  http://www.feynmanlectures.caltech.edu/II_15.html
        Resolving www.feynmanlectures.caltech.edu (www.feynmanlectures.caltech.edu)... 131.215.235.206
        Connecting to www.feynmanlectures.caltech.edu (www.feynmanlectures.caltech.edu)|131.215.235.206|:80... connected.
        HTTP request sent, awaiting response... 404 Not Found
        2013-12-06 01:49:32 ERROR 404: Not Found.

        % echo $?
        8

  3. Because wget returns !0, later make tasks don't run.

Is there a way around this, besides wrapping wget to lie about what it
saw?

[fmap]

The bug here, of course, is that we're now failing 'silently' if wget
can't retrieve useful pages for other reasons: if the client lacks
network access, or if resources we actually want from the server are
unavailable.

One alternative solution -> maintaining a list of resources we need, e.g.:

% for vol in I III; do (./bin/chapters $vol; ./bin/images $vol) >> TARGETS: done
% vim Makefile                                                                 
[...]
dl:
  wget -i TARGETS
[...]