scanner doesn't switch to next protocol in some cases

[poeml]

                                                                                                      [          ]

Issue migrated (2015-06-05) from old issue tracker http://mirrorbrain.org/issues/issue11

Title    scanner doesn't switch to next protocol in some cases
 Priority   bug             Status          deferred
Superseder                 Nosy List        poeml
Assigned To                Keywords         scanner

msg25 (view) Author: poeml Date: 2009-10-08.08:38:23

Normally, the scanner falls back to another protocol if an URL doesn't work. For some reason, this
doesn't seem to happen when the (broken) opensuse-linuxmigratio.at mirror is being scanned. There,
rsync is unreachable and HTTP is reachable (although it doesn't serve any files anymore).

The scanner should fall back to HTTP and clean up the file list from there, but it doesn't happen:

% mb scan migratio
Thu Oct 8 10:24:47 2009 opensuse-linuxmigratio.at: starting
Thu Oct 8 10:24:49 2009 opensuse-linuxmigratio.at: total files before scan: 98701
DIE: (=> /usr/bin/scanner 312 main::rsync_readdir => /usr/bin/scanner 971 main::rsync_get_filelist)
opensuse-linuxmigratio.at: connect: Connection refused
Completed in 1 seconds

As a result, the file list is not cleaned up and the mirror continues to be used.

msg107 (view) Author: poeml Date: 2009-12-11.21:20:19

issue 28 seems to be another case where the fallback to the next protocol doesn't
happen as expected.

msg110 (view) Author: poeml Date: 2009-12-11.22:17:23

Slightly different reason, same symptom:

mirror.pacific.net.au: @error: max connections (15) reached -- try again later
Fri Dec 11 20:53:44 2009 mirror.pacific.net.au: starting
Fri Dec 11 20:53:45 2009 mirror.pacific.net.au: total files before scan: 2516
DIE: (=> /usr/bin/scanner 310 main::rsync_readdir => /usr/bin/scanner 972
main::rsync_get_filelist)
mirror.pacific.net.au: [#2, id=102 pid=9324 exit: 65280]

History
         Date              User       Action            Args
2014-01-22 20:05:20 poeml           set    files: - neli15.html
2013-10-17 17:28:58 funnycafeteria6 set    files: + neli15.html
2009-12-11 22:17:23 poeml           set    messages: + msg110
2009-12-11 21:20:19 poeml           set    messages: + msg107
2009-12-07 03:15:54 poeml           set    status: unread -> deferred
2009-12-01 20:55:59 poeml           set    keyword: + scanner
2009-10-08 08:38:23 poeml           create

(end of migrated issue)