"1 segments are failed to download, retry..."

[svaningelgem]

This is related to #13, but I think it's still different enough to warrant another ticket.

I'm in "v2_multithreaded_processor" line +- 100: one of the "key_segments_pairs" failed, but the retry-counter doesn't jump in to stop the while-loop from retrying endlessly.

Grtz,
Steven

[p3nOnion]

[hbattat]

The max_retry_times is already passed to the crawler, you can just use it
line 100 would be

max_rety_times = self.max_retry_times