Fix for the Cloudflare 524 Timeout Error after 99s

[dehlirious]

The crawl.php code as it is, at line 160 (line 153 for crawl-manual) could use these bits:
$spacer_size = 8; // increment me until it works echo str_pad('', (1024 * $spacer_size), "\n"); if(ob_get_level()) ob_end_clean();

It will prevent a inevitable 524 Timeout if you are using Cloudflare services (ran into this issue myself)

Source of fix: https://github.com/marcialpaulg/Fixing-Cloudflare-Error-524

Example in action:
dd72700