[bug] workflow runs are failing intermittently (and running 10x longer than intended)

[nberlette]

It seems that about 1 in every 5-10 runs is currently failing to complete, and that's after it has run for an average of 10x minutes longer than a typical successful run.

A normal workflow run without any issues, takes about 45-60 seconds to successfully complete from start to finish. In comparison, the average failed run is currently taking over 6 minutes and 30 seconds before it gives up and fails.

This has something to do with the "retries" logic of the scraper; I have a hunch that when the first attempt fails to retrieve new content, it is running into cached content while making its additional attempts.

I'm investigating the issue currently and will be pushing through a PR to fix it as soon as I can.