Errors should not escape black holes #994

Merged
merged 2 commits into from Dec 23, 2016

Projects

None yet

2 participants

@DL6ER
Member
DL6ER commented Dec 17, 2016 edited

By submitting this pull request, I confirm the following (please check boxes, eg [X])Failure to fill the template will close your PR:

  • I have read and understood the contributors guide.
  • I have checked that another pull request for this purpose does not exist.
  • I have considered, and confirmed that this submission will be valuable to others.
  • I accept that this submission may not be used, and the pull request closed at the will of the maintainer.
  • I give this submission freely, and claim no ownership to its content.

How familiar are you with the codebase?:

  • 10 (very familiar)

Currently we try to download the lists and if the resulting file is an empty file we interpret it as being not downloaded because there is no update for this list available.

However, that might be wrong, since there could also be another error causing this: Forbidden (Error 403), Not Found (Error 404), Time-Out (Error 408), Internal Server Error (Error 500), among others.

This PR implements a proper error handling to avoid accreting particles with zero mass (i.e. empty files) and thinking that that causes no harm to the accretion disk (in the end we assume that we will always have a list, even if it is a cached one).

Also, this PR adds a check for the existence of a file that we want to aggregate to the list of domains. This is error-prone since the list might have never gotten downloaded correctly.

Exemplary output:

::: Getting raw.githubusercontent.com list... done
:::   Status: Success (OK)
:::   List updated, transport successful!
::: Getting mirror1.malwaredomains.com list... done
:::   Status: Not modified
:::   No changes detected, transport skipped!
::: Getting sysctl.org list... done
:::   Status: Not modified
:::   No changes detected, transport skipped!
::: Getting zeustracker.abuse.ch list... done
:::   Status: Not modified
:::   No changes detected, transport skipped!
::: Getting s3.amazonaws.com list... done
:::   Status: Not modified
:::   No changes detected, transport skipped!
::: Getting s3.amazonaws.com list... done
:::   Status: Not modified
:::   No changes detected, transport skipped!
::: Getting hosts-file.net list... done
:::   Status: Not modified
:::   No changes detected, transport skipped!
::: Getting raw.githubusercontent.com list... done
:::   Status: Success (OK)
:::   List updated, transport successful!

This template was created based on the work of udemy-dl.

DL6ER added some commits Dec 17, 2016
@DL6ER DL6ER Improvements for gravity (errors shouldn't be able to escape the blac…
…k hole if even light cannot do this!)
efaee2b
@DL6ER DL6ER Better output if file has not been downloaded because it has not been…
… changed
cd0c3f9
@DL6ER DL6ER added the Enhancement label Dec 17, 2016
@DL6ER DL6ER changed the title from Errorsshouldnotescapeblackholes to Errors should not escape black holes Dec 17, 2016
@DL6ER DL6ER added this to the v2.11 milestone Dec 19, 2016
@DL6ER
Member
DL6ER commented Dec 22, 2016

Ready for review and merge.

@DL6ER DL6ER requested a review from jacobsalmela Dec 22, 2016
@PromoFaux
Member
PromoFaux commented Dec 23, 2016 edited

Approved

Approved with PullApprove

@DL6ER DL6ER merged commit c364fd8 into development Dec 23, 2016

5 checks passed

codacy/pr Good work! A positive pull request.
Details
code-review/pullapprove Approved by DL6ER, PromoFaux
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
continuous-integration/travis-ci/push The Travis CI build passed
Details
licence/cla Contributor License Agreement is signed.
Details
@DL6ER DL6ER deleted the errorsshouldnotescapeblackholes branch Dec 23, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment