pre-commit URL check fails on 403 - needs to pass them

[TheBigBear]

The pre-commit hook fails on URLs that actually do work.

Can this be fixed? @dafyddj

This is branch update-rakkudo-star on my own repo here

---( rakudo-star-no-jit_x86.sls )---
58
PROBLEM HERE (403) : rakudo-star-no-jit-2016.01_x86 -- 1.0.0 -- https://rakudo.org/dl/star/rakudo-star-2016.01-01-win-x86-(no%20JIT).msi 
59
PROBLEM HERE (Bad content type) : rakudo-star-no-jit-2016.01_x86 -- 1.0.0 -- https://rakudo.org/dl/star/rakudo-star-2016.01-01-win-x86-(no%20JIT).msi -- text/plain 
60
---( rakudo-star_x64.sls )---
61
PROBLEM HERE (403) : rakudo-star-2020.05.1_x64 -- 1.0.0 -- https://rakudo.org/dl/star/rakudo-star-2020.05.1-01-win-x86_64-(JIT).msi 
62
PROBLEM HERE (Bad content type) : rakudo-star-2020.05.1_x64 -- 1.0.0 -- https://rakudo.org/dl/star/rakudo-star-2020.05.1-01-win-x86_64-(JIT).msi -- text/plain 
63
PROBLEM HERE (403) : rakudo-star-2020.01_x64 -- 1.0.0 -- https://rakudo.org/dl/star/rakudo-star-2020.01-01-win-x86_64-(JIT).msi 
64
PROBLEM HERE (Bad content type) : rakudo-star-2020.01_x64 -- 1.0.0 -- https://rakudo.org/dl/star/rakudo-star-2020.01-01-win-x86_64-(JIT).msi -- text/plain 
65
PROBLEM HERE (403) : rakudo-star-2019.03_x64 -- 1.0.0 -- https://rakudo.org/dl/star/rakudo-star-2019.03-01-win-x86_64-(JIT).msi 
66
PROBLEM HERE (Bad content type) : rakudo-star-2019.03_x64 -- 1.0.0 -- https://rakudo.org/dl/star/rakudo-star-2019.03-01-win-x86_64-(JIT).msi -- text/plain 
67
PROBLEM HERE (403) : rakudo-star-2018.10_x64 -- 1.0.0 -- https://rakudo.org/dl/star/rakudo-star-2018.10-01-win-x86_64-(JIT).msi 
68
PROBLEM HERE (Bad content type) : rakudo-star-2018.10_x64 -- 1.0.0 -- https://rakudo.org/dl/star/rakudo-star-2018.10-01-win-x86_64-(JIT).msi -- text/plain 
69
PROBLEM HERE (403) : rakudo-star-2018.06_x64 -- 1.0.0 -- https://rakudo.org/dl/star/rakudo-star-2018.06-01-win-x86_64-(JIT).msi 
70
PROBLEM HERE (Bad content type) : rakudo-star-2018.06_x64 -- 1.0.0 -- https://rakudo.org/dl/star/rakudo-star-2018.06-01-win-x86_64-(JIT).msi -- text/plain 
71
PROBLEM HERE (403) : rakudo-star-2018.04_x64 -- 1.0.0 -- https://rakudo.org/dl/star/rakudo-star-2018.04-01-win-x86_64-(JIT).msi 
72
PROBLEM HERE (Bad content type) : rakudo-star-2018.04_x64 -- 1.0.0 -- https://rakudo.org/dl/star/rakudo-star-2018.04-01-win-x86_64-(JIT).msi -- text/plain 
73
PROBLEM HERE (403) : rakudo-star-2018.01_x64 -- 1.0.0 -- https://rakudo.org/dl/star/rakudo-star-2018.01-01-win-x86_64-(JIT).msi 
74
PROBLEM HERE (Bad content type) : rakudo-star-2018.01_x64 -- 1.0.0 -- https://rakudo.org/dl/star/rakudo-star-2018.01-01-win-x86_64-(JIT).msi -- text/plain 
75
PROBLEM HERE (403) : rakudo-star-2017.10_x64 -- 1.0.0 -- https://rakudo.org/dl/star/rakudo-star-2017.10-01-win-x86_64-(JIT).msi 
76
PROBLEM HERE (Bad content type) : rakudo-star-2017.10_x64 -- 1.0.0 -- https://rakudo.org/dl/star/rakudo-star-2017.10-01-win-x86_64-(JIT).msi -- text/plain 
77
PROBLEM HERE (403) : rakudo-star-2017.07_x64 -- -7 -- https://rakudo.org/dl/star/rakudo-star-2017.07-01-win-x86_64-(JIT).msi 
78
PROBLEM HERE (Bad content type) : rakudo-star-2017.07_x64 -- -7 -- https://rakudo.org/dl/star/rakudo-star-2017.07-01-win-x86_64-(JIT).msi -- text/plain 
79
PROBLEM HERE (403) : rakudo-star-2017.04_x64 -- 1.0.0 -- https://rakudo.org/dl/star/rakudo-star-2017.04-01-win-x86_64-(JIT).msi 
80
PROBLEM HERE (Bad content type) : rakudo-star-2017.04_x64 -- 1.0.0 -- https://rakudo.org/dl/star/rakudo-star-2017.04-01-win-x86_64-(JIT).msi -- text/plain 
81
PROBLEM HERE (403) : rakudo-star-2017.01_x64 -- 1.0.0 -- https://rakudo.org/dl/star/rakudo-star-2017.01-01-win-x86_64-(JIT).msi 
82
PROBLEM HERE (Bad content type) : rakudo-star-2017.01_x64 -- 1.0.0 -- https://rakudo.org/dl/star/rakudo-star-2017.01-01-win-x86_64-(JIT).msi -- text/plain 
83
PROBLEM HERE (403) : rakudo-star-2016.11_x64 -- 1.0.0 -- https://rakudo.org/dl/star/rakudo-star-2016.11-01-win-x86_64-(JIT).msi 
84
PROBLEM HERE (Bad content type) : rakudo-star-2016.11_x64 -- 1.0.0 -- https://rakudo.org/dl/star/rakudo-star-2016.11-01-win-x86_64-(JIT).msi -- text/plain 
85
PROBLEM HERE (403) : rakudo-star-2016.07_x64 -- 1.0.0 -- https://rakudo.org/dl/star/rakudo-star-2016.07-01-win-x86_64-(JIT).msi 
86
PROBLEM HERE (Bad content type) : rakudo-star-2016.07_x64 -- 1.0.0 -- https://rakudo.org/dl/star/rakudo-star-2016.07-01-win-x86_64-(JIT).msi -- text/plain 
87
PROBLEM HERE (403) : rakudo-star-2016.04_x64 -- 1.0.0 -- https://rakudo.org/dl/star/rakudo-star-2016.04-01-win-x86_64-(JIT).msi 
88
PROBLEM HERE (Bad content type) : rakudo-star-2016.04_x64 -- 1.0.0 -- https://rakudo.org/dl/star/rakudo-star-2016.04-01-win-x86_64-(JIT).msi -- text/plain 
89
PROBLEM HERE (403) : rakudo-star-2016.01_x64 -- 1.0.0 -- https://rakudo.org/dl/star/rakudo-star-2016.01-01-win-x86_64-(JIT).msi 
90
PROBLEM HERE (Bad content type) : rakudo-star-2016.01_x64 -- 1.0.0 -- https://rakudo.org/dl/star/rakudo-star-2016.01-01-win-x86_64-(JIT).msi -- text/plain 
91
--------------------------------------------------------------------------------
92

93
Total         16
94
----------  ----
95
text/plain    16
96

97
  Total    16
98
-------  ----
99
    403    16
100

101
Total      16
102
-------  ----
103
failed     16
104

105
Traceback (most recent call last):
106
  File ".travis/tests.py", line 204, in <module>
107
    assert TEST_STATUS, (
108
AssertionError: BUILD FAILING. You can grep for 'PROBLEM HERE' to find out how to fix this.
109

110
Error: The process '/usr/local/bin/pre-commit' failed with exit code 1

[TheBigBear]

pre-commit checks has similar issue on one dl version of vlc, since they introduced a html page with counter before redirecting to Start the download.
and it also has a URL that it thinks fails for patchmypc-free and patchmypc , which actually succeeds

maybe we introduce additional flags one can set in sls file such as 'ignore_403' or 'ignore_500' ... ?