New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
purl.obolibrary.org URLs return 403 when using urllib.request (Python) #923
Comments
Please provide example PURLs and code so we can try to replicate. |
|
Is this behaviour new? This works:
But I am wondering if the recent changes to PURL system now cause: https://www.pythonpool.com/urllib-error-httperror-http-error-403-forbidden The wheregoes trace works: |
Yes it's new. I think it still worked on Friday. Maybe some abuse-suspecting user agent block by GitHub? |
On Friday we changed something in our PURL config, cc @kltm, but it is a bit odd that all other tools other than Thanks for the report! |
EDIT: Posted too soon, the following is incorrect.
import urllib.request
url = "https://github.com/obophenotype/human-phenotype-ontology/releases/latest/download/hp.owl"
response = urllib.request.urlopen(url) |
Normal
|
Using @eliasweatherfield code in try catch also works:
This suggests that the request is successful, but the error is thrown regardless. |
I can confirm the 403 described by @eliasweatherfield in Python 3.9 and 3.11. I think @matentzn is seeing an old |
Ok, now I think that Cloudflare is rejecting the request, which makes sense given the timing of this issue: import urllib.request
url = "http://purl.obolibrary.org/obo/hp.owl"
try:
response = urllib.request.urlopen(url)
except urllib.error.HTTPError as e:
print(e)
print(e.code)
print(e.reason)
print(e.headers)
|
I think the cause is Cloudflare's Browser Integrity Check, which is a security setting that can be turned off: https://developers.cloudflare.com/support/firewall/settings/understanding-the-cloudflare-browser-integrity-check/ |
@jamesaoverton I believe that I've turned off BIC for this domain (Cloudflare docs are apparently wildly out of date and not great to begin with). |
Thanks @kltm! I'm now getting a 200 response from the first test code posted above -- no more error. @eliasweatherfield Can you confirm that this is now working for you? |
Yes, it's working again. Thanks everyone. |
Thanks for the report! |
No description provided.
The text was updated successfully, but these errors were encountered: