-
Notifications
You must be signed in to change notification settings - Fork 130
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Strip URLs before adding slashes #2015
base: master
Are you sure you want to change the base?
Strip URLs before adding slashes #2015
Conversation
fd5c322
to
f954b95
Compare
In general, URL manipulation has specified RFC semantics, and we should be using libraries to do that, e.g. https://docs.python.org/3/library/urllib.parse.html (specifically urlsplit and urljoin) For example:
that way we delegate the handling of slashes to RFC standards to people who've already done the legwork |
that's a good suggestion, I will draft this and code that later. |
f954b95
to
c8d3d32
Compare
999899b
to
e6c16b2
Compare
The worker (python 3.6.x) stops, wrong url, missing the repo info. (all urls before the wrong one are fine).
Running master vs master
Variable task sizes used. Opening offset = 14196
Downloading https://api.github.com/zipball/6138a0fd0e43753a86e4a170a5f6e2b7b6752677
Exception in requests.get():
404 Client Error: Not Found for url: https://api.github.com/zipball/6138a0fd0e43753a86e4a170a5f6e2b7b6752677
Exception running games:
Get request to https://api.github.com/zipball/6138a0fd0e43753a86e4a170a5f6e2b7b6752677 failed
Running master vs master
Variable task sizes used. Opening offset = 14022
Downloading https://api.github.com/repos/official-stockfish/Stockfish/zipball/6138a0fd0e43753a86e4a170a5f6e2b7b6752677
Build uses default net: nn-ddcfb9224cdb.nnue |
it looks like this library gives 3 different combination results of joining two URLs when using
hopefully I'm missing something |
@dubslow unfortunately your example seems to give three different results based on the slashes, I don't think splitting and then unsplitting, and then joining is reasonable even if it's done in a helper function..
|
9a6b1a7
to
49f4d6d
Compare
this adds the practice of always stripping before adding slashes to the URLs.. one example is that the run page doesn't work with a repo with a slash at the end.
49f4d6d
to
ea179ac
Compare
this adds the practice of always stripping before adding slashes to the URLs.. one example is that the run page doesn't work with a repo with a slash at the end.