Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Append relative base href to siteURL #99

Merged
merged 2 commits into from
May 17, 2024

Conversation

IndrekHaav
Copy link
Contributor

@IndrekHaav IndrekHaav commented May 3, 2024

Favicon extraction fails when the website's base href is set to a relative URL. For example, printables.com:

$ wget -q -O - https://printables.com | grep 'base href'
  <base href="/">

In this case fetching the icons fails because only the relative URL is used:

$ PORT=8000 bin/iconserver
iconserver v3.18.0 () (go1.21.1) - https://github.com/mat/besticon
server: 2024/05/03 14:23:12.407321 Starting server on 0.0.0.0:8000...
http:  2024/05/03 14:23:14.890857 GET http://printables.com 200 357.50ms -1
http:  2024/05/03 14:23:14.893386 Error: GET http:///favicon.ico Get "http:///favicon.ico": http: no Host in request URL 0.08ms
http:  2024/05/03 14:23:14.893609 Error: GET http:///assets/favicons/favicon-16x16.png Get "http:///assets/favicons/favicon-16x16.png": http: no Host in request URL 0.02ms
http:  2024/05/03 14:23:14.893601 Error: GET http:///assets/favicons/apple-touch-icon.png Get "http:///assets/favicons/apple-touch-icon.png": http: no Host in request URL 0.05ms
http:  2024/05/03 14:23:14.893640 Error: GET http:///apple-touch-icon-precomposed.png Get "http:///apple-touch-icon-precomposed.png": http: no Host in request URL 0.03ms
http:  2024/05/03 14:23:14.893713 Error: GET http:///apple-touch-icon.png Get "http:///apple-touch-icon.png": http: no Host in request URL 0.01ms
http:  2024/05/03 14:23:14.893705 Error: GET http:///assets/favicons/favicon-32x32.png Get "http:///assets/favicons/favicon-32x32.png": http: no Host in request URL 0.02ms
server: 2024/05/03 14:23:14.895058 GET /icons?url=printables.com 404 "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:125.0) Gecko/20100101 Firefox/125.0"  393.43ms 2780
^C

Note all the "no Host in request URL" errors. This PR should fix this by appending relative base href values to siteURL:

$ PORT=8000 bin/iconserver
iconserver v3.18.0 () (go1.21.1) - https://github.com/mat/besticon
server: 2024/05/03 14:23:48.280063 Starting server on 0.0.0.0:8000...
http:  2024/05/03 14:23:53.088582 GET http://printables.com 200 353.14ms -1
http:  2024/05/03 14:23:53.111309 GET https://www.printables.com/assets/favicons/apple-touch-icon.png 200 17.58ms 1867
http:  2024/05/03 14:23:53.112752 GET https://www.printables.com/apple-touch-icon.png 200 18.87ms -1
http:  2024/05/03 14:23:53.113204 GET https://www.printables.com/apple-touch-icon-precomposed.png 200 19.53ms -1
http:  2024/05/03 14:23:53.113392 GET https://www.printables.com/favicon.ico 200 19.62ms -1
http:  2024/05/03 14:23:53.113405 GET https://www.printables.com/assets/favicons/favicon-32x32.png 200 19.61ms 781
http:  2024/05/03 14:23:53.113232 GET https://www.printables.com/assets/favicons/favicon-16x16.png 200 19.68ms 616
server: 2024/05/03 14:23:53.116060 GET /icons?url=printables.com 200 "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:125.0) Gecko/20100101 Firefox/125.0"  384.42ms 5339
^C

Apologies in advance if the code is not up to best practices, this is literally my first time writing Golang.

@mat mat merged commit 22baa9a into mat:master May 17, 2024
3 checks passed
@mat
Copy link
Owner

mat commented May 17, 2024

Thank you for your contribution and the well written PR description, @IndrekHaav

@IndrekHaav IndrekHaav deleted the fix-relative-base-href branch May 17, 2024 17:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
2 participants