New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Enabling SSL Support #1
Comments
Hi. I'm glad you appreciate this package as well. This version can connect to https already. There's an additional command line option "-https" to send but otherwise it works as for http. Have you tried this? If it doesn't work for you, please let me know. Thank you. |
Hi, That seemed to work, but it's only checking the main page (and not really even parsing it as far as I can tell). I suspect there may be a robot restriction in place. I seem to recall that I used to be able to change the User-Agent but -http_header is not recognized (although it is in the documentation). I'd be happy to submit PRs but my Perl skills are virtually non-existent. I'm posting a couple of files of output to see if you can spot the problem. Given this command file:
I get the following result
warn.txt contains:
And httpfail.txt contains:
Any ideas what I'm doing wrong? Thanks in advance! |
Checking. Your site behaves differently than mine. |
In what regard? Asking because we have about 150 sites all configured the same on the same server. We've always understood that our server configuration is pretty standard. If linklint can't handle a server with a standard configuration, maybe it could somehow be updated? Do you have any test servers that are known to work that you can share? |
In the regard that linklint works for my sites, but not for yours. For example, this works (my site): This does not: I note that the
However, the
Can you test linklint against your website without going through Cloudflare? Is it different? |
That's very odd but I'm going to guess it's the server (SiteGround) firewall. I can't see the return code in the output files (an enhancement I'd love to add, btw) but with Cloudflare disabled I'm not seeing any difference. FWIW,
Could you send me the exact Perl command you are sending with instructions on how to execute it, if needed? My server is running HTTP/2.0. Is it possible that linklint is parsing the response string and since it is not finding HTTP/1.n it is giving up? I am seeing this in the warning.txt file: Also, I'm pretty sure that part of the issue is the User-Agent. IIRC, linklint used to allow you to change the User-Agent but I'm unable to figure out how. Any ideas there? Thanks for helping figure this out. Hopefully whatever resolution we find will help others. |
Actually, I now see how LinkLint generates the User-Agent: Line 3019 in 5b4c7cd
As it could be used for abuse, I'll just change my local version of the source code to pass in the User-Agent of an actual browser. I know for a fact that on another server we manage linklint is being blocked simply because of the User-Agent. |
I've been doing some testing. I'm beginning to believe it is an issue with sites running on nginx vs. apache. For example, the following sites, believed to be running apache, seem to work: And the following sites, all using nginx seem to fail with a warning (and from time to time an error) My testing has not be exhaustive and there is no reasonable reason why the type of server should be the culprit if they are all speaking the same protocol but this does seem to be a deciding factor. If I have some more time later I'll try setting up a local nginx server and examine the headers more closely to see if I can spot what's failing. Thanks again for listening :-) |
Wow, ok. I hadn't run in to the issue with the type of server. Maybe it is, as you say, related to the HTTP protocol version number. I notice that linklint does not use a standard perl module for issuing requests, instead it rolled its own. That code should be replaced with a module. Error handling in linklint is also poor, and needs to show better messages when it fails. Likely, adding a standard HTTP module would help with error handling. It's on my todo list. Is there anything else you were looking for? |
I think that the response code is an important thing to include in any report for all URLs (if it's not included already). For now, that's pretty much it. Linklint has always produced just the right amount of data for my needs. As you stated, linklint needs a standard HTTP Request module and I bet that will resolve a majority of issues. This one looks like a likely candidate for use: https://metacpan.org/pod/HTTP::Request If I get some time, I'll try and code it up myself and submit a PR (but don't hold your breath 😆). |
Thank you for reporting this issue, and helping me work through it. |
First of all, I am thrilled to see someone picking up the ball on this project. I love(d) linklint and was really disappointed when I realized I couldn't use it reliably with HTTPS web sites, and thus, my query: How can I use this version of linklint to check links on an HTTPS web site?
TIA and thanks a lot for giving this a go!
The text was updated successfully, but these errors were encountered: