Don't use bad canonical links

[pushcx]

On a story that was posted twice with a bad link, emmab suggested:

Modify Lobsters to not follow rel=canonical unless it can resolve a page with status 2**?

I think it's a good idea. I'm posting this issue for discussion and hopefully someone with free time can implement it.

[pushcx]

For more context: a lot of sites have bad <link rel=canonical> tags in their headers. The two most common behaviors are pointing it at the homepage (they didn't understand what the tag is for) and not updating one that comes in a template (like in this story's case, pointing at username.github.com). I contact site owners to tell them about the bug when I can easily find contact info, but it would be nice if Lobsters could catch the second case.

Now that I write this up, I think we should probably also ignore any canonical link that points right to a domain name. It's the most popular failure mode, and if a Lobsters user is submitting a homepage with one the system would correctly use the URL they entered anyways.

[pushcx]

For more context: a lot of sites have bad <link rel=canonical> tags in their headers. The two most common behaviors are pointing it at the homepage (they didn't understand what the tag is for) and not updating one that comes in a template (like in this story's case, pointing at username.github.com). I contact site owners to tell them about the bug when I can easily find contact info, but it would be nice if Lobsters could catch the second case.

Now that I write this up, I think we should probably also ignore any canonical link that points right to a domain name. It's the most popular failure mode, and if a Lobsters user is submitting a homepage with one the system would correctly use the URL they entered anyways.