Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use rel="canonical" for indexable stuff? #1400

Closed
systemed opened this issue Dec 31, 2016 · 18 comments
Closed

Use rel="canonical" for indexable stuff? #1400

systemed opened this issue Dec 31, 2016 · 18 comments

Comments

@systemed
Copy link
Contributor

OSM content sometimes shows up in Google at odd URLs rather than the standard openstreetmap.org domain, for example:

https://www.google.co.uk/search?q=site%3A89-16-162-21.no-reverse-dns-set.bytemark.co.uk

There is a Host: directive in robots.txt which should guard against this, but it appears that only Yandex supports it. Google's preferred solution appears to be <link rel="canonical" ...>. Probably only a nice-to-have and a workaround for a Google infelicity, but it could be good to support this.

@tomhughes
Copy link
Member

Well WTF did they get that name from? I mean I know it's reverse DNS but they don't just crawl random IPs do they, they just follow links from other sites, and nothing should be linking to that.

Where exactly do you propose we put that attribute? On every link? If so is there some way to make rails do that automatically?

@HolgerJeromin
Copy link
Contributor

This is intended to be in every web page.

But perhaps the content should be served only with the www.openstreetmap.org host name.
And redirect from all other which will catch openstreetmap.org and osm.org, too.
Could be easier to configure and maintain

@tomhughes
Copy link
Member

Where in every page? What tag do I add that attribute to? Every a tag?

We do redirect the actual published names (that aren't www.openstreetmap.org) but we don't redirect random made up nonsense because we can't predict what it might be. Is there even such a thing as a wildcard ServerAlias directive?

The only reason those names are working at all is because apache is default to the first virtual host as they don't match any ServerName or ServerAlias directive.

@tomhughes
Copy link
Member

Look like a wildcard ServerAlias might be a possibility actually... You can write one and I think an explicit name will take preference which is what we would need.

@Firefishy
Copy link
Member

I'd recommend a <link rel="canonical" ...> always linking to https://www.openstreetmap.org page url regardless of domain used to reach the site. It makes a health impact on SEO.

@Firefishy
Copy link
Member

Example for / url would be <link rel="canonical" href="https://www.openstreetmap.org/" /> and http://www.openstreetmap.org/about would be <link rel="canonical" href="https://www.openstreetmap.org/about" />

@HolgerJeromin
Copy link
Contributor

@tomhughes As noted above this is one new meta tag in head, no attribute to a-elements
In my experience the order of settings in Apache configuration is relevant.
So first www.openstreetmap.org config and after that an entry with the catch all with the redirect

@systemed
Copy link
Contributor Author

systemed commented Jan 1, 2017

There's a gem at https://github.com/jumph4x/canonical-rails which does canonical tags in the <head>. But solving it via Apache is probably easier and neater.

@tomhughes
Copy link
Member

I've applied openstreetmap/chef@e20f90d which should cause those names to be redirected.

@HolgerJeromin
Copy link
Contributor

Seems to work. Good work (as usual :-)
I am pretty sure that Google will clean up while the next update so the issue can be closed?

@systemed
Copy link
Contributor Author

systemed commented Jan 1, 2017

\o/

@systemed systemed closed this as completed Jan 1, 2017
@mmd-osm
Copy link
Contributor

mmd-osm commented Jan 3, 2017

A Merkaartor user reported an issue on the German Forum, which presumably is related to this change. Instead of the actual OSM XML payload, they're seeing some HTTP 301 status page now. Needless to say that the user has not much of an idea, what is going on here and already started to check for current issues on the Platform Status page on the Wiki. Obviously Merkaartor does not follow HTTP 301 and needs to be fixed I guess.

https://forum.openstreetmap.org/viewtopic.php?pid=624794#p624794

@tomhughes
Copy link
Member

tomhughes commented Jan 3, 2017

Well they'll only get a 301 if they're using a weird hostname. If they use www.openstreetmap.org or api.openstreetmap.org then there's no redirect, and most other names were already redirecting. It's only really odd things like that reverse DNS based name that have changed.

@tomhughes
Copy link
Member

Or I guess possibly if Merkaartor is HTTP/0.9 and doesn't send a host header!

@mmd-osm
Copy link
Contributor

mmd-osm commented Jan 3, 2017

I cannot really reproduce this here due to lack of a working Merkaartor installation. However, it seems like they're using some completely outdated version dating back to 2010/11. Urgent advise from my side was to first update and try again. Let's see if this works ...

@mmd-osm
Copy link
Contributor

mmd-osm commented Jan 6, 2017

Newest update from User:wycbtma was that even the API upload doesn't work anymore on Merkaartor 0.17 (see screenshot below). Requested more details, such as exact URL now. There were also some questions, if the latest Merkaartor version is also affected. User reported that the .osc file could be successfully uploaded via Level0 editor. No response yet, why moving to a newer version failed so far. Very strange, indeed.

@tomhughes
Copy link
Member

Really there's no point in reposting exactly what you've already told us. That picture adds nothing over the text of the error posted before.

In any case it doesn't belong here - if you can find an actual problem with the API then please open a new ticket but I think it's very unlikely there is one. This is almost certainly a Merkaartor issue.

@openstreetmap openstreetmap locked and limited conversation to collaborators Jan 6, 2017
@tomhughes
Copy link
Member

As a test I just made an edit using Merkaartor 0.18.2 (http://www.openstreetmap.org/changeset/44950760) and had no problems at all..

I would get the user to check what API URL they have set in the settings and if that isn't the problem then we'll really need a packet trace to demonstrate a problem.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants