Add rel="nofollow" on Versions links

[jsha]

Part of #1438. This avoids having search engines was time fetching versioned links which are just going to be canonical'ed away.

[syphar]

personally I would be fine with this, before I dig into testing & review I would like to have a last approval by @jyn514 and @Nemo157 , though I remember at least @jyn514 was already fine with this once.

[jyn514]

This seems fine, yeah, not sure of any downsides here other than the old versions not showing up in Google. People can always use our version selector if they want a specific version.

[syphar]

@jsha

• will this be enough for google or should we also set meta/no-index on these pages?
• this will still be incomplete, right? I'm thinking about the links to the crate dependencies in the dropdown that can point to old versions, or the recently extended release-lists (Include all versions of a crate on recent + recent-failures pages #1835)

[jsha]

will this be enough for google or should we also set meta/no-index on these pages?

I don't want to noindex the explicitly-versioned pages because it's possible there's some item name that only exists in old versions. In that case, the old pages won't be canonicalized away (because their <link rel="canonical"> points to a 404), and may still be findable via search engines.

Of course, those search engines would still have to find that old page, which this PR would make harder since we are encouraging them not to crawl explicitly versioned URLs, by removing some links to those URLs. However, other links may still exist on the web, and search engines may still discover old URLs that way. I realize this is a bit hand-wavy but it seems like the right balance to me at the moment. :-)

this will still be incomplete, right?

I think it's complete. This is a macro (macro releases_list) that is used (in my local checkout) in the topbar and in the crate details page.

[syphar]

will this be enough for google or should we also set meta/no-index on these pages?

I don't want to noindex the explicitly-versioned pages because it's possible there's some item name that only exists in old versions. In that case, the old pages won't be canonicalized away (because their <link rel="canonical"> points to a 404), and may still be findable via search engines.

Of course, those search engines would still have to find that old page, which this PR would make harder since we are encouraging them not to crawl explicitly versioned URLs, by removing some links to those URLs. However, other links may still exist on the web, and search engines may still discover old URLs that way. I realize this is a bit hand-wavy but it seems like the right balance to me at the moment. :-)

kk, understood.

this will still be incomplete, right?

I think it's complete. This is a macro (macro releases_list) that is used (in my local checkout) in the topbar and in the crate details page.

I won't include the list of dependencies, which also can (indirectly) point to old versions of crates.

But assuming we don't want to block everything for google, but only some old versions, I'm fine with this.