When we get to managing multiple replicates of content for Swift.org, one of the pieces we'll absolutely want to sort out is a definitive choice on "which version is canonical", and including that within the HTML output that's served for the static content.
The relevant tag is included in HTML's <head> tags, using the form: <link rel="canonical">
The gist and impact, particularly from search engines, is that replicated content that either doesn't have such a tag, or doesn't agree on that tag, treated replicated content as potentially spam, and the indexing engines may simply directly "de-list" the content from their index.
This will have a broad effect on any replicated content when we have versioned copies of the Swift.org documentation collection, and perhaps may be hitting us even earlier, with some content replicated between developer.apple.com and what we're including in the combined documentation to host at docs.swift.org.
This may be a DocC feature request, or something that we have to enable post-processing to inject into the relevant HTML of the various pages, and we'll want to make sure to keep this coordinated with any documentation hosting for relevant libraries on Swift Package Index as well.
/cc @daveverwer
(Notes for myself on canonical tags)
What <link rel="canonical"> is:
It's a hint in the HTML <head> telling search engines "if you've seen this content elsewhere, this URL is the one to index and rank." It doesn't block indexing of the other copy — it tells the crawler which copy should own the search-index entry, consolidating ranking signals onto one URL instead of splitting/penalizing them across duplicates.
On the canonical page itself
A self-referencing canonical — the tag points at the page's own URL:
<head>
...
<link rel="canonical" href="https://docs.swift.org/documentation/swift/array">
...
</head>
This is the version search engines should treat as authoritative.
On a mirror page (that references it)
Same tag, but href points at the other page's URL — the canonical one — not its own:
<head>
...
<!-- this page is served at https://developer.apple.com/documentation/swift/array -->
<link rel="canonical" href="https://docs.swift.org/documentation/swift/array">
...
</head>
The mirror is still fully served and browsable by humans; it's telling crawlers "don't index me separately, credit the docs.swift.org copy instead."
When we get to managing multiple replicates of content for Swift.org, one of the pieces we'll absolutely want to sort out is a definitive choice on "which version is canonical", and including that within the HTML output that's served for the static content.
The relevant tag is included in HTML's
<head>tags, using the form:<link rel="canonical">The gist and impact, particularly from search engines, is that replicated content that either doesn't have such a tag, or doesn't agree on that tag, treated replicated content as potentially spam, and the indexing engines may simply directly "de-list" the content from their index.
This will have a broad effect on any replicated content when we have versioned copies of the Swift.org documentation collection, and perhaps may be hitting us even earlier, with some content replicated between developer.apple.com and what we're including in the combined documentation to host at docs.swift.org.
This may be a DocC feature request, or something that we have to enable post-processing to inject into the relevant HTML of the various pages, and we'll want to make sure to keep this coordinated with any documentation hosting for relevant libraries on Swift Package Index as well.
/cc @daveverwer
(Notes for myself on canonical tags)
What
<link rel="canonical">is:It's a hint in the HTML
<head>telling search engines "if you've seen this content elsewhere, this URL is the one to index and rank." It doesn't block indexing of the other copy — it tells the crawler which copy should own the search-index entry, consolidating ranking signals onto one URL instead of splitting/penalizing them across duplicates.On the canonical page itself
A self-referencing canonical — the tag points at the page's own URL:
This is the version search engines should treat as authoritative.
On a mirror page (that references it)
Same tag, but href points at the other page's URL — the canonical one — not its own:
The mirror is still fully served and browsable by humans; it's telling crawlers "don't index me separately, credit the docs.swift.org copy instead."