Remove non-canonical docs versions from sitemap.xml

[tobiasmcnulty]

Acceptance criteria

• Board meeting minutes are accessible only on the www subdomain, not on the docs subdomain
• Community pages such as /community/desktops/ and /community/logos/ are available only on the www subdomain, not the docs subdomain
• If needed, links are updated or redirects are added to avoid 404s for these pages (investigation needed)

Original report

It looks like we may be sending confusing signals to Google by putting old docs versions in sitemap-<lang>.xml, but then telling it only one of those versions is canonical:

Google's commentary on this error: https://support.google.com/webmasters/answer/7440203#submitted_url_not_selected_as_canonical

[tobiasmcnulty]

I've also taken a snapshot of the index coverage report today (3/2/19) here, with some commentary and links to other, related issues: https://docs.google.com/spreadsheets/d/1l86YAEcw5CbvivuY-ZN81oy75Nh7T9g0eJX0sZ8Jww0/edit#gid=0

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[carltongibson]

@tobiasmcnulty — Could I ask you to confirm whether this still applies? (It looks quite addressable I think 🤔)

[tobiasmcnulty]

It looks like Google has changed the error message here, but maybe this is fixed? There are far fewer of the similar error, and none for URLs of the documentation.

That said, there appears to be a related issue with the 'docs' site (and possibly others) serving URLs that aren't meant for that site. We could rename this issue, leave it as is, or file a new one?

https://docs.djangoproject.com/foundation/donate/ | Oct 5, 2022
https://docs.djangoproject.com/foundation/conferences/ | Sep 26, 2022
https://docs.djangoproject.com/community/logos/ | Sep 22, 2022
https://docs.djangoproject.com/foundation/corporate-membership/ | Sep 19, 2022
https://docs.djangoproject.com/community/desktops/ | Sep 10, 2022
https://docs.djangoproject.com/community/badges/ | Sep 4, 2022
https://docs.djangoproject.com/foundation/cla/ | Aug 29, 2022
https://docs.djangoproject.com/foundation/faq/ | Aug 28, 2022
https://docs.djangoproject.com/foundation/records/minutes/2014-10-30/ | Aug 27, 2022
https://docs.djangoproject.com/foundation/records/minutes/2018-08-09/ | Aug 22, 2022
etc.

[arexnnnn]

/assign

[aditi25srivastava]

@tobiasmcnulty hey!!! I'm interestedcould you please assign me or I start contribution and send you PR!!!!

[tobiasmcnulty]

Hi @aditi25srivastava, this looks resolved, except some of the non-docs pages are still available on the docs site.

Here again are some examples:

• https://docs.djangoproject.com/foundation/records/minutes/2017-02-09/
• https://docs.djangoproject.com/foundation/records/minutes/2016-06-09/
• https://docs.djangoproject.com/foundation/records/minutes/2018-11-08/
• https://docs.djangoproject.com/foundation/records/minutes/2017-01-12/
• https://docs.djangoproject.com/foundation/records/minutes/2018-03-08/
• https://docs.djangoproject.com/community/desktops/
• https://docs.djangoproject.com/community/logos/
• https://docs.djangoproject.com/foundation/records/minutes/2018-07-12/
• https://docs.djangoproject.com/foundation/records/minutes/2015-04-02/
• https://docs.djangoproject.com/foundation/records/minutes/2014-02-06/

The fix is to make these URLs accessible only from the www subdomain (in particular, the issue seems to affect paths beginning with /community/ and /foundation/).

I'll update the issue description. Feel free to assign yourself and make a PR if it's of interest!

[aditi25srivastava]

@tobiasmcnulty heyy I'm interested I'll send you a PR within week .....thank you!!!