Remove generic locales differentiation page

[stephaniehobson]

Remove https://www.mozilla.org/locales/ and supporting code.

This page was identified as a candidate for removal in our page audit and we have confirmed it does not serve users or SEO and is safe to remove.

[stevejalim]

Re "supporting code" -- are we going to continue to serve this as the default page if you do to www.mozilla.org/ with no lang detectable (i.e if you're a search bot)?

IIRC this path was added as a way to surface the same content when you're not a bot

[janbrasna]

@stevejalim It's actually not the exact same page (but I'll investigate how much they all have in common under the hood tho…)

There's still a slight difference between / for unset Accept-Language, /404-locale (that looks alike but with different copy), and /locales linked from footer that has the nice separation by regions (:)

(Nonetheless the /locales ended up showing languages not available in prod, like af ach ff son uz xh etc. leading to default anyways, so this "user" page is not actually as useful as the two "system" pages that only offer enabled translations.)

TODO: The offered languages, esp. from the bot/baseline root page, should all be working. I've found some in prod that link to 404 — e.g. Tadjik (Тоҷикӣ) /tg is being offered there, but not working.

I'm currently trying to read up the exact background for all the changes over time… Basically:

• SEO - Build a country/language selector page #5827
  (via SEO Tweaks: Hreflang Fixes #5652)
• Add language selection page (Fixes #5827) #6397
• Automate /locales/ page via a configurable list in settings #6454
• Automate /locales/ page based on settings (fixes #6454) #11546
• Link to /locales/ page from the main footer #6455
• Do not redirect users/bots to localized content based on header information #11538 ⚠️ ❗

[stevejalim]

Thanks for doing the digging there @janbrasna while I idly checked Github notifications while on PTO ;)

Yeah, / for unset Accept-Language and, related, /404-locale were requested specifically by @a-kyne to ensure that bots got sent to a locale-selection page, rather than forcing them all to en-US.

I forgot that /locales/ is not the same content/purpose as those. As long as the first two links in this reply remain live, I'm happy.