SEO: add <meta name="robots" content="noindex" /> to pagination pages after page 1 to tackle duplicate content issues

[johnnyreilly]

Pre-flight checklist

• I have read the Contributing Guidelines on pull requests.
• If this is a code change: I have written unit tests and/or added dogfooding pages to fully verify the new behavior.
• If this is a new API or substantial change: the PR has an accompanying issue (closes #0000) and the maintainers have approved on my working plan.

Motivation

Improves SEO behaviour of Docusaurus by indicating that pagination pages which duplicate content should not be indexed by search engines. This does not apply to the initial page; but would apply to pages 2, 3, 4, 5 ....

See also the relevant portion of my blog on the topic: https://johnnyreilly.com/how-we-fixed-my-seo#1-remove-or-noindex-unnecessary-pages

Test Plan

See screenshots from PR preview below demonstrating presence of:

<meta name="robots" content="noindex" />

Test links

Note there is no noindex on the first page of the blog:

https://deploy-preview-9658--docusaurus-2.netlify.app/blog/

However, consider page 2 where there is a noindex: https://deploy-preview-9658--docusaurus-2.netlify.app/blog/page/2/

Note there is no noindex on the first page of a tags page:

https://deploy-preview-9658--docusaurus-2.netlify.app/blog/tags/release/

However, consider page 2 where there is: https://deploy-preview-9658--docusaurus-2.netlify.app/blog/tags/release/page/2/

Related issues/PRs

#9657

[netlify]

✅ [V2]

Built without sensitive environment variables

Name	Link
🔨 Latest commit	d113b0a
🔍 Latest deploy log	https://app.netlify.com/sites/docusaurus-2/deploys/6584932254aa010008731ff1
😎 Deploy Preview	https://deploy-preview-9658--docusaurus-2.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify site configuration.

[github-actions]

⚡️ Lighthouse report for the deploy preview of this PR

URL	Performance	Accessibility	Best Practices	SEO	PWA	Report
/	🔴 49	🟢 98	🟢 100	🟢 100	🟠 89	Report
/docs/installation	🟠 86	🟢 98	🟢 100	🟢 100	🟠 89	Report
/docs/category/getting-started	🟠 75	🟢 100	🟢 100	🟢 90	🟠 89	Report
/blog	🟠 72	🟢 100	🟢 100	🟢 90	🟠 89	Report
/blog/preparing-your-site-for-docusaurus-v3	🟠 65	🟢 97	🟢 100	🟢 100	🟠 89	Report
/blog/tags/release	🟠 75	🟢 100	🟢 100	🟠 80	🟠 89	Report
/blog/tags	🟠 75	🟢 100	🟢 100	🟢 90	🟠 89	Report

[johnnyreilly]

Not proceeding with this so closing