Separate nofollow from noindex tag

[nik-john]

Description of Change(s):

According to Google SEO Specs for the robots meta tag, noindex and nofollow values can be set separately. Currently, settings no-index automatically also sets nofollow as observed #77. A real world example is a website with an internal directory of links that the dev wants google to crawl but not index (no one wants the directory page to show up on the Google Search Results Page). Currently there is no good way of doing this in Next SEO.

This PR separates the two into their own props. As updated in the readme the behavior will be as follows:

noindex	nofollow	meta content of robots, googlebot
--	--	index,follow (default)
false	false	index,follow
true	--	noindex,follow
true	false	noindex,follow
--	true	index,nofollow
false	true	index,nofollow
true	true	noindex,nofollow

One thing that came to mind was that codebases that currently use next-seo and have set noindex to mean noindex,nofollow will have regression issues because when this PR is merged and the dev updates the version on their codebase,

<NextSeo noindex={true} />

will mean

<meta name="robots" content="noindex,follow"> //previously "noindex,nofollow"
<meta name="googlebot" content="noindex,follow"> //previously "noindex,nofollow"

which is necessarily not something that people might want.

Therefore we will need to mark this as a breaking change and require the user to update the above code to

<NextSeo noindex={true} nofollow={true} />

[garmeeh]

👏 @nik-john very well thought out PR, thank you for the contribution.

Just a couple of typos and a suggestion for one more test, but I can add that later if you don't have time.

[garmeeh]

Could you add a quick check to seo.spec.js > SEO overrides apply correctly to ensure this being set correctly?

[nik-john]

@garmeeh I added nofollow to the spec already - do you want me to update it to go index,nofollow or noindex,follow?

[garmeeh]

😅 Oh yeah, it was already set to nofollow my bad.

[nik-john]

@garmeeh I've updated the description blurb as well as the README comments (should've caught the typos myself 🤦‍♂) Do let me know if you want anything else added to the tests

Btw, I really love the level of test coverage in this lib 💯

[garmeeh]

😅 Ah no worries, I would be surprised if there is not a few typos in the docs from myself.

[garmeeh]

Looks good to me 💪

[garmeeh]

@all-contributors please add @nik-john docs, code, test

[allcontributors]

@garmeeh

I've put up a pull request to add @nik-john! 🎉

[garmeeh]

@nik-john 🎉 next-seo@v3.0.0 has been released.