Merge branch 'master' into plugin-twitter/add-pluginOptionsSchema-val…

…idation

.circleci/config.yml

@@ -375,6 +375,17 @@ jobs:
           test_path: e2e-tests/gatsby-static-image
           test_command: yarn test
 
+  e2e_tests_visual-regression:
+    <<: *e2e-executor
+    steps:
+      - e2e-test:
+          test_path: e2e-tests/visual-regression
+          test_command: yarn test
+      - store_artifacts:
+          path: e2e-tests/visual-regression/__diff_output__
+      - store_test_results:
+          path: e2e-tests/visual-regression/cypress/results
+
   starters_validate:
     executor: node
     steps:
@@ -582,6 +593,8 @@ workflows:
           <<: *e2e-test-workflow
       - e2e_tests_gatsby-static-image:
           <<: *e2e-test-workflow
+      - e2e_tests_visual-regression:
+          <<: *e2e-test-workflow
       - e2e_tests_development_runtime:
           <<: *e2e-test-workflow
       - e2e_tests_production_runtime:

.jestSetup.js

@@ -1,2 +1 @@
 process.env.GATSBY_RECIPES_NO_COLOR = "true"
-process.env.GATSBY_EXPERIMENTAL_PLUGIN_OPTION_VALIDATION = "true"

README.md

@@ -169,4 +169,4 @@ Licensed under the [MIT License](./LICENSE).
 
 ## 💜 Thanks
 
-Thanks to our many contributors and to [Netlify](https://www.netlify.com/) for hosting [GatsbyJs](https://www.gatsbyjs.com) and our example sites.
+Thanks to our many contributors and to [Netlify](https://www.netlify.com/) for hosting [Gatsby](https://www.gatsbyjs.com) and our example sites.

docs/docs/add-custom-webpack-config.md

@@ -2,7 +2,7 @@
 title: "Adding a Custom webpack Config"
 ---
 
-_Before creating custom webpack configuration, check to see if there's a Gatsby plugin already built that handles your use case in the [plugins section](/docs/plugins/). If there's not yet one and your use case is a general one, it is highly encouraged you to contribute back your plugin to the Gatsby Plugin Library so it's available to others (including your future self)._
+_Before creating custom webpack configuration, check to see if there's a Gatsby plugin already built that handles your use case in the [plugins section](/docs/plugins/). If there's not yet one and your use case is a general one, consider contributing your plugin to the Gatsby Plugin Library so it's available to others (including your future self)._
 
 To add custom webpack configurations, create (if there's not one already) a `gatsby-node.js` file in your root directory. Inside this file, export a function called `onCreateWebpackConfig`.
 

docs/docs/adding-a-shopping-cart-with-snipcart.md

@@ -190,5 +190,5 @@ The following quote is from the Snipcart [payment gateway page](https://app.snip
 - [OneShopper Gatsby starter](/starters/rohitguptab/OneShopper/)
 - Reference guide on [sourcing from Etsy](/docs/sourcing-from-etsy/)
 - Reference guide on [processing payments with Stripe](/docs/processing-payments-with-stripe/)
-- From the Snipcart blog: [E-Commerce for React Developers \[w/ Gatsby Tutorial\]](https://snipcart.com/blog/react-ecommerce-gatsby-tutorial)
+- From the Snipcart blog: [Gatsby E-Commerce Recipe: Integrate a Cart in a Few Steps](https://snipcart.com/blog/gatsby-recipes-ecommerce)
 - [Snipcart documentation](https://docs.snipcart.com/v3/setup/installation)

docs/docs/adding-analytics.md

@@ -26,24 +26,28 @@ You should now have a Tracking ID; take note of it, as your website will need to
 
 You can find this tracking ID later by going to `Admin > Tracking Info > Tracking Code`.
 
-## Using `gatsby-plugin-gtag`
+## Using `gatsby-plugin-google-gtag`
 
 Now, it's time to configure Gatsby to send page views to your Google Analytics account.
 
-We are going to use `gatsby-plugin-gtag`. For other analytics options (including Google Tag Manager), check [other Gatsby analytics plugins](#other-gatsby-analytics-plugins).
+We are going to use `gatsby-plugin-google-gtag`. For other analytics options (including Google Tag Manager), check [other Gatsby analytics plugins](#other-gatsby-analytics-plugins).
 
 ```shell
-npm install gatsby-plugin-gtag
+npm install gatsby-plugin-google-gtag
 ```
 
 ```js:title=gatsby-config.js
 module.exports = {
   plugins: [
     {
-      resolve: `gatsby-plugin-gtag`,
+      resolve: `gatsby-plugin-google-gtag`,
       options: {
-        // replace `UA-XXXXXXXXX-X` with your own Google Analytics Tracking ID
-        trackingId: `UA-XXXXXXXX-X`,
+        // You can add multiple tracking ids and a pageview event will be fired for all of them.
+        trackingIds: [
+          "GA-TRACKING_ID", // Google Analytics / GA
+          "AW-CONVERSION_ID", // Google Ads / Adwords / AW
+          "DC-FLOODIGHT_ID", // Marketing Platform advertising products (Display & Video 360, Search Ads 360, and Campaign Manager)
+        ],
       },
     },
   ],
@@ -52,7 +56,7 @@ module.exports = {
 
 > Note: Read more about [gatsby-config.js](/docs/gatsby-config/)
 
-Full documentation for the plugin can be found [here](/packages/gatsby-plugin-gtag/).
+Full documentation for the plugin can be found [here](/packages/gatsby-plugin-google-gtag/).
 
 There are a number of extra configuration options--both with the Gatsby plugin and also in your Google Analytics account--so you can tailor things to meet your website's needs.
 

docs/docs/configuring-usage-with-plugin-options.md

@@ -54,24 +54,230 @@ The following table lists possible options values and an example plugin that mak
 
 **Note**: Themes (which are a type of plugin) are able to receive options from a site's `gatsby-config` to be used in its `gatsby-config` in order to allow themes to be composed together. This is done by exporting the `gatsby-config` as a function instead of an object. You can see an example of this in the [`gatsby-theme-blog`](https://github.com/gatsbyjs/themes/tree/master/packages/gatsby-theme-blog) and [`gatsby-theme-blog-core`](https://github.com/gatsbyjs/themes/tree/master/packages/gatsby-theme-blog-core) repositories. Plugins are not capable of this functionality.
 
-## How to validate options
+## How to validate plugin options
 
-In order to make configuration easier for users, plugins can optionally define a [Joi](https://joi.dev) schema to enforce data types for each option using the [`pluginOptionsSchema` API](/docs/node-apis/#pluginOptionsSchema) in `gatsby-node.js`. When users of the plugin pass in configuration options, Gatsby will validate that the options match the schema.
+To help users [configure plugins](/docs/configuring-usage-with-plugin-options/) correctly, a plugin can optionally define a schema to enforce a type for each option. Gatsby will validate that the options users pass match the schema to help them correctly set up their site.
 
-For example, here is what the schema for `gatsby-plugin-console-log` looks like:
+### How to define an options schema
+
+You should use the [`pluginOptionsSchema`](/docs/node-apis/#pluginOptionsSchema) API to define your plugins' options schema. It gets passed an instance of [Joi](https://joi.dev), which you use to return a [`Joi.object`](https://joi.dev/api/?v=17.3.0#object) schema for the options you expect users to pass.
+
+For example, imagine you were creating a plugin called `gatsby-plugin-console-log`. You decide you want users to configure your plugin using the following options:
+
+```javascript:title=gatsby-config.js
+module.exports = {
+  plugins: [
+    {
+      resolve: `gatsby-plugin-console-log`,
+      options: {
+        optionA: true,
+        message: "Hello world"
+        optionB: false, // Optional.
+      },
+    },
+  ],
+}
+```
+
+You want users to pass in a boolean to `optionA` and a string to `message`, and they can optionally pass a boolean to `optionB`. To enforce these rules, you would create the following `pluginOptionsSchema`:
 
 ```javascript:title=plugins/gatsby-plugin-console-log/gatsby-node.js
 exports.pluginOptionsSchema = ({ Joi }) => {
   return Joi.object({
-    optionA: Joi.boolean().required(),
-    message: Joi.string().required().default(`default message`),
-    optionB: Joi.boolean(),
+    optionA: Joi.boolean().required().description(`Enables optionA.`),
+    message: Joi.string()
+      .required()
+      .description(`The message logged to the console.`),
+    optionB: Joi.boolean().description(`Enables optionB.`),
   })
 }
 ```
 
-This ensures users pass a boolean to `optionA` and `optionB`, and a string to `message`. If they pass options that do not match the schema, the validation will show an error when they run `gatsby develop`. It also defaults the `message` option to "default message" in case users do not pass their own.
+If users pass options that do not match the schema, the validation will show an error when they run `gatsby develop` and prompt them to fix their configuration.
+
+For example, if an integer is passed into `message` (which is marked as a required string) this message would be shown:
+
+```
+ ERROR #11331  PLUGIN
+
+Invalid plugin options for "gatsby-plugin-console-log":
+
+- "message" must be a string
+```
+
+### Best practices for option schemas
+
+The [Joi API documentation](https://joi.dev/api/) is a great reference to use while working on a `pluginOptionsSchema`, as it shows all the available types and methods.
+
+Here are some specific Joi best practices for `pluginOptionsSchema`:
+
+- [Add descriptions](#add-descriptions)
+- [Set default options](#set-default-options)
+- [Validate external access](#validate-external-access) where necessary
+- [Add custom error messages](#add-custom-error-messages) where useful
+- [Deprecate options](#deprecating-options) in a major version release rather than removing them
+
+#### Add descriptions
+
+Make sure that every option and field has a [`.description()`](https://joi.dev/api/?v=17.3.0#anydescriptiondesc) explaining its purpose. This is helpful for documentation as users can look at the schema and understand all the options. There might also be tooling in the future that auto-generates plugin option documentation from the schema.
+
+#### Set default options
+
+You can use the [`.default()` method](https://joi.dev/api/?v=17.3.0#anydefaultvalue) to set a default value for an option. For example, in the `gatsby-plugin-console-log` plugin above, you could have the `message` option default to `"default message"` if a user does not pass their own `message` value:
+
+```javascript:title=plugins/gatsby-plugin-console-log/gatsby-node.js
+exports.pluginOptionsSchema = ({ Joi }) => {
+  return Joi.object({
+    optionA: Joi.boolean().required().description(`Enables optionA.`),
+    message: Joi.string()
+      .default(`default message`) // highlight-line
+      .description(`The message logged to the console.`),
+    optionB: Joi.boolean().description(`Enables optionB.`),
+  })
+}
+```
+
+Accessing `pluginOptions.message` would then log `"default message"` in all plugin APIs if the user does not supply their own value.
+
+#### Validate external access
+
+Some plugins (particularly source plugins) query external APIs. With the [`.external()`](https://joi.dev/api/?v=17.3.0#anyexternalmethod-description) method, you can asynchronously validate that the user has access to the API, providing a better experience if they pass invalid secrets.
+
+For example, this is how the [Contentful source plugin](/plugins/gatsby-source-contentful/) might validate that the user has access to the space they are trying to query:
+
+```javascript:title=gatsby-source-contentful/gatsby-node.js
+exports.pluginOptionsSchema = ({ Joi }) => {
+  return Joi.object({
+    accessToken: Joi.string().required(),
+    spaceId: Joi.string().required(),
+    // ...more options here...
+  }).external(async pluginOptions => {
+    try {
+      await contentful
+        .createClient({
+          space: pluginOptions.spaceId,
+          accessToken: pluginOptions.accessToken,
+        })
+        .getSpace()
+    } catch (err) {
+      throw new Error(
+        `Cannot access Contentful space "${pluginOptions.spaceId}" with the provided access token. Double check they are correct and try again!`
+      )
+    }
+  })
+}
+```
+
+#### Add custom error messages
+
+Sometimes you might want to provide more detailed error messages when validation fails for a specific field. Joi provides a [`.messages()` method](https://joi.dev/api/?v=17.3.0#anymessagesmessages) which lets you override error messages for specific [error types](https://joi.dev/api/?v=17.3.0#list-of-errors) (e.g. `"any.required"` when a `.required()` call fails).
+
+For example, in the `gatsby-plugin-console-log` plugin above, this is how you would provide a custom error message if users do not specify `optionA`:
+
+```javascript:title=plugins/gatsby-plugin-console-log/gatsby-node.js
+exports.pluginOptionsSchema = ({ Joi }) => {
+  return Joi.object({
+    optionA: Joi.boolean()
+      .required()
+      .description(`Enables optionA.`)
+      // highlight-start
+      .messages({
+        // Override the error message if the .required() call fails
+        "any.required": `"optionA" needs to be specified to true or false. Get the correct value from your dashboard settings.`,
+      }),
+    // highlight-end
+    message: Joi.string()
+      .default(`default message`)
+      .description(`The message logged to the console.`),
+    optionB: Joi.boolean().description(`Enables optionB.`),
+  })
+}
+```
+
+#### Deprecating options
+
+While you can simply remove options from the schema in major versions, that causes cryptic error messages for users upgrading with existing configuration. Instead, deprecate them using the [`.forbidden()` method](https://joi.dev/api/?v=17.3.0#anyforbidden) in a major version release. Then, [add a custom error message](#add-custom-error-messages) explaining how users should upgrade the functionality using `.messages()`.
+
+For example:
+
+```javascript:title=plugins/gatsby-plugin-console-log/gatsby-node.js
+  return Joi.object({
+    optionA: Joi.boolean()
+      .required()
+      .description(`Enables optionA.`)
+      // highlight-start
+      .forbidden()
+      .messages({
+        // Override the error message if the .forbidden() call fails
+        "any.unknown": `"optionA" is no longer supported. Use "optionB" instead by setting it to the same value you had before on "optionA".`,
+      }),
+      // highlight-end
+    message: Joi.string()
+      .default(`default message`)
+      .description(`The message logged to the console.`),
+    optionB: Joi.boolean().description(`Enables optionB.`),
+  })
+}
+```
+
+### Unit testing an options schema
+
+To verify that a `pluginOptionsSchema` behaves as expected, unit test it with different configurations using the [`gatsby-plugin-utils` package](https://github.com/gatsbyjs/gatsby/tree/master/packages/gatsby-plugin-utils#testpluginoptionsschema).
+
+1. Add the `gatsby-plugin-utils` package to your site:
+
+   ```shell
+   npm install --dev gatsby-plugin-utils
+   ```
+
+2. Use the `testPluginOptionsSchema` function exported from the package in your test file. It takes two parameters, the plugin's actual Joi schema and an example options object to test. It returns an object with an `isValid` boolean, which will be true or false based on whether or not the options object fits the actual Joi schema, and an `errors` array, which will contain the error messages if the validation failed.
+
+For example, with [Jest](https://jestjs.io), your tests might look something like this:
+
+```javascript:title=plugins/gatsby-plugin-console/__tests__/gatsby-node.js
+// This is an example using Jest (https://jestjs.io/)
+import { testPluginOptionsSchema } from "gatsby-plugin-utils"
+import { pluginOptionsSchema } from "../gatsby-node"
+
+describe(`pluginOptionsSchema`, () => {
+  it(`should invalidate incorrect options`, () => {
+    const options = {
+      optionA: undefined, // Should be a boolean
+      message: 123, // Should be a string
+      optionB: `not a boolean`, // Should be a boolean
+    }
+    const { isValid, errors } = testPluginOptionsSchema(
+      pluginOptionsSchema,
+      options
+    )
+
+    expect(isValid).toBe(false)
+    expect(errors).toEqual([
+      `"optionA" is required`,
+      `"message" must be a string`,
+      `"optionB" must be a boolean`,
+    ])
+  })
+
+  it(`should validate correct options`, () => {
+    const options = {
+      optionA: false,
+      message: "string",
+      optionB: true,
+    }
+    const { isValid, errors } = testPluginOptionsSchema(
+      pluginOptionsSchema,
+      options
+    )
+
+    expect(isValid).toBe(true)
+    expect(errors).toEqual([])
+  })
+})
+```
 
 ## Additional resources
 
 - [Example Gatsby site using plugin options with a local plugin](https://github.com/gatsbyjs/gatsby/tree/master/examples/using-plugin-options)
+- [Joi API documentation](https://joi.dev/api/)
+- [`pluginOptionsSchema` for the Contentful source plugin](https://github.com/gatsbyjs/gatsby/blob/af973d4647dc14c85555a2ad8f1aff08028ee3b7/packages/gatsby-source-contentful/src/gatsby-node.js#L75-L159)

docs/docs/graphql-concepts.md

@@ -158,7 +158,7 @@ Gatsby will create a schema that looks something like this:
 title: String
 ```
 
-This makes it easy to pull data from anywhere and immediately start writing
+This makes it possible to pull data from anywhere and immediately start writing
 GraphQL queries against your data.
 
 This _can_ cause confusion as some data sources allow you to define

docs/docs/graphql-reference.md

@@ -85,7 +85,7 @@ In this query `filter` and the `ne` (not equals) operator is used to show only r
 
 Gatsby relies on [Sift](https://www.npmjs.com/package/sift) to enable MongoDB-like query syntax for object filtering. This allows Gatsby to support operators like `eq`, `ne`, `in`, `regex` and querying nested fields through the `__` connector.
 
-It is also possible to filter on multiple fields - just separate the individual filters by a comma (works as an AND):
+It is also possible to filter on multiple fields by separating the individual filters by a comma (works as an AND):
 
 ```js
 filter: { contentType: { in: ["post", "page"] }, draft: { eq: false } }

docs/docs/how-code-splitting-works.md

@@ -167,7 +167,7 @@ If the asset is CSS, you [inject it inline in the head](https://github.com/gatsb
 
 #### Prefetching chunks
 
-As shown above, Gatsby uses "preload" to speed up loading of resources required by the page. These are its CSS and its core JavaScript needed to run the page. But if you stopped there, then when a user clicked a link to another page, he would have to wait for that pages resources to download before showing it. To speed this up, once the current page has loaded, Gatsby looks for all links on the page, and for each starts prefetching the page that the link points to.
+As shown above, Gatsby uses "preload" to speed up loading of resources required by the page. These are its CSS and its core JavaScript needed to run the page. But if you stopped there, then when a user clicked a link to another page, they would have to wait for that pages resources to download before showing it. To speed this up, once the current page has loaded, Gatsby looks for all links on the page, and for each starts prefetching the page that the link points to.
 
 It does this using the `<link rel="prefetch" href="..." />` parameter. When the browser sees this tag, it will start downloading the resource but at an extremely low priority and only when the resources for the current page have finished loading. Check out the [MDN prefetch docs](https://developer.mozilla.org/en-US/docs/Web/HTTP/Link_prefetching_FAQ) for more.
 

docs/docs/how-gatsby-works-with-github-pages.md

@@ -2,7 +2,7 @@
 title: How Gatsby Works with GitHub Pages
 ---
 
-GitHub Pages is a service offered by GitHub that allows hosting for websites configured straight from the repository. A Gatsby site can be hosted on GitHub Pages with just a few configurations to the codebase and the repository's settings.
+GitHub Pages is a service offered by GitHub that allows hosting for websites configured straight from the repository. A Gatsby site can be hosted on GitHub Pages with a few configurations to the codebase and the repository's settings.
 
 You can publish your site on GitHub Pages several different ways: