diff --git a/README.md b/README.md index 7493ecd2..6ae4f2cf 100644 --- a/README.md +++ b/README.md @@ -1,17 +1,45 @@ + +
+ +[![gatsby-source-airtable](./header.png)](.) + # gatsby-source-airtable -[![npm](https://img.shields.io/npm/v/gatsby-source-airtable/latest.svg?style=flat-square)](https://www.npmjs.com/package/gatsby-source-airtable) -[![Build Status](https://travis-ci.com/jbolda/gatsby-source-airtable.svg?branch=master)](https://travis-ci.com/jbolda/gatsby-source-airtable) +Gatsby source plugin for sourcing data into your Gatsby application from your +Airtable base tables -## Install +[![npm](https://badgen.net/npm/v/gatsby-source-airtable)](https://www.npmjs.com/package/gatsby-source-airtable) +[![travis](https://badgen.net/travis/jbolda/gatsby-source-airtable)](https://travis-ci.com/jbolda/gatsby-source-airtable) + +
+ +--- -via npm +- [Install](#install) +- [Example](#example) +- [How it works](#how-it-works) + - [Deep linking across tables](#deep-linking-across-tables) + - [Using markdown and attachments](#using-markdown-and-attachments) + - [The power of views](#the-power-of-views) + - [Naming conflicts](#naming-conflicts) + - [Column Names](#column-names) + - [API Keys](#api-keys) + - [Columns without any values (yet)](#columns-without-any-values-yet) +- [History](#history) +- [Contributors](#contributors) +- [License](#license) -`npm install --save gatsby-source-airtable` +--- -or via yarn +## Install + +```sh +# using npm +npm install --save gatsby-source-airtable -`yarn add gatsby-source-airtable` +# using yarn +yarn add gatsby-source-airtable +``` ## Example @@ -48,11 +76,14 @@ plugins: [ ]; ``` -Get one single record (table row), where `Field_1 === YOUR_VALUE` +Get one single record (table row), where `Field_1 === YOUR_VALUE`: -``` +```graphql { - airtable(table: {eq: "YOUR_TABLE_NAME"}, data: {Field_1: {eq: "YOUR_VALUE"}}) { + airtable( + table: { eq: "YOUR_TABLE_NAME" } + data: { Field_1: { eq: "YOUR_VALUE" } } + ) { data { Field_1 Field_2 @@ -66,16 +97,20 @@ Get one single record (table row), where `Field_1 === YOUR_VALUE` } ``` -Get all records from `YOUR_TABLE_NAME` where `Field_1 === YOUR_VALUE` +Get all records from `YOUR_TABLE_NAME` where `Field_1 === YOUR_VALUE`: -``` +```graphql { - allAirtable(filter: {table: {eq: "YOUR_TABLE_NAME"}, data: {Field_1: {eq: "YOUR_VALUE"}}}) { + allAirtable( + filter: { + table: { eq: "YOUR_TABLE_NAME" } + data: { Field_1: { eq: "YOUR_VALUE" } } + } + ) { edges { node { data { Field_1 - ... } } } @@ -85,57 +120,123 @@ Get all records from `YOUR_TABLE_NAME` where `Field_1 === YOUR_VALUE` ## How it works -When running `gatsby develop` or `gatsby build`, this plugin will fetch all data for all rows in each of the tables you specify, making them available for query throughout your gatsby.js app, and to other Gatsby plugins as well. +When running `gatsby develop` or `gatsby build`, this plugin will fetch all data +for all rows in each of the tables you specify, making them available for query +throughout your Gatsby app, and to other Gatsby plugins as well. -As seen in the example above, `tables` is always specified as an array of table objects. These tables may be sourced from different bases. +As seen in the example above, `tables` is always specified as an array of table +objects. These tables may be sourced from different bases. -Querying for `airtable` will always only return one record (defaulting to the first record in the table), and querying for `allAirtable` will return any records that match your query parameters. +Querying for `airtable` will always only return one record (defaulting to the +first record in the table), and querying for `allAirtable` will return any +records that match your query parameters. -As in the examples above, you can narrow your query by filtering for table names, and field values. +As in the examples above, you can narrow your query by filtering for table +names, and field values. ### Deep linking across tables -One powerful feature of Airtable is the ability to specify fields which link to records in other tables-- the `Link to a Record` field type. If you wish to query data from a linked record, you must specify the field name in `tableLinks` (matching the name shown in Airtable, not the escaped version). +One powerful feature of Airtable is the ability to specify fields which link to +records in other tables-- the `Link to a Record` field type. If you wish to +query data from a linked record, you must specify the field name in `tableLinks` +(matching the name shown in Airtable, not the escaped version). -This will create nested nodes accessible in your graphQL queries, as shown in the above example. If you do not specify a linked field in `tableLinks`, you will just receive the linked record's Airtable IDs as `strings`. The name of the column/field does not have to match the related table, but you do need to make sure that the related table is included as an object in your `gatsby-config.js` as well. +This will create nested nodes accessible in your GraphQL queries, as shown in +the above example. If you do not specify a linked field in `tableLinks`, you +will just receive the linked record's Airtable IDs as `strings`. The name of the +column/field does not have to match the related table, but you do need to make +sure that the related table is included as an object in your `gatsby-config.js` +as well. ### Using markdown and attachments -Optionally, you may provide a "mapping". This will alert the plugin that column names you specify are of a specific, non-string format of your choosing. This is particularly useful if you would like to have Gatsby pick up the fields for transforming, e.g. `text/markdown`. If you do not provide a mapping, Gatsby will just "infer" what type of value it is, which is most typically a `string`. - -For an example of a markdown-and-airtable-driven site using `gatsby-transformer-remark`, see the examples folder in this repo. - -If you are using the `Attachment` type field in Airtable, you may specify a column name with `fileNode` and the plugin will bring in these files. Using this method, it will create "nodes" for each of the files and expose this to all of the transformer plugins. A good use case for this would be attaching images in Airtable, and being able to make these available for use with the `sharp` plugins and `gatsby-image`. Specifying a `fileNode` does require a peer dependency of `gatsby-source-filesystem` otherwise it will fall back as a non-mapped field. The locally available files and any ecosystem connections will be available on the node as `localFiles`. - -If you are specifying more than one type of `mapping`, you may potentially run into issues with data types clashing and throwing errors. An additional option that you may specify is `separateMapType` which will create a gatsby node type for each type of data. This should prevent issues with your data types clashing. - -When using the Attachment type field, this plugin governs requests to download the associated files from Airtable to 5 concurrent requests to prevent excessive requests on Airtable's servers - which can result in refused / hanging connections. You can adjust this limit with the concurrency option in your gatsby-config.js file. Set the option with an integer value for your desired limit on attempted concurrent requests. A value of 0 will allow requests to be made without any limit. +Optionally, you may provide a "mapping". This will alert the plugin that column +names you specify are of a specific, non-string format of your choosing. This is +particularly useful if you would like to have Gatsby pick up the fields for +transforming, e.g. `text/markdown`. If you do not provide a mapping, Gatsby will +just "infer" what type of value it is, which is most typically a `string`. + +For an example of a markdown-and-airtable-driven site using +`gatsby-transformer-remark`, see the examples folder in this repo. + +If you are using the `Attachment` type field in Airtable, you may specify a +column name with `fileNode` and the plugin will bring in these files. Using this +method, it will create "nodes" for each of the files and expose this to all of +the transformer plugins. A good use case for this would be attaching images in +Airtable, and being able to make these available for use with the `sharp` +plugins and `gatsby-image`. Specifying a `fileNode` does require a peer +dependency of `gatsby-source-filesystem` otherwise it will fall back as a +non-mapped field. The locally available files and any ecosystem connections will +be available on the node as `localFiles`. + +If you are specifying more than one type of `mapping`, you may potentially run +into issues with data types clashing and throwing errors. An additional option +that you may specify is `separateMapType` which will create a gatsby node type +for each type of data. This should prevent issues with your data types clashing. + +When using the Attachment type field, this plugin governs requests to download +the associated files from Airtable to 5 concurrent requests to prevent excessive +requests on Airtable's servers - which can result in refused/hanging +connections. You can adjust this limit with the concurrency option in your +`gatsby-config.js` file. Set the option with an integer value for your desired +limit on attempted concurrent requests. A value of 0 will allow requests to be +made without any limit. ### The power of views -Within Airtable, every table can have one or more named Views. These Views are a convenient way to pre-filter and sort your data before querying it in Gatsby. If you do not specify a view in your table object, raw data will be returned in no particular order. +Within Airtable, every table can have one or more named Views. These Views are a +convenient way to pre-filter and sort your data before querying it in Gatsby. If +you do not specify a view in your table object, raw data will be returned in no +particular order. -For example, if you are creating a blog or documentation site, specify a `published` field in Airtable, create a filter showing only published posts, and specify this as the (optional) `tableView` option in `gatsby-config.js` +For example, if you are creating a blog or documentation site, specify a +`published` field in Airtable, create a filter showing only published posts, and +specify this as the (optional) `tableView` option in `gatsby-config.js` ### Naming conflicts -You may have a situation where you are including two separate bases, each with a table that has the exact same name. With the data structure of this repo, both bases would fall into allAirtable and you wouldn't be able to tell them apart when building graphQL queries. This is what the optional `queryName` setting is for-- simply to provide an alternate name for a table. +You may have a situation where you are including two separate bases, each with a +table that has the exact same name. With the data structure of this repo, both +bases would fall into allAirtable and you wouldn't be able to tell them apart +when building GraphQL queries. This is what the optional `queryName` setting is +for-- simply to provide an alternate name for a table. -If you would like to have the query names for tables be different from the default `allAirtable` or `airtable`, you may specify `separateNodeType` as `true`. +If you would like to have the query names for tables be different from the +default `allAirtable` or `airtable`, you may specify `separateNodeType` as +`true`. ### Column Names -Within graphql (the language you query information from and that this plugin puts nodes into), there are character limitations. Most specifically we cannot have spaces in field names. We don't want to force you to change your Airtable names, so we will "clean" the keys and replace the spaces with an underscore (e.g. The Column Name becomes The_Column_Name). We use the cleaned name everywhere including `gatsby-config.js` and within your queries. We don't warn you when this happens to cut down on the verbosity of the output. +Within GraphQL (the language you query information from and that this plugin +puts nodes into), there are character limitations. Most specifically we cannot +have spaces in field names. We don't want to force you to change your Airtable +names, so we will "clean" the keys and replace the spaces with an underscore +(e.g. `The Column Name` becomes `The_Column_Name`). We use the cleaned name +everywhere including `gatsby-config.js` and within your queries. We don't warn +you when this happens to cut down on the verbosity of the output. ### API Keys Keys can be found in Airtable by clicking `Help > API Documentation`. -The API key can be hard coded directly in `gatsby-config.js` as noted in the previous section-- **this exposes your key to anyone viewing your repository and is not recommended. You should inject your API key as recommended below to prevent it from being committed to source control**. - -We recommended specifying your API key using an [Environment Variable](https://www.gatsbyjs.org/docs/environment-variables/). You may also specify it in your command line such as `AIRTABLE_API_KEY=XXXXXX gatsby develop`. Note: If you use an environment variable prepended with `GATSBY_`, it takes advantage of some syntactic sugar courtesy of Gatsby, which automatically makes it available - but any references to environment variables like this that are rendered client-side will **automatically** expose your API key within the browser. To avoid accidentally exposing it, we recommend _not_ prepending it with `GATSBY_`. - -To be safe, you can also setup your API key via a config variable, `apiKey` defined in `gatsby-config.js`. This is the recommended way to inject your API key. +The API key can be hard coded directly in `gatsby-config.js` as noted in the +previous section-- **this exposes your key to anyone viewing your repository and +is not recommended. You should inject your API key as recommended below to +prevent it from being committed to source control**. + +We recommended specifying your API key using an +[Environment Variable](https://www.gatsbyjs.org/docs/environment-variables/). +You may also specify it in your command line such as +`AIRTABLE_API_KEY=XXXXXX gatsby develop`. Note: If you use an environment +variable prepended with `GATSBY_`, it takes advantage of some syntactic sugar +courtesy of Gatsby, which automatically makes it available - but any references +to environment variables like this that are rendered client-side will +**automatically** expose your API key within the browser. To avoid accidentally +exposing it, we recommend _not_ prepending it with `GATSBY_`. + +To be safe, you can also setup your API key via a config variable, `apiKey` +defined in `gatsby-config.js`. This is the recommended way to inject your API +key. ```javascript // In gatsby-config.js @@ -143,22 +244,28 @@ plugins: [ { resolve: `gatsby-source-airtable`, options: { - //not prefaced with "GATSBY_", will not automatically be included client-side unless you explicitly expose it + // not prefaced with "GATSBY_", will not automatically be included client-side unless you explicitly expose it apiKey: process.env.AIRTABLE_API_KEY, - //...etc + // ...etc }, }, ]; ``` -You can either use a node tool like "dotenv" to load secrets like your Airtable API key from a .env file, or you can specify it in your command line such as `AIRTABLE_API_KEY=XXXXXX gatsby develop`. +You can either use a node tool like "dotenv" to load secrets like your Airtable +API key from a .env file, or you can specify it in your command line such as +`AIRTABLE_API_KEY=XXXXXX gatsby develop`. -If you add or change your API key in an environment variable at the system level, you may need to reload your code editor / IDE for that variable to reload. +If you add or change your API key in an environment variable at the system +level, you may need to reload your code editor/IDE for that variable to reload. ### Columns without any values (yet) -If you want to perform conditional logic based on data that may or may not be present in Airtable, but you do not yet have tabular data for the "may" case, you can update the gatsby-source-airtable section of `gatsby-config.js` to include sensible defaults for those fields -so that they will be returned via your graphql calls: +If you want to perform conditional logic based on data that may or may not be +present in Airtable, but you do not yet have tabular data for the "may" case, +you can update the `gatsby-source-airtable` section of `gatsby-config.js` to +include sensible defaults for those fields so that they will be returned via +your GraphQL calls: ```javascript // In gatsby-config.js @@ -172,11 +279,11 @@ plugins: [ baseId: process.env.AIRTABLE_BASE, tableName: process.env.AIRTABLE_TABLE_NAME, defaultValues: { - //currently does not accept null / undefined. use empty string instead - //and perform your conditional logic on name_of_field.length > 0 ? condition_1 : condition_2 + // currently does not accept null / undefined. use empty string instead + // and perform your conditional logic on name_of_field.length > 0 ? condition_1 : condition_2 NAME_OF_FIELD_THAT_WILL_OTHERWISE_NOT_BE_RETURNED_IF_ALL_VALUES_ARE_BLANK: "", - //... etc + // ... etc }, }, ], @@ -187,13 +294,23 @@ plugins: [ ## History -A Gatsby source plugin for pulling rows from multiple tables and bases in Airtable. This was originally inspired by [kevzettler/gatsby-source-airtable](https://github.com/kevzettler/gatsby-source-airtable) and eventually superseeded the original plugin with the introduction of Gatsby v2. +A Gatsby source plugin for pulling rows from multiple tables and bases in +Airtable. This was originally inspired by +[`kevzettler/gatsby-source-airtable`](https://github.com/kevzettler/gatsby-source-airtable) +and eventually superseeded the original plugin with the introduction of Gatsby +v2. -If you are looking for the documentation on `gatsby-source-airtable-linked`, see the additional branch. We do recommend moving your dependency over to this plugin, `gatsby-source-airtable`, for Gatsby v2. (If you are still on Gatsby v1, see `gatsby-source-airtable-linked` for compatible code.) +If you are looking for the documentation on `gatsby-source-airtable-linked`, see +the additional branch. We do recommend moving your dependency over to this +plugin, `gatsby-source-airtable`, for Gatsby v2. (If you are still on Gatsby v1, +see +[`gatsby-source-airtable-linked`](https://github.com/jbolda/gatsby-source-airtable/tree/gatsby-source-airtable-linked) +for compatible code.) -## Contributors ✨ +## Contributors -Thanks goes to these wonderful people ([emoji key](https://allcontributors.org/docs/en/emoji-key)): +Thanks goes to these wonderful people +([emoji key](https://allcontributors.org/docs/en/emoji-key)): @@ -221,4 +338,10 @@ Thanks goes to these wonderful people ([emoji key](https://allcontributors.org/d -This project follows the [all-contributors](https://github.com/all-contributors/all-contributors) specification. Contributions of any kind welcome! +This project follows the +[all-contributors](https://github.com/all-contributors/all-contributors) +specification. Contributions of any kind welcome! + +## License + +[MIT License, Copyright (c) 2018 Jacob Bolda](./LICENSE) diff --git a/gatsby-node.js b/gatsby-node.js index 9a3fe755..ff331fe6 100644 --- a/gatsby-node.js +++ b/gatsby-node.js @@ -1,12 +1,12 @@ const Airtable = require("airtable"); -const crypto = require(`crypto`); -const { createRemoteFileNode } = require(`gatsby-source-filesystem`); +const crypto = require("crypto"); +const { createRemoteFileNode } = require("gatsby-source-filesystem"); const { map } = require("bluebird"); -const mime = require('mime/lite'); +const mime = require("mime/lite"); exports.sourceNodes = async ( { actions, createNodeId, store, cache }, - { apiKey, tables, concurrency } + { apiKey, tables, concurrency }, ) => { // tables contain baseId, tableName, tableView, queryName, mapping, tableLinks const { createNode, setPluginStatus } = actions; @@ -15,7 +15,7 @@ exports.sourceNodes = async ( // hoist api so we can use in scope outside of this block if (!apiKey && process.env.GATSBY_AIRTABLE_API_KEY) { console.warn( - "\nImplicit setting of GATSBY_AIRTABLE_API_KEY as apiKey will be deprecated in future release, apiKey should be set in gatsby-config.js, please see Readme!" + "\nImplicit setting of GATSBY_AIRTABLE_API_KEY as apiKey will be deprecated in future release, apiKey should be set in gatsby-config.js, please see Readme!", ); } var api = await new Airtable({ @@ -32,7 +32,7 @@ exports.sourceNodes = async ( // exit if tables is not defined if (tables === undefined || tables.length === 0) { console.warn( - "\ntables is not defined for gatsby-source-airtable in gatsby-config.js" + "\ntables is not defined for gatsby-source-airtable in gatsby-config.js", ); return; } @@ -112,7 +112,7 @@ exports.sourceNodes = async ( : false, cleanMapping, cleanLinks, - ]) + ]), ); }); @@ -131,7 +131,7 @@ exports.sourceNodes = async ( row.mapping = currentValue[5]; // mapping from tableOptions above row.tableLinks = currentValue[6]; // tableLinks from tableOptions above return row; - }) + }), ); }, []); }) @@ -171,7 +171,7 @@ exports.sourceNodes = async ( if (row.separateNodeType && (!row.queryName || row.queryName === "")) { console.warn( `You have opted into separate node types, but not specified a queryName. - We use the queryName to suffix to node type. Without a queryName, it will act like separateNodeType is false.` + We use the queryName to suffix to node type. Without a queryName, it will act like separateNodeType is false.`, ); } @@ -200,7 +200,7 @@ exports.sourceNodes = async ( nodes.forEach((node) => createNode(node)); }); }, - { concurrency: concurrency } + { concurrency: concurrency }, ); }; @@ -227,7 +227,7 @@ const processData = async (row, { createNodeId, createNode, store, cache }) => { // `data` is direct from Airtable so we don't use // the cleanKey here processedData[useKey] = data[key].map((id) => - createNodeId(`Airtable_${id}`) + createNodeId(`Airtable_${id}`), ); } else if (row.mapping && row.mapping[cleanedKey]) { // A child node comes from the mapping, where we want to @@ -260,7 +260,7 @@ const checkChildNode = async ( key, row, processedData, - { createNodeId, createNode, store, cache } + { createNodeId, createNode, store, cache }, ) => { let data = row.fields; let mapping = row.mapping; @@ -273,7 +273,7 @@ const checkChildNode = async ( }); processedData[`${cleanedKey}___NODE`] = createNodeId( - `AirtableField_${row.id}_${cleanedKey}` + `AirtableField_${row.id}_${cleanedKey}`, ); return buildNode( @@ -282,33 +282,33 @@ const checkChildNode = async ( cleanedKey, data[key], mapping[key], - createNodeId + createNodeId, ); }; const localFileCheck = async ( key, row, - { createNodeId, createNode, store, cache } + { createNodeId, createNode, store, cache }, ) => { let data = row.fields; let mapping = row.mapping; let cleanedKey = cleanKey(key); - if (mapping[cleanedKey] === `fileNode`) { + if (mapping[cleanedKey] === "fileNode") { try { let fileNodes = []; // where data[key] is the array of attachments // `data` is direct from Airtable so we don't use // the cleanKey here data[key].forEach((attachment) => { - const ext = mime.getExtension(attachment.type) // unknown type returns null + const ext = mime.getExtension(attachment.type); // unknown type returns null let attachmentNode = createRemoteFileNode({ url: attachment.url, store, cache, createNode, createNodeId, - ext: !!ext ? `.${ext}` : undefined + ext: !!ext ? `.${ext}` : undefined, }); fileNodes.push(attachmentNode); }); @@ -316,13 +316,13 @@ const localFileCheck = async ( // ___NODE tells Gatsby that this field will link to another nodes const resolvedFileNodes = await Promise.all(fileNodes); const localFiles = resolvedFileNodes.map( - (attachmentNode) => attachmentNode.id + (attachmentNode) => attachmentNode.id, ); return localFiles; } catch (e) { console.log( "You specified a fileNode, but we caught an error. First check that you have gatsby-source-filesystem installed?\n", - e + e, ); } } @@ -332,7 +332,7 @@ const localFileCheck = async ( const buildNode = (localFiles, row, cleanedKey, raw, mapping, createNodeId) => { const nodeType = row.separateNodeType ? `Airtable${cleanKey(row.queryName ? row.queryName : row._table.name)}` - : `Airtable`; + : "Airtable"; if (localFiles) { return { id: createNodeId(`AirtableField_${row.id}_${cleanedKey}`), diff --git a/header.png b/header.png new file mode 100644 index 00000000..b4097ec4 Binary files /dev/null and b/header.png differ