feat(gatsby-source-filesystem): add an environment variable to contro…

…l concurrent queue size (#13110)
gatsbyjs · May 31, 2019 · 90aa247 · 90aa247
1 parent 159d1d5
commit 90aa247
Show file tree

Hide file tree

Showing 2 changed files with 3 additions and 1 deletion.
diff --git a/packages/gatsby-source-filesystem/README.md b/packages/gatsby-source-filesystem/README.md
@@ -62,6 +62,8 @@ They will be added to the following default list:
 ../**/dist/**
 ```
 
+To prevent concurrent requests overload of `processRemoteNode`, you can adjust the `200` default concurrent downloads, with `GATSBY_CONCURRENT_DOWNLOAD` environment variable.
+
 ## How to query
 
 You can query file nodes like the following:

diff --git a/packages/gatsby-source-filesystem/src/create-remote-file-node.js b/packages/gatsby-source-filesystem/src/create-remote-file-node.js
@@ -81,7 +81,7 @@ const createFilePath = (directory, filename, ext) =>
 const queue = new Queue(pushToQueue, {
   id: `url`,
   merge: (old, _, cb) => cb(old),
-  concurrent: 200,
+  concurrent: process.env.GATSBY_CONCURRENT_DOWNLOAD || 200,
 })
 
 /**