feat(gatsby-source-drupal): Provide proxyUrl in addition to baseUrl to allow using CDN, API gateway, etc.

[Vacilando]

Description

Provide param proxyUrl to be used to fetch the Drupal JSON API resources. It can be a CDN or API Gateway URL.

In my particular situation - fetch from a Drupal server (32 GB RAM, 8 vCPUs) vs fetch from a CDN (CloudFront) improves the "Fetch all data from Drupal" build phase by a factor of 10. (17s instead of 170s for almost 100k Drupal nodes)

Documentation

Documentation is provided in README in section "CDN".

Related Issues

Fixes / addresses:

• [gatsby-source-drupal] file download not working if using baseUrl of a proxy with path #36774
• [gatsby-source-drupal] provide proxyUrl in addition to baseUrl to allow using CDN, API gateway, etc. #36811

[Vacilando]

There are 7 failed tests. They seem unrelated to the pull request, am I right?
Is there anything else I can do to make this merge-worthy? :-)

[KyleAMathews]

Makes sense and looks good — can you add a test or two for this before we merge it?

[KyleAMathews]

on the failed tests — yeah we just switched yesterday master to point to the Gatsby v5 beta code and tests are completely passing yet for v5. The only ones this PR needs to pass are the unit tests (where the gatsby-source-drupal tests are run).

[Vacilando]

@KyleAMathews thanks for the approving words and the READMe suggestions - already committed.

I've never written tests for Gatsby, not sure where to begin or end. Will start looking around but any suggestion of a minimal test you would like to see / any pointers how to go about it will be much appreciated!

[KyleAMathews]

The tests are here: https://github.com/gatsbyjs/gatsby/blob/master/packages/gatsby-source-drupal/src/__tests__/index.js

You can run them locally by navigating to the gatsby-source-drupal directory and running npx jest . --watch.

Probably the easiest way to test this is to copy an existing test but with a proxyUrl in the config and then mock the url replacement (you'll need to make it a standalone function to do this I believe) so you can then check that it was called with the variables you expect and returns the right result. Hope that helps!

[Vacilando]

@KyleAMathews - thanks for the testing tips. I tried several times from scratch setting up the testing environment according to https://www.gatsbyjs.com/docs/how-to/testing/unit-testing/ but no luck. So did not even get over to writing the tests.

I get errors like

and cannot run any tests

Perhaps the instructions are outdated, or it has something to do with Gatsby v5 beta, not sure.

Can someone help here with writing a test, or can we merge this without more tests for now. If not I am afraid I have to give it up for some period :-(

NB I run the modified plugin locally and also in Gatsby Cloud, no issues found, the speedup over direct pull from a powerful Drupal server is 10x, so it's definitely a useful feature.

[Vacilando]

Hi again, I need help with the tests / merging this little feature.

I am writing a blog article about this performance improvement for Drupal backends and would very much like to mention the version of gatsby-source-drupal where this will have been implemented.

[TylerBarnes]

@KyleAMathews @Vacilando I took a crack at adding a test. lmk if that looks right