fix(gatsby-source-wordpress): adjust how endpoint urls are constructed to fix fetching for wordpress.com hosted sites and proxied urls

[lukemorton]

Description

As reported in #10427 some WordPress.com sites return a _links.$.self with a fullpath https://public-api.wordpress.com/wp/v2/sites/$site/users/me and others without https://public-api.wordpress.com/. This means it is not a reliable way of removing the baseUrl from the fullUrl.

Instead we pass the route key in as the fullPath rather than the fullUrl. We then extract the basePath from the baseUrl before finally removing the basePath from the fullPath.

We also need to extract the raw entity type from the fullPath rather than route.

We also need to build the fullUrl from baseUrl and fullPath so I introduced the buildFullUrl function to do just that.

I’ve included tests for both WordPress.com and WordPress.org.

Related Issues

Fixes #10427.

[pieh]

This is pretty wild on wp.com side (maybe it was changed in .org / self-hosted too in recent wp version?). Do you perhaps have wp.com I could use to verify everything works correctly with proposed changes

[lukemorton]

@pieh Thanks for taking a look. I've just DM'd you credentials on Twitter (sorry, couldn't find your email address).

[pieh]

Status update - there are currently some issues with non-wp.com sites with proposed changes (missing /wp-json path fragment in front of endpoint urls), but this whole issue with _links.$.self pointing to https://public-api.wordpress.com/ for every endpoint (actually all links point to same url) does seem like a bug on wordpress.com side of things.

There is support ticket https://en.forums.wordpress.com/topic/rest-api-returning-different-self-links/ created by PR author, but it wasn't responded yet, so I'd like to hold on a bit with taking action on this. For anyone else hitting the issue - in meantime You can use @lukemorton's fork of the plugin https://www.npmjs.com/package/@madetech/gatsby-source-wordpress that contains changes proposed in this PR.

[ghost]

I can confirm this is not a bug on WordPress.com – I thought so too in the beginning.

But after testing with Postman and removing _accessToken from the axios header, all responses are good.

See #11021 for more details.

[zilahir]

it does not seem that @lukemorton's PR working either :(

⠁ source and transform nodesRoute discovered : /wp/v2
error Plugin @madetech/gatsby-source-wordpress returned an error
  ReferenceError: URL is not defined

[zilahir]

i investigated this a bit further, after adding the missing const URL = require('url').URL;, it seems, that the checkRouteList always returning false, so the condition validRoute = inWhiteList && !inBlackList; will never evaluate to true, if the includedRoutes defined as the following (from the documentation:

includedRoutes: [
          "**/*/*/categories",
          "**/*/*/posts",
          "**/*/*/pages",
          "**/*/*/media",
          "**/*/*/tags",
          "**/*/*/taxonomies",
          "**/*/*/users",
          "/wp/v2/*/*"
        ],

[kharrop]

Is this currently being worked on? I ended up resolving the issues by removing instances of _accessToken from the fetch.js file.

[zilahir]

Is this currently being worked on? I ended up resolving the issues by removing instance of _accessToken from the fetch.js file.

what exactly you removed? The issue on my side was resolved aswell, but that was crealy an issue with the includedRoutes

[kharrop]

@zilahir I was referencing #11021 (comment)

[pieh]

Thanks @lukemorton!

[gatsbot]

Holy buckets, @lukemorton — we just merged your PR to Gatsby! 💪💜

Gatsby is built by awesome people like you. Let us say “thanks” in two ways:

1. We’d like to send you some Gatsby swag. As a token of our appreciation, you can go to the Gatsby Swag Store and log in with your GitHub account to get a coupon code good for one free piece of swag. (Currently we’ve got a couple t-shirts available, plus some socks that are really razzing our berries right now.)
2. We just invited you to join the Gatsby organization on GitHub. This will add you to our team of maintainers. Accept the invite by visiting https://github.com/orgs/gatsbyjs/invitation. By joining the team, you’ll be able to label issues, review pull requests, and merge approved pull requests.

If there’s anything we can do to help, please don’t hesitate to reach out to us: tweet at @gatsbyjs and we’ll come a-runnin’.

Thanks again!