Built site fails when deployed to Amazon S3 in 2.9.0 and later

[gjhltn]

Description

Site builds and serves successfully with gatsby build && gatsby serve, but when I copy the built files to an AWS S3 web bucket, the site generates 'resource missing' errors.

(the same AWS bucket has served gatsby sites successfully in the past)

UPDATE: I believe rolling back in same project to

  "dependencies": {
    "gatsby": "2.8.8",
    ...

fixes the issue. 2.9.0 and later all have the problem)

Steps to reproduce

(1)

gatsby new demo
cd demo
gatsby build

(2)
upload files to AWS S3 website-bucket

Expected result

Website should appear as it does from gatsby serve

Actual result

The js does not run. Console shows errors:

According to the 'Network' tab of my browser, all files loaded successfully.

Environment

System:
OS: macOS 10.14.5
CPU: (4) x64 Intel(R) Core(TM) m3-7Y32 CPU @ 1.10GHz
Shell: 3.2.57 - /bin/bash
Binaries:
Node: 12.4.0 - ~/.nvm/versions/node/v12.4.0/bin/node
Yarn: 1.16.0 - /usr/local/bin/yarn
npm: 6.9.0 - ~/.nvm/versions/node/v12.4.0/bin/npm
Languages:
Python: 2.7.10 - /usr/bin/python
Browsers:
Chrome: 75.0.3770.100
npmPackages:
gatsby: ^2.10.4 => 2.10.4
gatsby-image: ^2.2.3 => 2.2.3
gatsby-plugin-manifest: ^2.2.0 => 2.2.0
gatsby-plugin-offline: ^2.2.0 => 2.2.0
gatsby-plugin-react-helmet: ^3.1.0 => 3.1.0
gatsby-plugin-sharp: ^2.2.1 => 2.2.1
gatsby-source-filesystem: ^2.1.0 => 2.1.0
gatsby-transformer-sharp: ^2.2.0 => 2.2.0
npmGlobalPackages:
gatsby-cli: 2.7.2

[gjhltn]

Still fails in 2.11.0

[gjhltn]

Still fails in 2.11.5, but with a different error

(There IS a 404 page)

[KyleAMathews]

Can you investigate more what's happening? gatsbyjs.org is working fine e.g. https://www.gatsbyjs.org/rhhdjd

[gjhltn]

One thing that I do notice is that this error is in the same function as the previous one, loadPage... which might mean it's still the same error after all?

Also here's a difference between the local site and S3: one of the json files from s3 is being sent as an octet stream. I wonder if that would be sufficient to break it?

[gjhltn]

update: yes this indeed seems to be the issue.

manually setting the content type on everything in /page-data/*/*.json to application/json fixes it. it's quite a drag to work through all the json files but it works i think.

i think the issue is this code

gatsby/packages/gatsby/cache-dir/loader.js

Line 38 in 68e15e7

const contentType = req.getResponseHeader(`content-type`)

gatsby/packages/gatsby/cache-dir/loader.js

Line 39 in 68e15e7

const isJson = contentType && contentType.startsWith(`application/json`)

...which is not unreasonable, it just didnt break before if by chance the server wasnt setting the mime type correctly

[hussainakram]

@KyleAMathews I'm having the exact same issue on production and it seems on Amazon S3/CloudFront its receiving my page-data.json with content-type: text/plain whereas locally its xhr:

---

and on console I'm getting:

loader.js:172 Uncaught (in promise) Error: 404 page could not be found.

Any idea what might be the cause?

[silviopaganini]

also having this problem, anyone found a fix?

[gjhltn]

@silviopaganini the fix is in the thread. change the MIME type of each page-data.json to text/json (or if your s3 client lets you, set it during upload). it's annoying im afraid, if you have a large site. good luck

[gjhltn]

@KyleAMathews could this be problem be avoided by changing

const isJson = contentType && contentType.startsWith(application/json) to just check the file extension instead?

[silviopaganini]

@gswirrl that's happening locally when you build and serve the files with gatsby serve

[silviopaganini]

OMG

[colbyfayock]

development environment fixes aside, is Gatsby's position that this is an environment issue or will there be thought putting into a gatsby solution? i think this kind of problem creates a higher barrier of entry for newcomers

[KyleAMathews]

is Gatsby's position that this is an environment issue or will there be thought putting into a gatsby solution

The expectation is currently that people’s servers would set the mime type to application/json but yeah — that’s probably too strict given how many variants there is out there (see e.g. https://stackoverflow.com/questions/477816/what-is-the-correct-json-content-type)

Checking the extension in addition to the mime type seems reasonable — would someone like to PR that in?

[wardpeet]

I just created a PR that doesn't look at content type but tries to parse the json. This should fix your issues.

[gjhltn]

@wardpeet woah. thats a much better idea than mine. thank you. 🙏

[pieh]

Hey, we just published gatsby@2.13.6 that should fix this. Please give it a go and let us know if it solved the issue

[gjhltn]

@pieh confirmed fixed for me! 🎉 thank you! 👍

[Rajarml]

Our PWA is hosted in Azure websites and we can't upgrade Gatsby higher than 2.9.2 (since the page-data.json change). The issue occurs only after an app deployment (locally everything is fine).
Testing the version 2.13.12 fix the problem :)

[wardpeet]

Seems like we fixed this bug 🎉 let's close 😄

[gatsbot]

Hiya!

This issue has gone quiet. Spooky quiet. 👻

We get a lot of issues, so we currently close issues after 30 days of inactivity. It’s been at least 20 days since the last update here.

If we missed this issue or if you want to keep it open, please reply here. You can also add the label "not stale" to keep this issue open!

As a friendly reminder: the best way to see this issue, or any other, fixed is to open a Pull Request. Check out gatsby.dev/contributefor more information about opening PRs, triaging issues, and contributing!

Thanks for being a part of the Gatsby community! 💪💜