Images missing in production build, based on local presence of ".cache" or "public" folders

[jakubjafra]

Description

I've encountered a weird error where gatsby is not building images in production. This behaviour is based on the local state of .cache & public folders - if one of them is missing, then the resulting build contains no images.

Steps to reproduce

I've created a repository with minimal steps to reproduce: https://github.com/jakubjafra/gatsby-image-build-error
It's a basic gatsby example, with just an image added using gatsby-image.

Basic example:

git clone git@github.com:jakubjafra/gatsby-image-build-error.git && cd gatsby-image-build-error
npm i

# steps:
gatsby develop # run the gatsby in development mode - so we have .cache & public folders
rm -rf ./public && gatsby build # remove one of the folders, and run gatsby build
gatsby serve # go to localhost:9000 - image is not loaded
# image URL like "http://localhost:9000/static/da9be9d13642e6487c6e7c7f3bcd1522/034c8/test.jpg" returns 404 with HTML content inside

Expected result

Images should be present in a production build.

Actual result

Images are not optimized and thus not bundled in the production build. The URLs are built, however, and return 404 HTML page.

Environment

  System:
    OS: macOS 10.15.7
    CPU: (12) x64 Intel(R) Core(TM) i9-8950HK CPU @ 2.90GHz
    Shell: 5.7.1 - /bin/zsh
  Binaries:
    Node: 10.23.0 - ~/.nvm/versions/node/v10.23.0/bin/node
    Yarn: 1.22.10 - /usr/local/bin/yarn
    npm: 6.14.8 - ~/.nvm/versions/node/v10.23.0/bin/npm
  Languages:
    Python: 3.7.3 - /usr/local/anaconda3/bin/python
  Browsers:
    Chrome: 86.0.4240.111
    Firefox: 81.0.1
    Safari: 14.0
  npmPackages:
    gatsby: ^2.24.85 => 2.24.85
    gatsby-image: ^2.4.21 => 2.4.21
    gatsby-plugin-sharp: ^2.6.43 => 2.6.43
    gatsby-source-filesystem: ^2.3.37 => 2.3.37
    gatsby-transformer-sharp: ^2.5.20 => 2.5.20
  npmGlobalPackages:
    gatsby-cli: 2.12.113

(I've also tested this on Node v15.0.1)

[KyleAMathews]

This is a known problem — @vladar has been doing some work to validate the state of these folders — #27549

Can you check if this fixes your reproduction?

[LekoArts]

Or in other words: This is expected behavior. You'll need to keep both folders around when building or when deleting things you'll need to delete both.

[jakubjafra]

Yes, removing both folders solves the issue.

[vladar]

Can you try it on the latest gatsby (2.24.89)? It should delete the .cache folder for you when the public folder is missing.

[jakubjafra]

I've tested 2.24.89 and 2.24.90 and I could reproduce the problem in both.

[vladar]

You are right! This happens because sharp job results are not purged even after we purge the cache directory and redux state. Created a new issue from this discussion: #27699 Let's track it there.