[bug]: responses with 404 code should not be cached

[mercs600]

Describe the bug
Hi guys. I see we use apicache connected with hastly to optimize and cache images. We face with the issue caching of 404 responses on media proxy. I mean when something is wrong with the media server and proxy is not able to create image then we get 404, but after when media server is back and images are available, then cache is still valid and we see 404 error page still.

The second issue is about webp support and caching:

When you make a deploy and you open the website on Chrome first (which supports webp) you will be able to see webp images. Then if you open the same page on the safari (which doesn't supports webp) you won't be able to see images because they were cached as webp with the first hit ( on chrome)

To reproduce
Steps to reproduce the behavior:

1. When you use image optimize onboard
2. Open some image which doesn't exist on your media server
3. You should see 404 code response and cache headers
4. Create this missing image on backend server
5. You should see 404 code response

Expected behavior
Images with 404 and error headers should not be cached.

Screenshots

Possible solutions
My solution is simple according to documentation of apicache, but should we done it here or maybe it should be fixed on server side, because this solution is transfering the headers ?

  cacheMiddleware = cache(cacheExpires, hastily.hasSupportedExtension, {
            debug: cacheDebug,
            statusCodes: {
                include: [200, 304]
              },
        });

EDIT

include, exclude option in statusCode brings no results until we changed second parameter of apicache instance this is
hastily.hasSupportedExtension:

(req, res) => hastily.hasSupportedExtension(req) && res.statusCode === 200

Can I change this setup from my project level - webpackConfig or somewhere ? Because now I have to modify packages/pwa-buildpack/lib/Utilities/addImgOptMiddleware.js to make it works.

Please let us know what packages this bug is in regards to:

• venia-concept
• venia-ui
• pwa-buildpack
• peregrine
• pwa-devdocs
• upward-js
• upward-spec
• create-pwa

[awilcoxa]

Created PWA-749 in backlog for grooming

[zetlen]

@mercs600 Thanks for your patience. Maybe the simplest solution here would be to remove the apicache middleware entirely. The image opt middleware is NOT meant to directly handle HTTP requests in production; caching should be handled in another layer, ideally the web server or CDN.

The only reason we'd keep this middleware is if there's a severe performance degradation when you remove it. Can you do me a favor, @mercs600, and remove the apicache middleware entirely, and see what happens to your development and staging server performance?

[larsroettig]

@magento I am working on this