[miniflare] feature: respect incoming Accept-Encoding header

[mrbbot]

What this PR solves / how to test

Fixes #5246 (CUSTESC-39344).

This PR updates Miniflare to respect the incoming Accept-Encoding header, and set the correct value of Accept-Encoding/request.cf.clientAcceptEncoding inside the Worker. This is to match the production/documented behaviour here: https://developers.cloudflare.com/fundamentals/reference/http-request-headers/#accept-encoding.

Author has addressed the following

• Tests
  • TODO (before merge)
  • Included
  • Not necessary because:
• Changeset (Changeset guidelines)
  • TODO (before merge)
  • Included
  • Not necessary because:
• Public documentation
  • TODO (before merge)
  • Cloudflare docs PR(s): https://github.com/cloudflare/cloudflare-docs/pull/...
  • Not necessary because: matching already documented behaviour

[changeset-bot]

🦋 Changeset detected

Latest commit: 18ac0fa

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 4 packages

Name	Type
miniflare	Minor
@cloudflare/pages-shared	Patch
@cloudflare/vitest-pool-workers	Patch
wrangler	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[github-actions]

A wrangler prerelease is available for testing. You can install this latest build in your project with:

npm install --save-dev https://prerelease-registry.devprod.cloudflare.dev/workers-sdk/runs/8629582197/npm-package-wrangler-5409

You can reference the automatically updated head of this PR with:

npm install --save-dev https://prerelease-registry.devprod.cloudflare.dev/workers-sdk/prs/5409/npm-package-wrangler-5409

Or you can use npx with this latest build directly:

npx https://prerelease-registry.devprod.cloudflare.dev/workers-sdk/runs/8629582197/npm-package-wrangler-5409 dev path/to/script.js

Additional artifacts:

npx https://prerelease-registry.devprod.cloudflare.dev/workers-sdk/runs/8629582197/npm-package-create-cloudflare-5409 --no-auto-update

npm install https://prerelease-registry.devprod.cloudflare.dev/workers-sdk/runs/8629582197/npm-package-cloudflare-kv-asset-handler-5409

npm install https://prerelease-registry.devprod.cloudflare.dev/workers-sdk/runs/8629582197/npm-package-miniflare-5409

npm install https://prerelease-registry.devprod.cloudflare.dev/workers-sdk/runs/8629582197/npm-package-cloudflare-pages-shared-5409

npm install https://prerelease-registry.devprod.cloudflare.dev/workers-sdk/runs/8629582197/npm-package-cloudflare-vitest-pool-workers-5409

Note that these links will no longer work once the GitHub Actions artifact expires.

---

wrangler@3.48.0 includes the following runtime dependencies:

Package	Constraint	Resolved
miniflare	workspace:*	3.20240404.0
workerd	1.20240404.0	1.20240404.0
workerd --version	1.20240404.0	2024-04-04

Please ensure constraints are pinned, and miniflare/workerd minor versions match.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 72.28%. Comparing base (2789f26) to head (bcb298e).
Report is 7 commits behind head on main.

❗ Current head bcb298e differs from pull request most recent head 18ac0fa. Consider uploading reports for the commit 18ac0fa to get more accurate results

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #5409      +/-   ##
==========================================
- Coverage   72.44%   72.28%   -0.17%     
==========================================
  Files         331      322       -9     
  Lines       17298    16663     -635     
  Branches     4422     4260     -162     
==========================================
- Hits        12532    12045     -487     
+ Misses       4766     4618     -148     

see 94 files with indirect coverage changes

[frandiox]

@mrbbot Hey 👋 I understand this is a fix but it actually creates a breaking change for those proxying Miniflare in Node.js. It works in miniflare@3.20240405.0 but breaks in miniflare@3.20240405.1. Just mentioning it in case you want to document it in the changelog or somewhere:

Running mf.dispatchFetch(...) (or plain fetch(workerUrl)) is decompressing the response but keeping Content-Encoding: gzip or similar in the response headers. Then, once we forward this response from Node (e.g. Vite server) to the browser, the browser expects encoded content but it is not encoded anymore.

Unfortunately, I think there's no way to avoid decompressing the response in undici. The workaround is to remove Content-Encoding from the Miniflare response or recompressing it in Node -- or perhaps use Node's http.get instead of fetch to get the response... which is not intuitive with mf.dispatchFetch().

Related:

• fetch raw bytes without decompressing (and sending accept-encoding) nodejs/undici#1462
• [fetch] Omit Content-Length and Content-Encoding headers for compressed responses nodejs/undici#2514

cc @hi-ogawa probably relevant for your plugins if you update to miniflare@3.20240405.1