🐛 BUG: Pages header response differences - prod vs local differences

[Cherry]

What version of Wrangler are you using?

2.0.5

What operating system are you using?

Windows 10

Describe the Bug

content-type

In local, if you request a file with an unrecognised mime type via env.ASSETS.fetch or next, you'll get a header of content-type: application/octet-stream.

In production, the header is omitted entirely.

An example of this is a file like image.jxl, since jxl isn't in the mime-db yet.

etag

This header is returned in prod. Not in dev.

Additional headers

In prod, you also get the following:

"access-control-allow-origin": "*",
"cache-control": "public, max-age=0, must-revalidate",
"referrer-policy": "strict-origin-when-cross-origin",
"x-content-type-options": "nosniff"

Whereas in dev, you only get content-type.

Missing headers

In both prod and dev, the content-length header is missing. This would be an incredibly nice addition for my pages plugin pages-plugin-asset-negotiation so I could determine the smallest file without having to buffer it into memory.

[Cherry]

I just retested this now that the issue is closed, and while it's very much improved, it doesn't seem to be completely resolved:

Using wrangler pages dev 2.1.0, I get the following for a cat.jxl file:

access-control-allow-origin: *
cache-control: public, max-age=0, must-revalidate
Connection: keep-alive
content-type: application/octet-stream
Date: Mon, 12 Sep 2022 19:27:45 GMT
etag: "279af46407a45ba0f5379cd802350ca8"
Keep-Alive: timeout=5
referrer-policy: strict-origin-when-cross-origin
Transfer-Encoding: chunked
x-content-type-options: nosniff
x-server-env: dev

In prod, I get:

access-control-allow-origin: *
alt-svc: h3=":443"; ma=86400, h3-29=":443"; ma=86400
cache-control: public, max-age=0, must-revalidate
cf-ray: -snip-
content-length: 9618
content-type: application/octet-stream
date: Mon, 12 Sep 2022 19:32:29 GMT
etag: "279af46407a45ba0f5379cd802350ca8"
nel: {"success_fraction":0,"report_to":"cf-nel","max_age":604800}
referrer-policy: strict-origin-when-cross-origin
report-to: -snip-
server: cloudflare
vary: Accept-Encoding
x-content-type-options: nosniff

This is mostly the same where it matters (though content-type is missing likely because of how new jxl is and not in mime-db yet), but there's a missing content-length in dev. This is also true for other images I tried such as cat.avif - the content-type here was accurate in both, but content-length was only present in prod.

For an HTML file:

Dev:

access-control-allow-origin: *
cache-control: public, max-age=0, must-revalidate
Connection: keep-alive
content-type: text/html
Date: Mon, 12 Sep 2022 19:34:04 GMT
etag: "277fef6d252f329813a333e286f6a5fb"
Keep-Alive: timeout=5
referrer-policy: strict-origin-when-cross-origin
Transfer-Encoding: chunked
x-content-type-options: nosniff
x-server-env: dev

Prod:

access-control-allow-origin: *
alt-svc: h3=":443"; ma=86400, h3-29=":443"; ma=86400
cache-control: public, max-age=0, must-revalidate
cf-ray: -snip-
content-encoding: br
content-type: text/html; charset=utf-8
date: Mon, 12 Sep 2022 19:33:54 GMT
etag: W/"277fef6d252f329813a333e286f6a5fb"
nel: {"success_fraction":0,"report_to":"cf-nel","max_age":604800}
referrer-policy: strict-origin-when-cross-origin
report-to: -snip-
vary: Accept-Encoding
x-content-type-options: nosniff

This is also mostly the same, but content-length is omitted entirely, and the Etag is converted from a strong one in dev, to a weak one in prod. I imagine this one is likely due to the brotli encoding so not a huge amount can be done about the differences, outside of an option to perhaps enable compression in dev.