Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Vary headers in responses #536

Merged
merged 3 commits into from Mar 16, 2023
Merged

Add Vary headers in responses #536

merged 3 commits into from Mar 16, 2023

Conversation

kotx
Copy link
Contributor

@kotx kotx commented Mar 12, 2023

This prevents caching servers like Cloudflare from caching responses when Accept or X-Decrypt-Password request headers vary. Before this, servers like Cloudflare would cache the html preview page regardless of Accept header (meaning curls and non-browser requests would also return a cached preview page). Adding a Vary header should prevent that.

More info here: https://blog.cloudflare.com/vary-for-images-serve-the-correct-images-to-the-correct-browsers/

Copy link
Collaborator

@aspacca aspacca left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@kotx thank you very much for the contribution

please, see my comments

@@ -243,6 +243,8 @@ func canContainsXSS(contentType string) bool {

/* The preview handler will show a preview of the content for browsers (accept type text/html), and referer is not transfer.sh */
func (s *Server) previewHandler(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Vary", "Accept, X-Decrypt-Password")
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

previewHandler should not be sensible to X-Decrypt-Password

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

indeed not event to Accept, or do I miss anything?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the Vary header should always be the same for the same URL.

I had this doubt just after leaving the comment

the for previewHandler, getHandler and headHandler we must set Range, X-Decrypt-Password
for viewHandler only Accept

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

viewHandler is only for /, yes? I don't know if it should vary based on X-Decrypt-Password, only for the file route.
Is that okay?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should be fixed in the latest commit

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sorry, file route should have w.Header().Set("Vary", "Range, Referer, X-Decrypt-Password")

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, I thought getHandler was the file route, do you mean zipHandler, tarHandler, tarGzHandler, etc should also have the vary header?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

these three ones:

https://github.com/dutchcoders/transfer.sh/pull/536/files#diff-961509ed3ff668f3172148150903514fb3394d6ff090d6db78d986875b13689fR246
https://github.com/dutchcoders/transfer.sh/pull/536/files#diff-961509ed3ff668f3172148150903514fb3394d6ff090d6db78d986875b13689fR1161
https://github.com/dutchcoders/transfer.sh/pull/536/files#diff-961509ed3ff668f3172148150903514fb3394d6ff090d6db78d986875b13689fR1255

currently they have w.Header().Set("Vary", "Accept, Referer, X-Decrypt-Password")
it should be w.Header().Set("Vary", "Range, Referer, X-Decrypt-Password")

response changes based on Accept only for / (viewHandler)
reponse for /{token}/{filename} changes Range, Referer, X-Decrypt-Password

Range and X-Decrypt-Password are used on getHandler
according to the Referer value we route to previewHandler or getHandler

finally headHandler should have the same Vary value since this should be the same based on URL and not URL+HTTP Method

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got it, so sorry for the confusion!
Made the changes you requested

server/handlers.go Outdated Show resolved Hide resolved
server/handlers.go Outdated Show resolved Hide resolved
@aspacca
Copy link
Collaborator

aspacca commented Mar 12, 2023

More info here: https://blog.cloudflare.com/vary-for-images-serve-the-correct-images-to-the-correct-browsers/

I'm not 100% sure about the cloudflare behaviour on the Vary header: the article mentions only Accept, is just for the sake of the example?

If not, and they only "react" to Accept it seems to me that they are forcing the upstream service behind their cache to send a value that does not respect the real ones that should be set: in this case I would set the proper ones, and if it still does not work with cloudflare cache please ask them to fix ;)

@kotx
Copy link
Contributor Author

kotx commented Mar 12, 2023

More info here: https://blog.cloudflare.com/vary-for-images-serve-the-correct-images-to-the-correct-browsers/

I'm not 100% sure about the cloudflare behaviour on the Vary header: the article mentions only Accept, is just for the sake of the example?

I've only just discovered this, but apparently Cloudflare only allows Vary for images on the non-free plans? :(

But normally, Cloudflare and any server should not cache if the headers in Vary are different.

@aspacca
Copy link
Collaborator

aspacca commented Mar 12, 2023

But normally, Cloudflare and any server should not cache if the headers in Vary are different.

yes, I'm fine adding the Vary header, just I won't implement it to make cloudflare happy, but according to the rfc :)

@aspacca
Copy link
Collaborator

aspacca commented Mar 14, 2023

@kotx could you please merge the main branch on yours?
thanks

@kotx
Copy link
Contributor Author

kotx commented Mar 15, 2023

could you please merge the main branch on yours? thanks

Done, hope this works

@aspacca aspacca merged commit 3dcbfe2 into dutchcoders:main Mar 16, 2023
6 checks passed
@kotx kotx deleted the kot/vary branch March 16, 2023 03:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants