reverseproxy: Set the headers in the replacer before handle_response

[francislavoie]

Turns out this was an oversight, we assumed we could use {http.response.header.*} but that doesn't work because those are grabbed from the response writer, and we haven't copied any headers into the response writer yet.

So the fix is to set all the response headers into the replacer at a new namespace before running the handlers.

This adds the {http.reverse_proxy.header.*} replacer.

See https://caddy.community/t/empty-http-response-header-x-accel-redirect/12447

[mholt]

Thanks, interesting situation... I guess this'll work, assuming we never need to differentiate request headers and response headers in the future.

[francislavoie]

I think we'd go with http.reverse_proxy.request_header.* if we ever did, which matches the request_header handler.

[corny]

I have another related issue: If the upstream server returns a Content-Disposition header, it won't be copied into the response. I need to add this line to the handle_response block:

header Content-Disposition "{http.reverse_proxy.header.Content-Disposition}"

Update: Same applies to the Content-Type header.

[francislavoie]

That's normal. No headers are copied by default in handle_response, you need to do it yourself for anything that's relevant. It just gives you a clean slate to re-handle the request based on the response from the upstream.

[corny]

I see that Set-Cookie, X-Request-Id and a couple of other headers are copied. That's a little bit confusing, isn't it?

[francislavoie]

Where do you see that @corny? The code doesn't copy any headers.

caddy/modules/caddyhttp/reverseproxy/reverseproxy.go

Lines 613 to 652 in b82db99

	for i, rh := range h.HandleResponse {
	if rh.Match != nil && !rh.Match.Match(res.StatusCode, res.Header) {
	continue
	}
	// if configured to only change the status code, do that then continue regular proxy response
	if statusCodeStr := rh.StatusCode.String(); statusCodeStr != "" {
	statusCode, err := strconv.Atoi(repl.ReplaceAll(statusCodeStr, ""))
	if err != nil {
	return caddyhttp.Error(http.StatusInternalServerError, err)
	}
	if statusCode != 0 {
	res.StatusCode = statusCode
	}
	break
	}
	// otherwise, if there are any routes configured, execute those as the
	// actual response instead of what we got from the proxy backend
	if len(rh.Routes) == 0 {
	continue
	}
	res.Body.Close()
	// set up the replacer so that parts of the original response can be
	// used for routing decisions
	for field, value := range res.Header {
	repl.Set("http.reverse_proxy.header."+field, strings.Join(value, ","))
	}
	repl.Set("http.reverse_proxy.status_code", res.StatusCode)
	repl.Set("http.reverse_proxy.status_text", res.Status)
	h.logger.Debug("handling response", zap.Int("handler", i))
	if routeErr := rh.Routes.Compile(next).ServeHTTP(rw, req); routeErr != nil {
	// wrap error in roundtripSucceeded so caller knows that
	// the roundtrip was successful and to not retry
	return roundtripSucceeded{routeErr}
	}
	}

What does your Caddyfile look like? How did you build Caddy?