4.1.x sometimes duplicates Age and Accept-Ranges headers #1955

Closed
blblack opened this Issue May 17, 2016 · 6 comments

Projects

None yet

5 participants

@blblack
Contributor
blblack commented May 17, 2016 edited

With Varnish 4.1.x in a production configuration that layers multiple varnish caches requesting from other varnish caches, in the final output we often see duplicate (or even triplicate) Age: and Accept-Ranges: headers.

Relatedly, we backported d828a04 into our 4.1.x package, and this did seem to resolve the issue for 304 refreshes of stale objects, but not for the general case.

Expected Behavior

Expect only 1 copy each of Age: and Accept-Ranges: headers to be emitted by Varnish to the client.

Current Behavior

Seeing 2-3x of these headers headers sent to client, one for each layer of Varnish cache the request passes through. They seem to always have identical values, at least in short-TTL cases, but this may be mostly a coincidence.

Steps to Reproduce (for bugs)

Live example: https://maps.wikimedia.org/ (2-4 layers of varnish caching, depending where you are geographically).
Manual repro: Start 2x varnishd with default VCL on the CLI with the first backending to some standard webserver, and the second backending to the first. Make a request through the stack and observe output headers:

$ /usr/sbin/varnishd -a :3133 -b <IP of some apache server>:80 -p default_ttl=5 -p default_grace=10 -F -T :8188 -n backvarnish &
$ /usr/sbin/varnishd -a :3134 -b localhost:3133 -p default_ttl=5 -p default_grace=10 -F -n frontvarnish &
$ curl -v http://localhost:3134/ >/dev/null
[....]
< Content-Type: text/html; charset=UTF-8
< X-Varnish: 2
< Age: 0
< Via: 1.1 varnish-v4
< Accept-Ranges: bytes
< X-Varnish: 2
< Age: 0
< Via: 1.1 varnish-v4
< Accept-Ranges: bytes
< Transfer-Encoding: chunked
< Connection: keep-alive
<
{ [data not shown]
@hermunn hermunn self-assigned this May 23, 2016
@hermunn
Contributor
hermunn commented May 24, 2016

I am working on this now, but I can't reproduce the problem. I have tried the realeases 4.1.1 and 4.1.2, as well as the 4.1 and master branches.

I can see you problem in your live example, but it would be very helpful to have a minimal example (preferably varnishtest) on my own computer where multiple Age headers appear.

I will try again tomorrow.

@fgsch
Member
fgsch commented May 24, 2016 edited

Here.

varnishtest ""

server s1 {
    rxreq
    txresp -hdr "Cache-Control: max-age=1"
} -start

varnish v1 -vcl+backend {
} -start

varnish v2 -arg "-pdefault_grace=1" -vcl {
    import std;
    backend b1 {
        .host = "${v1_addr}";
        .port = "${v1_port}";
    }
    sub vcl_deliver {
        std.collect(resp.http.age);
    }
} -start

client c1 -connect ${v2_sock} {
    txreq
    rxresp
    delay 2
    txreq
    rxresp
    expect resp.http.age == "2"
} -run
@fgsch
Member
fgsch commented May 25, 2016 edited

The second request in v2 will have age 2 and max-age 1, so the effective ttl is -1 (uncacheable).
For uncacheable objects we don't filter out the Age or Accept-Ranges headers when calling HTTP_Encode() in vbf_beresp2obj() as it's called with HTTPH_R_PASS.

@hermunn
Contributor
hermunn commented May 25, 2016

Thanks, @fgsch! Much appreciated.

@hermunn hermunn added a commit to hermunn/varnish-cache that referenced this issue May 25, 2016
@hermunn hermunn Eliminate duplicate Age and Accept-Ranges headers.
When we have several layers of Varnish, in some cases we would send
multiple Age and Accept-Ranges headers to the client. This is fixed in
this commit. The test case demonstrates the problem for the Age
header.

Fixes: #1955
08a872b
@ehocdet
Contributor
ehocdet commented May 25, 2016

We also have layers multiple Varnish. We fix this issue on Varnish front with:

sub vcl_deliver {
      set resp.http.Age = resp.http.Age;
}
@hermunn hermunn added a commit to hermunn/varnish-cache that referenced this issue May 26, 2016
@hermunn hermunn Eliminate duplicate Age and Accept-Ranges headers.
When we have several layers of Varnish, in some cases we would send
multiple Age and Accept-Ranges headers to the client. This is fixed in
this commit. The test case demonstrates the problem for the Age
header.

Fixes: #1955
dc82d53
@fgsch fgsch added a commit to fgsch/varnish-cache that referenced this issue May 29, 2016
@fgsch fgsch Don't emit multiple Age or Accept-Ranges headers
This might happen during pass'd requests or in a multi-tier setup.
Fixes #1955.
6f3feda
@fgsch fgsch added a commit that closed this issue May 30, 2016
@fgsch fgsch Don't emit multiple Age or Accept-Ranges headers
This might happen during pass'd requests or in a multi-tier setup.
Fixes #1955.
5d27296
@fgsch fgsch closed this in 5d27296 May 30, 2016
@hermunn hermunn added a commit that referenced this issue May 31, 2016
@fgsch @hermunn fgsch + hermunn Don't emit multiple Age or Accept-Ranges headers
This might happen during pass'd requests or in a multi-tier setup.
Fixes #1955.
925b5f7
@hermunn hermunn added a commit that referenced this issue May 31, 2016
@hermunn hermunn Add #1955 to the changelog.
Ref: #1955
b4c569a
@hermunn
Contributor
hermunn commented Jun 3, 2016

Backport to 4.1 complete (925b5f7)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment