New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

4.1.x sometimes duplicates Age and Accept-Ranges headers #1955

Closed
blblack opened this Issue May 17, 2016 · 6 comments

Comments

Projects
None yet
5 participants
@blblack
Contributor

blblack commented May 17, 2016

With Varnish 4.1.x in a production configuration that layers multiple varnish caches requesting from other varnish caches, in the final output we often see duplicate (or even triplicate) Age: and Accept-Ranges: headers.

Relatedly, we backported d828a04 into our 4.1.x package, and this did seem to resolve the issue for 304 refreshes of stale objects, but not for the general case.

Expected Behavior

Expect only 1 copy each of Age: and Accept-Ranges: headers to be emitted by Varnish to the client.

Current Behavior

Seeing 2-3x of these headers headers sent to client, one for each layer of Varnish cache the request passes through. They seem to always have identical values, at least in short-TTL cases, but this may be mostly a coincidence.

Steps to Reproduce (for bugs)

Live example: https://maps.wikimedia.org/ (2-4 layers of varnish caching, depending where you are geographically).
Manual repro: Start 2x varnishd with default VCL on the CLI with the first backending to some standard webserver, and the second backending to the first. Make a request through the stack and observe output headers:

$ /usr/sbin/varnishd -a :3133 -b <IP of some apache server>:80 -p default_ttl=5 -p default_grace=10 -F -T :8188 -n backvarnish &
$ /usr/sbin/varnishd -a :3134 -b localhost:3133 -p default_ttl=5 -p default_grace=10 -F -n frontvarnish &
$ curl -v http://localhost:3134/ >/dev/null
[....]
< Content-Type: text/html; charset=UTF-8
< X-Varnish: 2
< Age: 0
< Via: 1.1 varnish-v4
< Accept-Ranges: bytes
< X-Varnish: 2
< Age: 0
< Via: 1.1 varnish-v4
< Accept-Ranges: bytes
< Transfer-Encoding: chunked
< Connection: keep-alive
<
{ [data not shown]

@hermunn hermunn self-assigned this May 23, 2016

@hermunn

This comment has been minimized.

Show comment
Hide comment
@hermunn

hermunn May 24, 2016

Contributor

I am working on this now, but I can't reproduce the problem. I have tried the realeases 4.1.1 and 4.1.2, as well as the 4.1 and master branches.

I can see you problem in your live example, but it would be very helpful to have a minimal example (preferably varnishtest) on my own computer where multiple Age headers appear.

I will try again tomorrow.

Contributor

hermunn commented May 24, 2016

I am working on this now, but I can't reproduce the problem. I have tried the realeases 4.1.1 and 4.1.2, as well as the 4.1 and master branches.

I can see you problem in your live example, but it would be very helpful to have a minimal example (preferably varnishtest) on my own computer where multiple Age headers appear.

I will try again tomorrow.

@fgsch

This comment has been minimized.

Show comment
Hide comment
@fgsch

fgsch May 24, 2016

Member

Here.

varnishtest ""

server s1 {
    rxreq
    txresp -hdr "Cache-Control: max-age=1"
} -start

varnish v1 -vcl+backend {
} -start

varnish v2 -arg "-pdefault_grace=1" -vcl {
    import std;
    backend b1 {
        .host = "${v1_addr}";
        .port = "${v1_port}";
    }
    sub vcl_deliver {
        std.collect(resp.http.age);
    }
} -start

client c1 -connect ${v2_sock} {
    txreq
    rxresp
    delay 2
    txreq
    rxresp
    expect resp.http.age == "2"
} -run
Member

fgsch commented May 24, 2016

Here.

varnishtest ""

server s1 {
    rxreq
    txresp -hdr "Cache-Control: max-age=1"
} -start

varnish v1 -vcl+backend {
} -start

varnish v2 -arg "-pdefault_grace=1" -vcl {
    import std;
    backend b1 {
        .host = "${v1_addr}";
        .port = "${v1_port}";
    }
    sub vcl_deliver {
        std.collect(resp.http.age);
    }
} -start

client c1 -connect ${v2_sock} {
    txreq
    rxresp
    delay 2
    txreq
    rxresp
    expect resp.http.age == "2"
} -run
@fgsch

This comment has been minimized.

Show comment
Hide comment
@fgsch

fgsch May 25, 2016

Member

The second request in v2 will have age 2 and max-age 1, so the effective ttl is -1 (uncacheable).
For uncacheable objects we don't filter out the Age or Accept-Ranges headers when calling HTTP_Encode() in vbf_beresp2obj() as it's called with HTTPH_R_PASS.

Member

fgsch commented May 25, 2016

The second request in v2 will have age 2 and max-age 1, so the effective ttl is -1 (uncacheable).
For uncacheable objects we don't filter out the Age or Accept-Ranges headers when calling HTTP_Encode() in vbf_beresp2obj() as it's called with HTTPH_R_PASS.

@hermunn

This comment has been minimized.

Show comment
Hide comment
@hermunn

hermunn May 25, 2016

Contributor

Thanks, @fgsch! Much appreciated.

Contributor

hermunn commented May 25, 2016

Thanks, @fgsch! Much appreciated.

hermunn added a commit to hermunn/varnish-cache that referenced this issue May 25, 2016

Eliminate duplicate Age and Accept-Ranges headers.
When we have several layers of Varnish, in some cases we would send
multiple Age and Accept-Ranges headers to the client. This is fixed in
this commit. The test case demonstrates the problem for the Age
header.

Fixes: #1955
@ehocdet

This comment has been minimized.

Show comment
Hide comment
@ehocdet

ehocdet May 25, 2016

Contributor

We also have layers multiple Varnish. We fix this issue on Varnish front with:

sub vcl_deliver {
      set resp.http.Age = resp.http.Age;
}
Contributor

ehocdet commented May 25, 2016

We also have layers multiple Varnish. We fix this issue on Varnish front with:

sub vcl_deliver {
      set resp.http.Age = resp.http.Age;
}

hermunn added a commit to hermunn/varnish-cache that referenced this issue May 26, 2016

Eliminate duplicate Age and Accept-Ranges headers.
When we have several layers of Varnish, in some cases we would send
multiple Age and Accept-Ranges headers to the client. This is fixed in
this commit. The test case demonstrates the problem for the Age
header.

Fixes: #1955

fgsch added a commit to fgsch/varnish-cache that referenced this issue May 29, 2016

Don't emit multiple Age or Accept-Ranges headers
This might happen during pass'd requests or in a multi-tier setup.
Fixes #1955.

@fgsch fgsch closed this in 5d27296 May 30, 2016

hermunn added a commit that referenced this issue May 31, 2016

Don't emit multiple Age or Accept-Ranges headers
This might happen during pass'd requests or in a multi-tier setup.
Fixes #1955.

hermunn added a commit that referenced this issue May 31, 2016

@hermunn

This comment has been minimized.

Show comment
Hide comment
@hermunn

hermunn Jun 3, 2016

Contributor

Backport to 4.1 complete (925b5f7)

Contributor

hermunn commented Jun 3, 2016

Backport to 4.1 complete (925b5f7)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment