Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
varnishd stalls with two consecutive Range requests using HTTP persistent connections #2035
While testing Varnish handling of Range requests we've discovered that varnishd seems to block when serving two Range requests over a HTTP persistent connection and saving a large object to file storage at the same time.
Varnish should serve Range requests for a small number of bytes fairly quickly, even if at the same time the object received from the backend is being written to disk.
Instead of quickly serving two successive Range requests for a fairly small number of bytes, varnishd seems to block and serves the second request only after a long time.
Avoiding HTTP persistent connections makes the problem unreproducible.
Steps to Reproduce (for bugs)
Start Varnish with stock VCL and a 10G file storage backend. Also make sure a web server is running on port 80 and serving a large file under /test.img. We've used a 5G file in our test environment.
Once varnishd is ready to serve requests, run the following script:
#!/usr/bin/env python import socket REPRODUCE = True dest_name = "127.0.0.1" port = 3128 s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) s.connect((dest_name, port)) s.send("""GET /test.img HTTP/1.1 Range: bytes=18-22 """) data = (s.recv(1024)) print data if not REPRODUCE: s.shutdown(1) s.close() s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) s.connect((dest_name, port)) s.send("""GET /test.img HTTP/1.1 Range: bytes=50-60 """) data = (s.recv(1024)) print data s.shutdown(1) s.close()
The first response will come back quickly and then varnish will stall:
After a while, the second response will arrive. Note the value of
Now stop varnishd, start it again, set
The issue was discovered while testing Range requests handling with two varnishes, the first one with
Please note however that the issue can be reproduced with one single varnishd as described above.