Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add troubleshooting guide for http_resp_hdr_len #519

Merged
1 commit merged into from Feb 11, 2016

Conversation

Projects
None yet
3 participants
@maxbucknell
Copy link
Contributor

maxbucknell commented Feb 10, 2016

We recently had an issue where Varnish would serve some pages (startlingly quickly), and give a backend fetch error (503) on others. It turns out that it is due to the length of the cache tags list being longer than the maximum header length that varnish expects.

Fortunately, this is configurable.

Attached is a screenshot of what the change looks like once compiled.

Screenshot

@ghost ghost self-assigned this Feb 10, 2016

@piotrekkaminski

This comment has been minimized.

Copy link
Contributor

piotrekkaminski commented Feb 10, 2016

@xcomSteveJohnson this is quite critical issue, high priority

@ghost

This comment has been minimized.

Copy link

ghost commented Feb 11, 2016

@maxbucknell I need more context from your /etc/sysconfig/varnish file. I do not have Varnish installed right now and I need to know in particular:

  • Is http_resp_hdr_len typically already in the configuration file or do you have to add it?
  • If you have to add it, what section does it go in?

Thanks

@davidalger

This comment has been minimized.

Copy link
Member

davidalger commented Feb 11, 2016

@xcomSteveJohnson the default /etc/sysconfig/varnish file looks like this:

# Configuration file for Varnish Cache
#
# /etc/init.d/varnish expects the variable $DAEMON_OPTS to be set from this
# shell script fragment.
#

# Maximum number of open files (for ulimit -n)
NFILES=131072

# Locked shared memory (for ulimit -l)
# Default log size is 82MB + header
MEMLOCK=82000

# Maximum number of threads (for ulimit -u)
NPROCS="unlimited"

# Maximum size of corefile (for ulimit -c). Default in Fedora is 0
# DAEMON_COREFILE_LIMIT="unlimited"

# Init script support to reload/switch vcl without restart.
# To make this work, you need to set the following variables
# explicit: VARNISH_VCL_CONF, VARNISH_ADMIN_LISTEN_ADDRESS,
# VARNISH_ADMIN_LISTEN_PORT, VARNISH_SECRET_FILE.
RELOAD_VCL=1

# Main configuration file.
VARNISH_VCL_CONF=/etc/varnish/default.vcl
#
# Default address and port to bind to
# Blank address means all IPv4 and IPv6 interfaces, otherwise specify
# a host name, an IPv4 dotted quad, or an IPv6 address in brackets.
VARNISH_LISTEN_ADDRESS=127.0.0.1
VARNISH_LISTEN_PORT=6081
#
# Telnet admin interface listen address and port
VARNISH_ADMIN_LISTEN_ADDRESS=127.0.0.1
VARNISH_ADMIN_LISTEN_PORT=6082
#
# Shared secret file for admin interface
VARNISH_SECRET_FILE=/etc/varnish/secret
#
# The minimum number of worker threads to start
VARNISH_MIN_THREADS=50
#
# The Maximum number of worker threads to start
VARNISH_MAX_THREADS=1000
#
# Cache file size: in bytes, optionally using k / M / G / T suffix.
VARNISH_STORAGE_SIZE=256M
#
# Backend storage specification
VARNISH_STORAGE="malloc,${VARNISH_STORAGE_SIZE}"
#
# Default TTL used when the backend does not specify one
VARNISH_TTL=120
#
# DAEMON_OPTS is used by the init script.
DAEMON_OPTS="-a ${VARNISH_LISTEN_ADDRESS}:${VARNISH_LISTEN_PORT} \
             -f ${VARNISH_VCL_CONF} \
             -T ${VARNISH_ADMIN_LISTEN_ADDRESS}:${VARNISH_ADMIN_LISTEN_PORT} \
             -p thread_pool_min=${VARNISH_MIN_THREADS} \
             -p thread_pool_max=${VARNISH_MAX_THREADS} \
             -S ${VARNISH_SECRET_FILE} \
             -s ${VARNISH_STORAGE}"

That option could/should be added after the thread_pool_max param that's set in DAEMON_OPTS

ghost pushed a commit that referenced this pull request Feb 11, 2016

Steve Johnson
Merge pull request #519 from maxbucknell/varnish-http_resp_hdr_len
Add troubleshooting guide for http_resp_hdr_len

@ghost ghost merged commit dd490b1 into magento:2.0 Feb 11, 2016

@ghost

This comment has been minimized.

Copy link

ghost commented Feb 11, 2016

Thanks @davidalger.

@maxbucknell, I'm going to merge this and make some changes to it. I'm going to move it to a troubleshooting section and reword some things.

I'll update the closed PR so you can check it out.

@ghost

This comment has been minimized.

Copy link

ghost commented Feb 11, 2016

@maxbucknell Please check out the new topic.

@maxbucknell

This comment has been minimized.

Copy link
Contributor Author

maxbucknell commented Feb 11, 2016

Thanks for taking this and running with it. The new topic looks great, but the particular error caused by this is 503 Backend Fetch Failed, rather than 503 Service Unavailable.

@maxbucknell

This comment has been minimized.

Copy link
Contributor Author

maxbucknell commented Feb 11, 2016

Also, there is a typo in the first line:

If the length of cache tags used by Magento exceed Varnish’s default of 8192 errors,...

Should be "8192 bytes".

It would also be handy to have a recommendation on what to set this value to. Setting it to 64000 will give you the same error if you have a category with around 3050 products (every product tag is about 21 characters long). While that's a lot, it's totally conceivable.

I wrote a blog post on the topic, wherein I suppose that a sensible value is the product count of your largest category multiplied by 30. This gives you room to grow and keeps memory usage in check, since I suspect increasing this parameter to something really large will have an adverse impact on memory consumption of Varnish.

@ghost

This comment has been minimized.

Copy link

ghost commented Feb 11, 2016

Thanks. I made those changes.

magento-cicd2 pushed a commit that referenced this pull request Dec 14, 2016

This issue was closed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.