New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Problem with pantheon recipe, advanced page cache , and varnish proxy header limits configuration. #1142

Closed
bob-hinrichs opened this Issue Aug 17, 2018 · 6 comments

Comments

Projects
None yet
5 participants
@bob-hinrichs
Copy link

bob-hinrichs commented Aug 17, 2018

Bug Report

running pantheon recipe, on a mac High Sierra (likely not mac-specific), running Drupal 7 with pantheon advanced page cache.
v3.0.0-beta.47

Tell us about your .lando.yml
name: cse-1
recipe: pantheon
config:
framework: drupal
site: cse1
id: edc7f73f-ddbb-404b-a5e2-e8890ee8c86a

It is also helpful to include other relevant config files. For example, include pantheon.yml for a pantheon recipe
api_version: 1

php_version: 7.1

web_docroot: true

Paste in a series of commands that caused the bug.
Firing up a somewhat complex site that issues a healthy number of cache tags. It does not take many to cause lando to fail. The pantheon advanced page cache threshold is MUCH more data than is being allowed by the edge service. Fix is below.

Tell us about the error you got

Varnish chokes on some pages and issues a 503 for some pages because the Drupal backend issues a HTTP reply header that exceeds the default size allowed by varnish.

Does this bug prevent you from using lando?
It reduces its value because we cannot test the site with edge URL.

TELL US HOW TO FIX IT

If you enter the varnish container in a bash shell and run a curl command to retrieve this url (using 'nginx' in place of cse1.lndo.site in the example URL), you'll get a response.

However, if you tail the varnish log (by running "varnishlog") and then try to request the URL in your browser, varnishlog will show it complains about the HTTP header size returned by drupal.

There's probably an easier way to remedy this, but the workaround I have for now is the following, which involves editing the varnish configuration:

(1) Go into the varnish container:

docker exec -it cse1_edge_1

(2) install vim (or whatever cli text editor you prefer)

apt-get update && apt-get install -y vim

(3) Edit the chaperone config file (chaperone is used to start varnish with specific parameters) so that the following line is change from

PARAM_VALUE: "${PARAM_VALUE:--p default_ttl=3600 -p default_grace=3600}",

to:

PARAM_VALUE: "${PARAM_VALUE:--p default_ttl=3600 -p default_grace=3600 -p http_resp_hdr_len=32768}",

@dustinleblanc

This comment has been minimized.

Copy link
Member

dustinleblanc commented Oct 23, 2018

Do you have http.response.debug_cacheability_headers: true in your development.services.yml and are you loading that file? I've run into this frequently on Pantheon sites as it increases the size of the headers beyond what Styx can handle. This will often fail locally and not on Pantheon because you only include the file locally.

@bob-hinrichs

This comment has been minimized.

Copy link
Author

bob-hinrichs commented Oct 23, 2018

Thank you. No, this file does not exist. When I tried to find the cause, it seems lando's setting for the header length is small. Another fix was to edit the pantheon_advanced_page_cache.module and cut the tags string value to a much shorter length. So it seems the issue was not bloated headers from debugging, but a threshold that is very low, lower than the module is expecting. I checked again and it is still happening, using the latest lando rc1.

@possiBri

This comment has been minimized.

Copy link

possiBri commented Jan 7, 2019

Hi I'm also experiencing this issue. We are on D7, but as far as I can tell we don't have the Pantheon Advanced Page Cache module (unless I'm misunderstanding what you're referring to). I was hoping to get some clarity on how I can debug this issue to determine the root cause. I suspect it's related to query timeouts and some stuff that I should optimize, but since it's only happening on my local it's hard to be sure it's that and not also this issue with the Lando settings. Any suggestions/guidance on how I might fix these settings is greatly appreciated!

@torgospizza

This comment has been minimized.

Copy link

torgospizza commented Jan 9, 2019

I ran into this issue, and disabled the Pantheon Advanced Page Cache module solved it for us as well. Happy to attempt other methods of debugging to get this fixed. Thanks!

@pirog pirog self-assigned this Jan 28, 2019

@pirog pirog modified the milestone: 3.0.0-rc.2 Jan 28, 2019

@pirog pirog removed their assignment Jan 29, 2019

@pirog pirog removed the support request label Jan 31, 2019

@pirog pirog added this to the 3.0.0-rc.3 milestone Feb 1, 2019

@pirog pirog self-assigned this Feb 3, 2019

@pirog

This comment has been minimized.

Copy link
Member

pirog commented Feb 4, 2019

you probably want to disable the Pantheon Advanced Page Cache module locally but i think we should also just bump the max varnish header size

pirog added a commit that referenced this issue Feb 4, 2019

@pirog

This comment has been minimized.

Copy link
Member

pirog commented Feb 4, 2019

set http_resp_hdr_len to 64000 because yolo

@pirog pirog closed this Feb 4, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment