Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Slow requests to ao_speedup_cachebuster: Revival (AWS environment health Degraded) #195

Closed
Blackbam opened this Issue Oct 2, 2018 · 11 comments

Comments

Projects
None yet
4 participants
@Blackbam
Copy link

Blackbam commented Oct 2, 2018

This ticket is very similar to #104 . I got hundreds of AWS alerts today saying things similar to this:

Message: Environment health has transitioned from Ok to Degraded. ELB health is failing or not available for 1 out of 2 instances.

Looking up the Apache log files I noticed thousands of requests like this:

[02/Oct/2018:11:47:04 +0000] "GET /at/?ao_speedup_cachebuster=91641 HTTP/1.1" 200 11765 "https://example.com/at/?ao_speedup_cachebuster=91641" "WordPress/4.9.8; https://example.com/at"
[02/Oct/2018:11:47:06 +0000] "GET /de/?ao_speedup_cachebuster=95599 HTTP/1.1" 200 10971 "https://example.com/de/?ao_speedup_cachebuster=95599" "WordPress/4.9.8; https://example.com/de"
[02/Oct/2018:11:47:06 +0000] "GET /de/examplesystem/images/rl7vv/index.asp?tid=4579 HTTP/1.1" 404 6878 "-" "Mozilla/5.0 (compatible; MJ12bot/v1.4.8; http://mj12bot.com/)"
[02/Oct/2018:11:47:07 +0000] "GET /at/?ao_speedup_cachebuster=50662 HTTP/1.1" 200 11765 "https://example.com/at/?ao_speedup_cachebuster=50662" "WordPress/4.9.8; https://example.com/at"
[02/Oct/2018:11:47:07 +0000] "GET /at/?ao_speedup_cachebuster=91659 HTTP/1.1" 200 11765 "https://example.com/at/?ao_speedup_cachebuster=91659" "WordPress/4.9.8; https://example.com/at"
[02/Oct/2018:11:47:08 +0000] "GET /de/?ao_speedup_cachebuster=18647 HTTP/1.1" 200 10856 "https://example.com/de/?ao_speedup_cachebuster=18647" "WordPress/4.9.8; https://example.com/de"

... which obviously are generated by Autoptimize. The root cause MIGHT have been a PHP fatal error with a missing function in the Theme. However there should be at the Plugins' site to prevent this problem escalating?

@futtta

This comment has been minimized.

Copy link
Owner

futtta commented Oct 2, 2018

Hmm ... this issue isn't slow requests, but too many requests, no?

Now the issue with stale object caches (which was mentioned in the linked issue) as reason for too many ao_speedup_cachebuster requests was fixed by checking a transient with an hour lifetime (see https://github.com/futtta/autoptimize/blob/beta/classes/autoptimizeVersionUpdatesHandler.php#L79-L91), so either transients are not functioning correctly on your installation or (more probable) you're having an issue with something triggering very frequent page cache purges (which AO now listens for to clear it's own cache). Are you using a page caching plugin and if so which one?

If this is the problem, you can stop AO from listening for cache purges with this code snippet;
add_filter('autoptimize_filter_main_hookpagecachepurge','__return_false');

@Blackbam

This comment has been minimized.

Copy link
Author

Blackbam commented Oct 2, 2018

Thanks a lot for your fast reaction and detailed answer. Yes I think the problem are too many requests (like 50-100 within 5-10 seconds regarding to the logfiles, which should be enough to slow the server down).

Indeed a page caching plugin is used. Comet Cache is quite a simple Plugin just rendering all pages to static as they are (https://de.wordpress.org/plugins/comet-cache/). The cache expiry is set to 7 days as long as no user triggers deletion and there is no change in the page contents.

Theoretically it might be responsible for those problems though Autoptimize and Comet Cache have been working well together as it seems until the latest Autoptimize update. Was there a major change in the cache clearing policy?

(Definitly I will implement your approach regarding the the cleared caches as in our deployment cycle autoptimize only has to generate the files after each deployment per multisite site once).

@futtta

This comment has been minimized.

Copy link
Owner

futtta commented Oct 2, 2018

@baksteendegeweldige

This comment has been minimized.

Copy link

baksteendegeweldige commented Nov 9, 2018

Hello Frank, I wrote you a mail. I have the same problem. Too many requests to ao_speedup_cachebuster.
My website (www.lazertag.be) goes down a few times a day for a few minutes due to this. I use WP Super Cache as caching plugin. Where should I add the code add_filter('autoptimize_filter_main_hookpagecachepurge','__return_false'); ?

@futtta

This comment has been minimized.

Copy link
Owner

futtta commented Nov 10, 2018

@futtta

This comment has been minimized.

Copy link
Owner

futtta commented Nov 20, 2018

fyi; I've since learned Divi is clearing AO's cache way to much (as in per post/page publish or even save), see https://blog.futtta.be/2018/11/17/warning-divi-purging-autoptimizes-cache/. as a cache clear causes AO to (p)re-populate the cache by doing a HTTP-req to the homepage (with the cachebuster string), this could be the culprit if you're on Divi.

@futtta

This comment has been minimized.

Copy link
Owner

futtta commented Dec 6, 2018

closing due to lack of actionable feedback, feel free to re-open when more info can be provided.

@futtta futtta closed this Dec 6, 2018

@Frankytank1212

This comment has been minimized.

Copy link

Frankytank1212 commented Dec 11, 2018

Hi!

I've read this thread and I had the same problem this weekend, I think the code snippet would prevent my site from crashing due to the bot crawl.

I just have a question: where do I insert it? anywhere in function.php of my theme?

Many thanks, sorry for opening up this thread again.

@futtta

This comment has been minimized.

Copy link
Owner

futtta commented Dec 11, 2018

the ao_speedup_cachebuster request are never due to bot crawls @Frankytank1212 , they are only executed when AO's cache is cleared (manually by you or by plugins that -sometimes wrongly think they have to- clear AO's cache-.

But to answer your question; the safest/ easiest way to add code snippets is using the code snippets plugin (there are alternatives, some people add them to the theme's functions.php but that would get overwritten during theme updates).

hope this helps,
frank

@Frankytank1212

This comment has been minimized.

Copy link

Frankytank1212 commented Dec 11, 2018

Hi futta,

I'm sorry if I'm mistaken or if I replied to the wrong thread.

This week-end the MJ12bot crawled my site and made a lot of requests witch caused the site to crashed two night in a row for a couple of hours.

[proxy_fcgi:error] [pid 10580:tid 140613471172352] (70007)The timeout specified has expired: [client 108.162.216.157:29102] AH01075: Error dispatching request to : (polling), referer: https://example.ca/?ao_speedup_cachebuster=93945

Here is the explanation from my host support : This is likely why the site crawler was causing such high load because it tries to index the entire site and if it keeps receiving timeouts such as the one above, it would cause even more unnecessary load on the server.

He suggested me to disable auptimize but I like the plugin, so I was wondering if this solution could help.

Many thanks for your time

@futtta

This comment has been minimized.

Copy link
Owner

futtta commented Dec 11, 2018

no worries @Frankytank1212, some more questions maybe:

  • were the cachebuster-requests coming from "outside" or from the server's own IP-address?
  • did the MJ12bot all have the ao_speedup_cachebuster in the querystring?
  • how many requests with ao_speedup_cachebuster in the querystring were we talking about?
  • are you sure you don't have a plugin or theme that is trying to clear AO's cache (thereby triggering the "ao cache preload" which is the function responsible for the ao_speedup_cachebuster requests?
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.