Skip to content
This repository has been archived by the owner on Apr 21, 2023. It is now read-only.

Infinite recursion of mod_pagespeed requesting its own pages #85

GoogleCodeExporter opened this issue Apr 6, 2015 · 9 comments


Copy link

What steps will reproduce the problem?
1. Install this mod ( mod_pagespeed 64-bit .rpm (CentOS/Fedora) )

Some WP and some normal HTML sites did had this problem. A lot of connection to 
port 80, with the servers own IP... it was in some kind of loop to connect to 
the same file. At first I taught I had some kind of trojan or virus.. but after 
being checked, nothing was found. So this was the only possible solution, to 
remove this :).

What is the expected output? What do you see instead?
Speed increase, but server got stuck.. 

What version of the product are you using (please check X-Mod-Pagespeed
I downloaded it last week... Should be the new version.

On what operating system?
Centos 5.5

Which version of Apache?
Apache 2.2

Which MPM?

Please provide any additional information below, especially a URL or an
HTML file that exhibits the problem.
The problem happend on, it's just an html site, but when I 
opened the site in my browser, I could see that in my connection log files and 
also netstat that I had a lot of conections being made to the same file...

You can also see more info on this site:

Original issue reported on by on 16 Nov 2010 at 5:20

Copy link

Thanks for the report.  The symptom of 20k open connections is interesting.

The reason that it's "looping back" to itself is that this is (currently) how 
mod_pagespeed rewrites resources (css images javascript) -- by initiating an 
HTTP fetch using the "serf" package.  When this works, it works well.  But in 
the version of code you have (the last binary that was shipped) the failure 
mode is not good.

We are working on the problem.  We believe the current 'trunk' in SVN solves 
the symptom you are seeing, which is that failed fetches retry.  In your case, 
with 20k *open* connections it looks like your fetches are not failing; they 
are hanging.  In either case, there this will not happen with the latest 
source, or with a binary we will distribute soon, because we've added locking 
to prevent the same URL from being fetched more than once at a time.

But we still want to eliminate the root cause.  Can you share with us more 
about your networking setup?   If you login to a shell on the machine running  
HTTPD, can you 'wget' all the resources reference in the HTML file using the 
full URL of each resource?  That is what mod_pagespeed is trying to do.  In 
setups where HTTPD rejects fetches from localhost or the URLs for the resources 
are not valid from the HTTPD host, this failure might be possible.  Can you 
give further details on this aspect of your system?

We have also run into a certain amount of trouble interfacing to some sites 
with WP plugins, but I'm not sure if this is related to your issue or not.

Original comment by on 16 Nov 2010 at 5:46

Copy link

We should use this bug to track addressing the symptom of requests hanging 
indefinitely.  In particular, we need to very that our usage of the serf 
fetcher that we only allow requests to remain outstanding for a bounded amount 
of time -- when that time expires we should cancel the request and report a 
failure so that we 'remember' the fetch failure in the HTTP cache and avoid 
retrying for 5 minutes.

We'll still need help to identify why these fetches are hanging on

Original comment by on 16 Nov 2010 at 5:49

Copy link

Summary was: Server load above 90 !! Apache extremly overloaded, 20k connection 
on port 80 and high usage of processor

I adjusted the summary because that describes a symptom which we believe will 
be resolved with changes made in the latest binary.  However the root cause is 
still not resolved and thus we leave this bug open.

Original comment by on 17 Nov 2010 at 12:59

  • Changed title: URL Fetch requests back to same Apache server do not work properly in some environments

Copy link

It looks like there's a weird situation in which we can end up rewriting our 
own fetches (in the process spawning new fetches which themselves get 
rewritten...).  I don't really understand how that would happen -- maybe you 
have a link to a bad resource, and apache is configured to redirect 404s back 
to /index.html?

Anyway, I'll add some code to prevent mod_pagespeed from ever rewriting its own 

Original comment by on 19 Nov 2010 at 9:40

  • Changed state: Accepted

Copy link

Fixed in revision 238.

Original comment by on 20 Nov 2010 at 12:38

  • Changed title: Infinite recursion of mod_pagespeed requesting its own pages
  • Changed state: Fixed

Copy link

Issue 46 has been merged into this issue.

Original comment by on 23 Nov 2010 at 4:04

Copy link

Issue 110 has been merged into this issue.

Original comment by on 23 Nov 2010 at 4:06

Copy link


Yes, able to get the files with wget... that is the whole problem as at first I 
was tracking the errors but didn't see anything there. Then when I tracked the 
normal connections I saw A LOT of the same trying to connect to the same page.

If you want, I can check my LOG files and give some ? There you can see that it 
get's in a loop and requesting the same file over and over again...

Let me know.


Original comment by on 25 Nov 2010 at 9:44

Copy link

aLeMiM2008: have you upgraded to the latest release,  We believe 
we have resolved this infinite recursion issue.

Original comment by on 26 Nov 2010 at 4:43

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
None yet

No branches or pull requests

1 participant