Fetching sharded resources from original domain

[GoogleCodeExporter]

What steps will reproduce the problem?
1. ModPagespeedShardDomain example.com s1.cdn.net,s2.cdn.net,s3.cdn.net
2. s*.cdn.net does not have mod_pagespeed installed and just acts as a reverse 
proxy converting all requests for s*.cdn.net/foo -> example.com/foo
3. User requests http://s1.cdn.net/static/mycss.css.ce.asdfasdf.css -> CDN 
requests http://example.com/static/mycss.css.ce.asdfasdf.css -> mod_pagespeed 
cannot find resource because it is being requested on the wrong domain.


This is actually documented behavoir. Specifically, ShardDomain requires that 
the servers you rewrite the domains to point to have mod_pagespeed installed.

However, this seems like a reasonable use-case and we should be able to figure 
out which resource was intended.

Original issue reported on code.google.com by sligocki@google.com on 19 Aug 2011 at 2:31

• Blocked on: pagespeed css fetched by CDN result in cache-control:private,max-age=300 #494

[GoogleCodeExporter]

One possible solution, canonicalize the domain name before cache lookup (say 
s*.cdn.net all get canonicalized to example.com).

Original comment by sligocki@google.com on 19 Aug 2011 at 2:33

[GoogleCodeExporter]

Any movement on this?

Original comment by tre...@rallydev.com on 6 Mar 2012 at 11:48

[GoogleCodeExporter]

I think this is similar what I'm trying to fix to address Issue 494.  Issue 494 
does not include sharding, but does manifest as a different symptom: a 
mismatching hash causes MPS to serve the CSS file with 
cache-control:private,max-age=300.  In this report it is a 404.

The difference might be that the use-case in 494 includes a strategy for 
mod_pagespeed to fetch the resources for the CDN using ModPagespeedLoadFromFile.

Original comment by jmara...@google.com on 14 Sep 2012 at 7:12

• Changed state: Started
• Added labels: Type-Defect
• Removed labels: Type-Enhancement

[GoogleCodeExporter]

Original comment by jmara...@google.com on 14 Sep 2012 at 7:25

• Now blocked on: pagespeed css fetched by CDN result in cache-control:private,max-age=300 #494

[GoogleCodeExporter]

One thing that was not mentioned here was the site's strategy for serving the 
resources to the CDN when the CDN requests them.

In Issue 494 this was done via
   ModPagespeedLoadFromFile HTTP_PATH_FETCHED_BY_CDN LOCATION_OF_FILES_ON_DISK
and that is also the mechanism I've used to test my candidate fix.  Another 
option is to use ModPagespeedMapOriginDomain.

CaptinDave, what's your strategy for serving the fetches made from the CDN back 
to the origin?

Original comment by jmara...@google.com on 14 Sep 2012 at 8:02

[GoogleCodeExporter]

I have tested this and everything works as I expect:
* fetching of the resource succeeds
* if the correct hash is used then resource is optimized
* if the wrong hash is used then, in my testing, the original resource is
  returned unoptimized because we get a cache miss as the URL doesn't match
  that in the cache (b/c of the hash) and we fail to fetch and optimize the
  original resource within the deadline. However, the fetch DOES succeed.

Note that in the last case we end up with the same optimized resource to
put back into the cache (same URL and contents) which is wasteful but I
don't think we can much about that - we need the content to determine the
correct hash after all.

Original comment by matterb...@google.com on 2 Nov 2012 at 6:32

• Changed state: Fixed
• Added labels: Milestone-v23, release-note