New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add memcached support as an alternative to the file-based cache. #475

Closed
GoogleCodeExporter opened this Issue Apr 6, 2015 · 10 comments

Comments

Projects
None yet
1 participant
@GoogleCodeExporter

GoogleCodeExporter commented Apr 6, 2015

Some basic support has been implemented and is under review.

Original issue reported on code.google.com by jmara...@google.com on 20 Jul 2012 at 7:51

@GoogleCodeExporter

This comment has been minimized.

GoogleCodeExporter commented Apr 6, 2015

Original comment by jmara...@google.com on 20 Jul 2012 at 8:28

  • Now blocking: #474
@GoogleCodeExporter

This comment has been minimized.

GoogleCodeExporter commented Apr 6, 2015

http://code.google.com/p/modpagespeed/source/detail?r=1704 contains the first 
round of changes needed to add memcached support.

we don't have get performance metrics; actually as it stands right now we have 
bad performance metrics, but there is hope.

Original comment by jmara...@google.com on 20 Jul 2012 at 8:30

@GoogleCodeExporter

This comment has been minimized.

GoogleCodeExporter commented Apr 6, 2015

http://code.google.com/p/modpagespeed/source/detail?r=1707 adds an option 
"ModPagespeedMemcachedServers" to mod_pagespeed.  Note that this support for 
this appears functional but is completely untuned.  In particular:

1. No testing for multiple memcached servers is done yet, though support for 
this
   appears to be built into the underlying apr_memcache* calls in apr_util.
2. The performance under load using a single local memcached server is inferior 
to
   what you get with a tmpfs, although this can be probably be improved by batching
   requests and other techniques.
3. The size of objects sent to memcached is limited to about 1M, so images 
larger
   than this cannot be optimized at all in the current revision.  To address this
   we may need to still enable the file-cache for large objects.

Original comment by jmara...@google.com on 24 Jul 2012 at 1:36

  • Added labels: Type-Enhancement
  • Removed labels: Type-Defect
@GoogleCodeExporter

This comment has been minimized.

GoogleCodeExporter commented Apr 6, 2015

http://code.google.com/p/modpagespeed/source/detail?r=1723 gets memcached 
system tests working in the Prefork MPM (previous testing was in Worker).

Original comment by jmara...@google.com on 25 Jul 2012 at 8:11

@GoogleCodeExporter

This comment has been minimized.

GoogleCodeExporter commented Apr 6, 2015

Progress report.  As of 
http://code.google.com/p/modpagespeed/source/detail?r=1777 I am reasonably 
confident that this implementation is correct.  Prior to that change there was 
a small chance of incorrect data on lookup due to md5 collision; now we store 
the key with the data and use that to verify.

It could use more tuning but is not too bad.  In my loadtests I was thinking it 
was performing worse than tmpfs but that was because the default memcached max 
size is 64Mb and we had way more data than that in our load-test scripts.  I 
bumped up the capacity to 1G by running "memcached -m 1000" and now it performs 
almost as well as tmpfs, but provide the benefit of  network scalability.

We don't have doc published yet, but if you build from trunk and put in 

   ModPagespeedMemcachedServers host1:port1,host2:port2,...

then you can give this a try.


Still to come:  proper doc, and try to get more performance using multiget.

Original comment by jmara...@google.com on 11 Aug 2012 at 3:03

@GoogleCodeExporter

This comment has been minimized.

GoogleCodeExporter commented Apr 6, 2015

To fully support memcached we need to solve Issue 488, which observes that with 
ModPagespeedLoadFromFile we are storing local filesystem timestamps into our 
metadata cache.

Original comment by jmara...@google.com on 5 Sep 2012 at 1:31

  • Now blocked on: #488
@GoogleCodeExporter

This comment has been minimized.

GoogleCodeExporter commented Apr 6, 2015

Original comment by jmara...@google.com on 12 Oct 2012 at 1:24

  • Now blocked on: #522
@GoogleCodeExporter

This comment has been minimized.

GoogleCodeExporter commented Apr 6, 2015

Original comment by j...@google.com on 26 Oct 2012 at 4:58

  • Added labels: Milestone-v23
@GoogleCodeExporter

This comment has been minimized.

GoogleCodeExporter commented Apr 6, 2015

memcached support is now load-tested, failure-tested, and fully integrated into 
the trunk.

For any & all listening, please build from trunk & give it a try!

Original comment by jmara...@google.com on 1 Nov 2012 at 6:04

  • Added labels: release-note
@GoogleCodeExporter

This comment has been minimized.

GoogleCodeExporter commented Apr 6, 2015

Original comment by jmara...@google.com on 1 Nov 2012 at 6:04

  • Changed state: Fixed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment