Changing SERVER_MAX_VALUE_SIZE doesn't work

[alexf101]

Apparently, changing the value of server_max_value_length in the memcache module does NOT change the value inherited by the clients.

From looking at the source, I can not for the life of me figure out why!

This example was run on Ubuntu 13.04, and I've also observed it on a remote machine running Ubuntu 12.04 (on Amazon cloud).

Python 2.7.4 (default, Apr 19 2013, 18:28:01) 
[GCC 4.7.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import memcache
>>> memcache.SERVER_MAX_VALUE_LENGTH
1048576
>>> memcache.SERVER_MAX_VALUE_LENGTH *= 10
>>> memcache.SERVER_MAX_VALUE_LENGTH
10485760
>>> c = memcache.Client(['localhost'])
>>> c.server_max_value_length
1048576
>>> memcache.SERVER_MAX_VALUE_LENGTH
10485760

[linsomniac]

As the name implies, thi sis not a CLIENT setting, it is a SERVER setting. You have to rebuild the server and then change this value to match the server.

[alexf101]

No, I already did that - the memcached part works just fine (tested in bash). The problem is in the python memcache code.

By the way, in memcache versions 1.4.2 the max item length is a user-configurable parameter - it doesn't need to be recompiled.

[linsomniac]

I don't know then, I've had reports of others successfully using it... Perhaps it cannot be modified at runtime and has to be modified in the file before importing? Looking at the code, I think it might take effect at import time, not at run time. Can you verify that?

[alexf101]

I'm not sure what you mean... If I modify the parameter in the file with a text editor before importing it, then yes, it works.

If I run the code as in the example I posted, it doesn't.

If the server_max_value_length is passed to the client explicitly, it does work.

I would've expected that as long as the parameter was changed before the client was instantiated, it would work. I guess it doesn't really matter, except that it was confusing for debugging.

[linsomniac]

It is documented to work that way, but the code doesn't implement that. I'm going to see how hard it would be to make it happen. Probably won't be too bad.

[linsomniac]

I've pushed a change for this, can you please test to see if this works for you? If it does, I'll include it in the next version. Thanks!

[alexf101]

Works great!

Thank you.

I've copied a summary of my test session below.

$ echo 'stats settings' | nc localhost 11211 | grep 'item_size'
STAT item_size_max 104857600
$ python

Python 2.7.4 (default, Apr 19 2013, 18:28:01) 
[GCC 4.7.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from sys import getsizeof
>>> a = 'y'*104857600
>>> getsizeof(a)
104857637
....
>>> a = 'y'*104857563
>>> getsizeof(a)
104857600
>>> import memcache
>>> memcache.SERVER_MAX_VALUE_LENGTH
1048576
>>> memcache.SERVER_MAX_VALUE_LENGTH = 104857600
>>> memcache.SERVER_MAX_VALUE_LENGTH
104857600
>>> client = memcache.Client(['localhost'])
>>> client.server_max_value_length
104857600
>>> client.set('key', a)
False
>>> a = 'y'*104857500
>>> client.set('key', a)
True
...
>>> a = 'y'*104857524
>>> client.set('key', a)
True

So the value is propagated correctly, and I'm satisfied that it's actually able to store values up to within 80 bytes of the stated limit. Even then, it's not the Python code that's choking, it's the memcached backend's overheads.

Thanks again for the insta-fix!

[linsomniac]

Thanks for testing it, I'll push that out in the next release.