Discrepancy between pylibmc and python-memcached in storing bools

[hyperair]

While comparing python-memcached and pylibmc behaviours, I found that python-memcached was able to store and retrieve bool values as bools, while pylibmc would store them as integers being 1 or 0. It looks like python-memcached pickles bools but pylibmc stores them as integers.

[lericson]

Did it cause errors? - Ludvig

…

On 12 Nov 2018, at 10:29, Chow Loong Jin ***@***.***> wrote: While comparing python-memcached and pylibmc behaviours, I found that python-memcached was able to store and retrieve bool values as bools, while pylibmc would store them as integers being 1 or 0. It looks like python-memcached pickles bools but pylibmc stores them as integers. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

[hyperair]

On Mon, Nov 12, 2018 at 02:58:27PM +0000, Ludvig Ericson wrote: Did it cause errors?

Well, storing and retrieving a bool using pylibmc converts bools into ints, but doing the same in python-memcached returns a bool. The latter behaviour seems more correct.

…

-- Kind regards, Loong Jin

[lericson]

I see. Maybe we should reintroduce bool storage but as int-with-extra flag. Going through the pickle machinery for something so simple seems... counterproductive, if you see my reasoning. - Ludvig

…

On 14 Nov 2018, at 19:33, Chow Loong Jin ***@***.***> wrote: On Mon, Nov 12, 2018 at 02:58:27PM +0000, Ludvig Ericson wrote: > Did it cause errors? Well, storing and retrieving a bool using pylibmc converts bools into ints, but doing the same in python-memcached returns a bool. The latter behaviour seems more correct. -- Kind regards, Loong Jin — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.

[hyperair]

On Wed, Nov 14, 2018 at 12:54:47PM -0800, Ludvig Ericson wrote: I see. Maybe we should reintroduce bool storage but as int-with-extra flag. Going through the pickle machinery for something so simple seems... counterproductive, if you see my reasoning.

Yes, I see your reasoning. Perhaps you could reintroduce FLAG_BOOL as `1 << 31` to hopefully avoid conflicts with any new flags python-memcached introduces in the future.

…

-- Kind regards, Loong Jin

[sergeief]

We've encountered this after switching from python-memcached. In our case, uncached value returns as True and cached as 1, which caused a bug in the code. Is there workaround available?

[hyperair]

You can subclass pylibmc.Client and override the serialize method to match python-memcached's behaviour

import pickle
import pylibmc


class Client(pylibmc.Client):
    def serialize(self, value):
        """
        Overridden serialize method. See pylibmc.client.Client.serialize for
        more info
        """

        # Ensure that bools are pickled to match python-memcached behaviour.
        # Remove this when https://github.com/lericson/pylibmc/issues/245 is
        # fixed.
        if isinstance(value, bool):
            return pickle.dumps(value), 1
        else:
            return super().serialize(value)