Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add hashing client #59

Merged
merged 23 commits into from
Jul 17, 2015
Merged
Show file tree
Hide file tree
Changes from 8 commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
f394d42
Initial implementation of the hash client
sontek Jun 20, 2015
babb8c0
This adds failover support.
sontek Jun 20, 2015
8879147
Add bringing dead servers back into rotation
sontek Jun 20, 2015
2470508
Implemented the other functions and added tests
sontek Jun 21, 2015
9892226
Fix tests + pep8
sontek Jun 21, 2015
236643a
drop support for pypy3 for now
sontek Jun 21, 2015
abf4bd8
Fix API to be backwards compatible
sontek Jun 21, 2015
381715f
Added unit tests
sontek Jun 21, 2015
f43cf74
Merge branch 'master' of github.com:pinterest/pymemcache into add_has…
sontek Jul 2, 2015
a935261
Code review clean ups and doc fixes
sontek Jul 2, 2015
cff5abf
Remove dependency on clandestined and use pure python murmur3
sontek Jul 2, 2015
8b19853
Fix flake8 issues
sontek Jul 2, 2015
d6fc5ce
Merge pull request #1 from sontek/make_murmur3_optional
sontek Jul 2, 2015
b7c02a3
Make ignoring exceptions optional
sontek Jul 2, 2015
deb2cbf
flake8 fixes
sontek Jul 2, 2015
75bba31
Merge pull request #2 from sontek/make_default_values_configurable
sontek Jul 16, 2015
6ef4ab0
Merge branch 'master' of github.com:pinterest/pymemcache into add_has…
sontek Jul 16, 2015
88b9fc5
Updated docs
sontek Jul 16, 2015
aca08b5
Added pypy3 support back
sontek Jul 16, 2015
fe53429
Import PooledClient in its old location for backwards compat
sontek Jul 16, 2015
917167a
Merge branch 'master' of github.com:pinterest/pymemcache into add_has…
sontek Jul 16, 2015
a844d20
Added ChangeLog entries for the unreleased code
sontek Jul 16, 2015
ed85bcf
Use restructured text instead of markdown for README
sontek Jul 16, 2015
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
1 change: 0 additions & 1 deletion .travis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,6 @@ env:
- TOXENV=py33
- TOXENV=py34
- TOXENV=pypy
- TOXENV=pypy3

install:
- travis_retry pip install tox
Expand Down
30 changes: 30 additions & 0 deletions docs/apidoc/pymemcache.client.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
pymemcache.client package
=========================

Submodules
----------

pymemcache.client.base module
-----------------------------

.. automodule:: pymemcache.client.base
:members:
:undoc-members:
:show-inheritance:

pymemcache.client.hash module
-----------------------------

.. automodule:: pymemcache.client.hash
:members:
:undoc-members:
:show-inheritance:


Module contents
---------------

.. automodule:: pymemcache.client
:members:
:undoc-members:
:show-inheritance:
21 changes: 19 additions & 2 deletions docs/getting_started.rst
Original file line number Diff line number Diff line change
Expand Up @@ -7,20 +7,37 @@ Basic Usage

.. code-block:: python

from pymemcache.client import Client
from pymemcache.client.base import Client

client = Client(('localhost', 11211))
client.set('some_key', 'some_value')
result = client.get('some_key')

Using a memcached cluster
-------------------------
This will use a consistent hashing algorithm to choose which server to
set/get the values from. It will also automatically rebalance depending
on if a server goes down.

.. code-block:: python

from pymemcache.client.hash import HashClient

client = HashClient([
('127.0.0.1', 11211),
('127.0.0.1', 11212)
])
client.set('some_key', 'some value')
result = client.get('some_key')


Serialization
--------------

.. code-block:: python

import json
from pymemcache.client import Client
from pymemcache.client.base import Client

def json_serializer(key, value):
if type(value) == str:
Expand Down
2 changes: 2 additions & 0 deletions pymemcache/client/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
# API Backwards compatibility
from pymemcache.client.base import Client # noqa
58 changes: 12 additions & 46 deletions pymemcache/client.py → pymemcache/client/base.py
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,15 @@

from pymemcache import pool

from pymemcache.exceptions import (
MemcacheClientError,
MemcacheUnknownCommandError,
MemcacheIllegalInputError,
MemcacheServerError,
MemcacheUnknownError,
MemcacheUnexpectedCloseError
)


RECV_SIZE = 4096
VALID_STORE_RESULTS = {
Expand Down Expand Up @@ -56,51 +65,9 @@
b'slab_automove': lambda value: int(value) != 0,
}


class MemcacheError(Exception):
"Base exception class"
pass


class MemcacheClientError(MemcacheError):
"""Raised when memcached fails to parse the arguments to a request, likely
due to a malformed key and/or value, a bug in this library, or a version
mismatch with memcached."""
pass


class MemcacheUnknownCommandError(MemcacheClientError):
"""Raised when memcached fails to parse a request, likely due to a bug in
this library or a version mismatch with memcached."""
pass


class MemcacheIllegalInputError(MemcacheClientError):
"""Raised when a key or value is not legal for Memcache (see the class docs
for Client for more details)."""
pass


class MemcacheServerError(MemcacheError):
"""Raised when memcached reports a failure while processing a request,
likely due to a bug or transient issue in memcached."""
pass


class MemcacheUnknownError(MemcacheError):
"""Raised when this library receives a response from memcached that it
cannot parse, likely due to a bug in this library or a version mismatch
with memcached."""
pass


class MemcacheUnexpectedCloseError(MemcacheServerError):
"Raised when the connection with memcached closes unexpectedly."
pass


# Common helper functions.


def _check_key(key, key_prefix=b''):
"""Checks key and add key_prefix."""
if isinstance(key, six.text_type):
Expand Down Expand Up @@ -131,7 +98,7 @@ class Client(object):
Values must have a __str__() method to convert themselves to a byte
string. Unicode objects can be a problem since str() on a Unicode object
will attempt to encode it as ASCII (which will fail if the value contains
code points larger than U+127). You can fix this will a serializer or by
code points larger than U+127). You can fix this with a serializer or by
just calling encode on the string (using UTF-8, for instance).

If you intend to use anything but str as a value, it is a good idea to use
Expand Down Expand Up @@ -656,7 +623,6 @@ def _fetch_cmd(self, name, keys, expect_cas):
while True:
buf, line = _readline(self.sock, buf)
self._raise_errors(line, name)

if line == b'END':
return result
elif line.startswith(b'VALUE'):
Expand Down Expand Up @@ -684,7 +650,7 @@ def _fetch_cmd(self, name, keys, expect_cas):
result[key] = value
else:
raise MemcacheUnknownError(line[:32])
except Exception:
except Exception as e:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we need the "as e" here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oops, added that during a debugging session, wasn't supposed to be included in the commit

self.close()
if self.ignore_exc:
return {}
Expand Down