Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
85 commits
Select commit Hold shift + click to select a range
e5faa12
replace pylibc to pymemcache
Apr 10, 2017
532d28a
add ignore_exc parameter
Apr 10, 2017
2ecc21c
rename module name
Apr 11, 2017
e1f0ba1
rename readme
Apr 11, 2017
bea9225
add options
Apr 11, 2017
8c3c15a
fix README path
Apr 11, 2017
cc8e30e
rm png
Apr 11, 2017
64ec2a1
memo self._client
Apr 11, 2017
e90da03
rm options from example
Apr 11, 2017
0dd0160
rm print
Apr 11, 2017
f630816
Update LICENSE
opapy Apr 11, 2017
9edd161
fix README
Apr 11, 2017
0715186
fix markdown
Apr 11, 2017
6255a8b
fix manifest.in
Apr 11, 2017
eabbfc1
Merge pull request #1 from uncovertruth/feature/pymemcache
Surgo Apr 11, 2017
027b83d
Re-generate via gibo
Surgo Apr 11, 2017
aab80f7
Set up Travis-CI
Surgo Apr 11, 2017
abb81fe
Remove support for Django 1.7
Surgo Apr 11, 2017
4263c21
Merge pull request #2 from uncovertruth/feature/enable_ci
opapy Apr 11, 2017
4c90b99
Add dependencies for test
Surgo Apr 11, 2017
99d800d
Update setup.py - fix license etc.
Surgo Apr 11, 2017
0be478d
Fix flake8 bugs
Surgo Apr 11, 2017
e419a90
Fix flake8, check-manifest and isort bugs
Surgo Apr 11, 2017
4d2685d
Use rest format
Surgo Apr 11, 2017
cb60e0a
Add codecov badge
Surgo Apr 11, 2017
156630c
Enable include_trailing_comma option
Surgo Apr 11, 2017
242b83c
Test only project files
Surgo Apr 11, 2017
f314e6c
Fix wrong commands
Surgo Apr 11, 2017
87764a3
Fix rest format
Surgo Apr 11, 2017
9546134
Merge pull request #3 from uncovertruth/fix/ci_passed
opapy Apr 11, 2017
eb00cfe
Add deployment for pypi
Surgo Apr 12, 2017
3f95c37
Add pypi badge
Surgo Apr 12, 2017
8d1df6f
Merge pull request #4 from uncovertruth/feature/release_pypi
opapy Apr 12, 2017
dc9a1d2
Deploy only specific env
Surgo Apr 12, 2017
2268a33
Merge pull request #5 from uncovertruth/fix/deploy
heavenshell Apr 12, 2017
c68373a
Fix wrong env name
Surgo Apr 12, 2017
4f04a7f
Fix wrong requirements file path
Surgo Apr 12, 2017
651d1bd
Run only specific env
Surgo Apr 12, 2017
7cb7c50
Fix wrong type
Surgo Apr 12, 2017
a7cd620
Apply patch for django<1.11
Surgo Apr 12, 2017
feda455
Codecov in specific version
Surgo Apr 12, 2017
ea29cce
Merge pull request #6 from uncovertruth/fix/env_name
opapy Apr 12, 2017
f01f28b
Fix typo
Surgo Apr 12, 2017
6fb2ba4
Skip upload docs
Surgo Apr 12, 2017
1e19168
Add tox env for pypy pypy3
Surgo Apr 12, 2017
4976e94
Bump version
Surgo Apr 12, 2017
76e71d1
Set codecov token
Surgo Apr 12, 2017
42e0ecc
Allow failure pypy3
Surgo Apr 12, 2017
2dbb39f
Fix worng env
Surgo Apr 12, 2017
d25f3ce
Merge pull request #7 from uncovertruth/fix/typo
opapy Apr 12, 2017
562fbfc
ignore error when get cluster info
Apr 12, 2017
7db72bd
set empty server when get cluster info
Apr 12, 2017
e095476
add timeout params
Apr 12, 2017
33d8dbe
edit README
Apr 12, 2017
0d89752
fix test
Apr 13, 2017
12b2276
set socket._GLOBAL_DEFAULT_TIMEOUT as default value for telnet
Apr 13, 2017
f4e8c7c
fix pep8
Apr 13, 2017
9e199ba
Merge pull request #8 from uncovertruth/feature/ignore_cluster_info_e…
Surgo Apr 13, 2017
2e397b9
bump version
Apr 13, 2017
13a6125
Merge pull request #9 from uncovertruth/feature/0.0.3
Surgo Apr 17, 2017
87ee7c4
add close method
May 10, 2017
2ca478a
replace serializer
May 10, 2017
08e38f6
fix test
May 10, 2017
45158a1
fix test
May 10, 2017
306ccfd
Merge pull request #10 from uncovertruth/bug/disconnect_all
Surgo May 10, 2017
e986e4a
bump version
May 11, 2017
0bcfa72
Merge pull request #11 from uncovertruth/feature/0.0.5
Surgo May 11, 2017
7fee4da
add ignore_cluster_errors params
May 15, 2017
bca3fa6
fix test
May 15, 2017
ebd2a61
fix flake8
May 15, 2017
f0899a0
Merge pull request #12 from uncovertruth/feature/ignore_cluster_errors
Surgo May 15, 2017
665651c
bump version
May 15, 2017
febe5f4
Merge pull request #13 from uncovertruth/feature/0.0.6
Surgo May 15, 2017
2b3b10d
Pin check-manifest to latest version 0.35
pyup-bot Jun 15, 2017
319f580
Pin coverage to latest version 4.4.1
pyup-bot Jun 15, 2017
2fb3be6
Pin flake8 to latest version 3.3.0
pyup-bot Jun 15, 2017
6b047d3
Pin isort to latest version 4.2.15
pyup-bot Jun 15, 2017
843df58
Pin mock to latest version 2.0.0
pyup-bot Jun 15, 2017
36070e9
Pin nose to latest version 1.3.7
pyup-bot Jun 15, 2017
dd802c4
Pin pymemcache to latest version 1.4.3
pyup-bot Jun 15, 2017
dd29490
Pin readme-renderer to latest version 17.2
pyup-bot Jun 15, 2017
0ff5328
Merge pull request #14 from uncovertruth/pyup-initial-update
Surgo Jun 15, 2017
844b190
Update flake8 from 3.3.0 to 3.4.1 (#16)
pyup-bot Jul 31, 2017
39f09c4
Update flake8 from 3.4.1 to 3.5.0 (#17)
pyup-bot Oct 24, 2017
f432a5f
Fix compatibly with Django-Cachalot
heavenshell Oct 27, 2017
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
104 changes: 84 additions & 20 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,37 +1,101 @@
### https://raw.github.com/github/gitignore/f57304e9762876ae4c9b02867ed0cb887316387e/python.gitignore

# Byte-compiled / optimized / DLL files
__pycache__/
*.py[cod]
*$py.class

# C extensions
*.so

# Packages
*.egg
*.egg-info
dist
build
eggs
parts
bin
var
sdist
develop-eggs
# Distribution / packaging
.Python
env/
build/
develop-eggs/
dist/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
wheels/
*.egg-info/
.installed.cfg
lib
lib64
__pycache__
*.egg

# PyInstaller
# Usually these files are written by a python script from a template
# before PyInstaller builds the exe, so as to inject date/other infos into it.
*.manifest
*.spec

# Installer logs
pip-log.txt
pip-delete-this-directory.txt

# Unit test / coverage reports
htmlcov/
.tox/
.coverage
.tox
.coverage.*
.cache
nosetests.xml
coverage.xml
*,cover
.hypothesis/

# Translations
*.mo
*.pot

# Django stuff:
*.log
local_settings.py

# Flask stuff:
instance/
.webassets-cache

# Scrapy stuff:
.scrapy

# Sphinx documentation
docs/_build/

# PyBuilder
target/

# Jupyter Notebook
.ipynb_checkpoints

# pyenv
.python-version

# celery beat schedule file
celerybeat-schedule

# SageMath parsed files
*.sage.py

# dotenv
.env

# virtualenv
.venv
venv/
ENV/

# Spyder project settings
.spyderproject

# Rope project settings
.ropeproject

# mkdocs documentation
/site


# Mr Developer
.mr.developer.cfg
.project
.pydevproject
.idea
69 changes: 69 additions & 0 deletions .travis.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
sudo: false
language: python
cache: pip
matrix:
fast_finish: true
include:
- python: 2.7
env: TOX_ENV=py27-dj18
- python: 3.3
env: TOX_ENV=py33-dj18
- python: 3.4
env: TOX_ENV=py34-dj18
- python: 3.5
env: TOX_ENV=py34-dj18
- python: 2.7
env: TOX_ENV=py27-dj19
- python: 3.4
env: TOX_ENV=py34-dj19
- python: 3.5
env: TOX_ENV=py35-dj19
- python: 2.7
env: TOX_ENV=py27-dj110
- python: 3.4
env: TOX_ENV=py34-dj110
- python: 3.5
env: TOX_ENV=py35-dj110
- python: 2.7
env: TOX_ENV=py27-dj111
- python: pypy
env: TOX_ENV=pypy-dj111
- python: 3.4
env: TOX_ENV=py34-dj111
- python: 3.5
env: TOX_ENV=py35-dj111
- python: 3.6
env: TOX_ENV=py36-dj111
- python: pypy3
env: TOX_ENV=pypy3-dj111
- python: 3.5
env: TOX_ENV=py35-djdev
- python: 3.6
env: TOX_ENV=py36-djdev
- python: 3.6
env: TOX_ENV=flake8
- python: 3.6
env: TOX_ENV=isort
- python: 3.6
env: TOX_ENV=readme
- python: 3.6
env: TOX_ENV=check-manifest
allow_failures:
- env: TOX_ENV=py35-djdev
- env: TOX_ENV=py36-djdev
- env: TOX_ENV=pypy3-dj111
install:
- pip install tox
script:
- tox -e "$TOX_ENV"
deploy:
provider: pypi
user: uncovertruth
password:
secure: FtNY+8/F79RYpJzfusYBC9HFfDWlevUbS4J5TECm8tyMxAxeHMpLOaL4dd/wPkRkuctUM5LyXux5rpVQTfMA4Dvjdufc7BqexCt056GznJPFRjHW19ExeguTHXW810RIVrFmo5Dk1/AlMpGEeiL3OXoI041h7/vRZe6QWP6P0NCz9UrkZ/Qc9JDZqkXBmhiwvt4P3gu5sbxaiM4IqxhSQ/4DoI87SQlD3nUfgRT2TkqN/Jx0ME0H7l+40HeqZvVq6sh9Muc5/XhDEx5HWt9BSgymeXXG6mT3sewBfinJlQ5/1Rrm+IHa48m/3mLJkfYEs15BQCizvDc6/dRfdtCc/9jYl3e67yEM6Akn+52pTOU4Oa727ZUpAYNCzfY+pHc3E9oI0YFuOl+WYNo5bWUh3IlWAl/eXDGWOvRMF4waUbyjUMiZ+C4pRmRKznYZB9xgbPxHDlR6CMn8P5F5j3ZVbW0BxUEjlDFcDT65gflBhKS1eWTOQRMobR+yjQKlYMlIeCgITbuEYbn2GbL84aN42U910rSQ4SryzqjaOIyxtZG53cGDw2NMm4q7bIwRkj6gbvliA36HIFaRQrI/wtrmr3guy4raPRpEGi7OhFoaPxaqkEs+yF55MEafNM4eJc9swM/K/mbN6FFKIWoWVyXUCUDIWvcPYLWPR/kLidWCNzw=
distributions: sdist bdist_wheel
skip_upload_docs: true
on:
repo: uncovertruth/django-elastipymemcache
tags: true
condition: $TOX_ENV = py36-dj111
1 change: 1 addition & 0 deletions LICENSE
Original file line number Diff line number Diff line change
Expand Up @@ -18,3 +18,4 @@ FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

7 changes: 4 additions & 3 deletions MANIFEST.in
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
include README.rst
include MANIFEST.in
include setup.py
graft django_elastipymemcache
graft tests
include README.rst AUTHORS LICENSE CHANGELOG.rst setup.py setup.cfg requirements.txt tox.ini
global-exclude __pycache__ *.pyc
136 changes: 35 additions & 101 deletions README.rst
Original file line number Diff line number Diff line change
@@ -1,31 +1,44 @@
Amazon ElastiCache backend for Django
=====================================
=======================
django-elastipymemcache
=======================

:Info: Simple Django cache backend for Amazon ElastiCache (memcached based).
:Author: UNCOVER TRUTH Inc. <develop@uncovertruth.co.jp>
:Copyright: © UNCOVER TRUTH Inc.
:Date: 2017-04-11
:Version: 0.0.6

.. index: README
.. image:: https://travis-ci.org/uncovertruth/django-elastipymemcache.svg?branch=master
:target: https://travis-ci.org/uncovertruth/django-elastipymemcache
.. image:: https://codecov.io/gh/uncovertruth/django-elastipymemcache/branch/master/graph/badge.svg
:target: https://codecov.io/gh/uncovertruth/django-elastipymemcache
.. image:: https://requires.io/github/uncovertruth/django-elastipymemcache/requirements.svg?branch=master
:target: https://requires.io/github/uncovertruth/django-elastipymemcache/requirements/?branch=master
.. image:: https://badge.fury.io/py/django-elastipymemcache.svg
:target: https://badge.fury.io/py/django-elastipymemcache

Purpose
-------

Simple Django cache backend for Amazon ElastiCache (memcached based). It uses
`pylibmc <http://github.com/lericson/pylibmc>`_ and sets up a connection to each
`pymemcache <https://github.com/pinterest/pymemcache>`_ and sets up a connection to each
node in the cluster using
`auto discovery <http://docs.aws.amazon.com/AmazonElastiCache/latest/UserGuide/AutoDiscovery.html>`_.

Originally forked from `django-elasticache <https://github.com/gusdan/django-elasticache>`_.

Requirements
------------

* pylibmc
* Django 1.5+.

It was written and tested on Python 2.7 and 3.4.
* pymemcache
* Django>=1.8

Installation
------------

Get it from `pypi <http://pypi.python.org/pypi/django-elasticache>`_::

pip install django-elasticache

or `github <http://github.com/gusdan/django-elasticache>`_::

pip install -e git://github.com/gusdan/django-elasticache.git#egg=django-elasticache
Get it from `pypi <http://pypi.python.org/pypi/django-elastipymemcache>`_::

pip install django-elastipymemcache

Usage
-----
Expand All @@ -34,95 +47,16 @@ Your cache backend should look something like this::

CACHES = {
'default': {
'BACKEND': 'django_elasticache.memcached.ElastiCache',
'LOCATION': 'cache-c.draaaf.cfg.use1.cache.amazonaws.com:11211',
'OPTIONS' {
'IGNORE_CLUSTER_ERRORS': [True,False],
},
}
}

By the first call to cache it connects to cluster (using ``LOCATION`` param),
gets list of all nodes and setup pylibmc client using full
list of nodes. As result your cache will work with all nodes in cluster and
automatically detect new nodes in cluster. List of nodes are stored in class-level
cached, so any changes in cluster take affect only after restart of working process.
But if you're using gunicorn or mod_wsgi you usually have max_request settings which
restart process after some count of processed requests, so auto discovery will work
fine.

The ``IGNORE_CLUSTER_ERRORS`` option is useful when ``LOCATION`` doesn't have support
for ``config get cluster``. When set to ``True``, and ``config get cluster`` fails,
it returns a list of a single node with the same endpoint supplied to ``LOCATION``.

Django-elasticache changes default pylibmc params to increase performance.

Another solutions
-----------------

ElastiCache provides memcached interface so there are three solution of using it:

1. Memcached configured with location = Configuration Endpoint
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

In this case your application
will randomly connect to nodes in cluster and cache will be used with not optimal
way. At some moment you will be connected to first node and set item. Minute later
you will be connected to another node and will not able to get this item.

::

CACHES = {
'default': {
'BACKEND': 'django.core.cache.backends.memcached.PyLibMCCache',
'LOCATION': 'cache.gasdbp.cfg.use1.cache.amazonaws.com:11211',
}
}


2. Memcached configured with all nodes
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

It will work fine, memcache client will
separate items between all nodes and will balance loading on client side. You will
have problems only after adding new nodes or delete old nodes. In this case you should
add new nodes manually and don't forget update your app after all changes on AWS.

::

CACHES = {
'default': {
'BACKEND': 'django.core.cache.backends.memcached.PyLibMCCache',
'LOCATION': [
'cache.gqasdbp.0001.use1.cache.amazonaws.com:11211',
'cache.gqasdbp.0002.use1.cache.amazonaws.com:11211',
]
}
}


3. Use django-elasticache
~~~~~~~~~~~~~~~~~~~~~~~~~

It will connect to cluster and retrieve ip addresses
of all nodes and configure memcached to use all nodes.

::

CACHES = {
'default': {
'BACKEND': 'django_elasticache.memcached.ElastiCache',
'LOCATION': 'cache-c.draaaf.cfg.use1.cache.amazonaws.com:11211',
'BACKEND': 'django_elastipymemcache.memcached.ElastiPyMemCache',
'LOCATION': '[configuration endpoint]:11211',
'OPTIONS': {
'cluster_timeout': 1, # its used when get cluster info
'ignore_exc': True, # pymemcache Client params
'ignore_cluster_errors': True, # ignore get cluster info error
}
}
}


Difference between setup with nodes list (django-elasticache) and
connection to only one configuration Endpoint (using dns routing) you can see on
this graph:

.. image:: https://raw.github.com/gusdan/django-elasticache/master/docs/images/get%20operation%20in%20cluster.png

Testing
-------

Expand Down
Loading