Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Loading…

prefixed proxies #1

Closed
wants to merge 3 commits into from

2 participants

@egradman

This works great for my purposes, hopefully you find it useful as well. I use redis to record object attributes using a standard prefix; that is, my object Person(id=100) has various data structures in redis ("person:100:name", "person:100:entries").

For reasons of object orientation, I want to be able to do:

p = Proxy()
class Person(object):
  def __init__(self, id, name):
    ...
    self.pp = p.prefixed('person:%d:' % id)
    p['name'] = name        # stored as 'person::name' 

The pull request contains code that lets me do exactly that.

FWIW, I plan to use this in a sneakier way, inspired by sqlalchemy: I define class properties that automagically use the underlying redis backing store. That allows me to do something like this:

class Person(object, RedisMixin):
  name = RedisProperty('name')

  def __init__(self, id, name):
    self.id = id
    self.prefix = 'person:%d:' % self.id
    self.name = name  # name is automatically backed by redis

Code to do that isn't in this pull request because I haven't finished it yet.

redish is really useful, btw! I'm also a happy user of celery.

@atl
Collaborator

Hey, excellent stuff. It looks like you're thinking along the same lines I was.

So before pulling this, I ask that you take a look at the experiments I did in my own fork (especially as regards keyspaces). I kept it over in my own space for a while to see how it worked in my own projects, and it's been working well.

http://github.com/atl/redish/commits/master/redish/proxy.py

@atl
Collaborator

Also, I wrote up the keyspaces (which in my case are formatstrings instead of prefixes) in the docs directory:

http://github.com/atl/redish/blob/master/docs/proxy.rst

I expose a few variations, accounting for different ways people might use it. I'm still in the process of discovering some patterns myself, which is why I hadn't pulled it into ask's repository just yet.

@egradman

atl, I think your format string thing is right on. I've actually now changed my "prefixed" to "namespaced," which emulates the behavior of keyspaces. I prefer the "class abstraction" to the "named keyspace" abstraction, but your solution does that too, and is definitely a superset of mine.

I've added two commits to my fork:

The first implements namespaced instead of prefixed.

The second is all contained in "redish/mixin.py" It lets you declare object properties that are automatically backed by redis namespaced objects. Its just a couple lines of code, but I think its a really powerful abstraction. Its totally trivial to reimplement this using keyspaces instead.

My prefixed/namespaced code represents just a couple of minutes of poking around in a codebase I'd never touched before. I'm sure keyspace is more robust. But please do pull one of them into ask's tree! I'll implement mixin.py using whichever flavor you prefer.

I'm really psyched to have found this project, and I'd love to hack on it some more.

@atl
Collaborator

egradman, I'm really excited that someone else is 1) looking at proxy.py and 2) was thinking along the same lines for layering namespaces on top of it.

I've just merged the branch that I wrote, am familiar with, and have tested. I'm very eager to see what you can build on top of that. I'm not sure what ask's policy is on adding contributors, but he was generous in adding me, I hope he could do the same for you.

@egradman

Sweet! I merged and modified RedisMixin to use keyspaces. It was little more than s/name/key. Our code was remarkably similar.

Let me flesh this Mixin stuff out a little bit, and I'll send you a pull request in a few days after I've tested and documented things.

This issue was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
This page is out of date. Refresh to see the latest.
Showing with 109 additions and 0 deletions.
  1. +69 −0 redish/mixin.py
  2. +40 −0 redish/proxy.py
View
69 redish/mixin.py
@@ -0,0 +1,69 @@
+import redish.proxy
+
+def Proxied(key):
+ """generate a property, which is a pair of closures over the named key"""
+ def get_proxied(instance, key=key):
+ return instance._nsproxy[key]
+
+ def set_proxied(instance, val, key=key):
+ instance._nsproxy[key] = val
+
+ return property(fget=get_proxied, fset=set_proxied)
+
+class RedisMixin(object):
+ """A mixin that allows your class to use redish namespaced proxies as
+ a backing store for member variables. To use, define a __namespace__
+ lambda function that returns the namespace for this object, and store the
+ master proxy in self._proxy
+
+
+ Example:
+ class Test(RedisMixin):
+ __namespace__ = lambda self: "test:%d:%%s" % self.id
+ foo = Proxied('foo')
+ bar = Proxied('bar')
+ baz = Proxied('baz')
+
+ def __init__(self, proxy, id):
+ self._proxy = proxy
+ self.id = id
+
+ proxy = redish.proxy.Proxy(db=4)
+ t = Test(proxy, 1)
+ t.foo = 1
+ t.bar = ['hello', 'goodbye']
+ t.baz = dict(a=1, b=2)
+
+ proxy.get('test:1:foo') -> 1
+ proxy.lrange('test:1:bar', 0, -1) -> ['hello', 'goodbye']
+ proxy.hgetall('test:1:baz') -> {'a': '1', 'b': '2'}
+ """
+ @property
+ def _nsproxy(self):
+ """when attribute access occurs, this method "memoizes" the namespaced
+ proxy, on the assumption that by the time you start messing about
+ with properties, your __namespace__ method is callable
+ """
+ self.__nsproxy = getattr(self, '__nsproxy', None) or self._proxy.namespaced(self.__namespace__())
+ return self.__nsproxy
+
+class Test(RedisMixin):
+ __namespace__ = lambda self: "test:%d:%%s" % self.id
+ foo = Proxied('foo')
+ bar = Proxied('bar')
+ baz = Proxied('baz')
+
+ def __init__(self, proxy, id):
+ self._proxy = proxy
+ self.id = id
+
+if __name__=='__main__':
+ proxy = redish.proxy.Proxy(db=4)
+ t = Test(proxy, 1)
+ t.foo = 1
+ t.bar = ['hello', 'goodbye']
+ t.baz = dict(a=1, b=2)
+
+ print proxy.get('test:1:foo')
+ print proxy.lrange('test:1:bar', 0, -1)
+ print proxy.hgetall('test:1:baz')
View
40 redish/proxy.py
@@ -126,4 +126,44 @@ def __delitem__(self, k):
def multikey(self, pattern):
for p in self.keys(pattern):
yield self[p]
+
+ def namespaced(self, formatstring):
+ return Namespaced(self, formatstring)
+
+class Namespaced(object):
+ """Decorates a Proxy object, such that any key passed to the underlying
+ proxy automatically is interpolated into the format string in string context
+ Note: this only works on proxy calls. it does not alter the underlying
+ behavior of Redis object methods. For example, the "keys()" method is
+ not namespaced
+
+ Example:
+ p = Proxy()
+ pp = Proxy.namespaced('foo:1:%s')
+ pp['bar'] = 1
+ pp.keys('foo:1:*')
+ -> ['foo:1:bar']
+ """
+
+ def __init__(self, proxy, formatstring):
+ """record the underlying proxy, so we can reuse proxies"""
+ self.proxy = proxy
+ self.formatstring = formatstring
+
+ # interpolate key into format string
+ format = lambda self, key: self.formatstring % key
+
+ __getitem__ = lambda self, key: self.proxy.__getitem__(self.format(key))
+ __setitem__ = lambda self, key, value: self.proxy.__setitem__(self.format(key), value)
+ __contains__ = lambda self, key: self.proxy.__contains__(self.format(key))
+ __delitem = lambda self, key: self.proxy.__delitem__(self.format(key))
+ def multikey(self, pattern):
+ for p in self.proxy.keys(pattern):
+ yield self.proxy[self.format(p)]
+
+ def __getattr__(self, attr):
+ """the prefixed object should behave like the proxy object, so delegate
+ any other attribute accesses to the underlying proxy
+ """
+ return getattr(self.proxy, attr)
Something went wrong with that request. Please try again.