SQL backend #27

Closed
eseifert opened this Issue Jan 27, 2013 · 13 comments

Projects

None yet

7 participants

@eseifert

It would be nice to have a storage backend for Cork that uses an SQL database. In addition to the actual backend code some changes would be necessary to define the backend that should be used for Cork.

I implemented a very raw version of an SQL backend based on SQLAlchemy: https://gist.github.com/4651239. It's not optimized or well tested at the moment, but it works.

At the moment the implementation uses an additional layer (SqlRowProxy, SqlDict, and SqlColumnDict) to emulate Python dictionaries for Cork's users, roles, or pending_registrations. The classes of this layer are used to translate dictionary access to SQLAlchemy calls.

Comments are very welcome.

@jcdyer
jcdyer commented Mar 3, 2013

I think more important than the creation of the SQL backend itself is allowing the user to pass in a custom Backend class when they create a new Cork() instance, so that additional backends can be created separate from the Cork project, whether it's based on SQL or MongoDB or unix passwd file or any other type of data store.

Something like:

class Cork(object):
    def __init__(self, backend=JsonBackend, **kwargs):
        self._store = self.backend(
            kwargs['directory'],
            kwargs['users_fname'],
            kwargs['role_fname'],
            kwargs['pending_reg_fname'],
            kwargs['initialize'])

And maybe expand the docs to describe the interface to the backend more clearly to help people who want to write their own.

@23min
Contributor
23min commented Mar 4, 2013

See https://github.com/23min/bottle-cork/blob/mongobackend-v1/cork/cork.py

I was basically forced to implement that in order to make mongo backend
pluggable.

On Sun, Mar 3, 2013 at 8:13 PM, Cliff Dyer notifications@github.com wrote:

I think more important than the creation of the SQL backend itself is
allowing the user to pass in a custom Backend class when they create a new
Cork() instance, so that additional backends can be created separate from
the Cork project, whether it's based on SQL or MongoDB or unix passwd file
or any other type of data store.

Something like:

class Cork(object):
def init(self, backend=JsonBackend, **kwargs):
self._store = self.backend(
kwargs['directory'],
kwargs['users_fname'],
kwargs['role_fname'],
kwargs['pending_reg_fname'],
kwargs['initialize'])

And maybe expand the docs to describe the interface to the backend more
clearly to help people who want to write their own.


Reply to this email directly or view it on GitHubhttps://github.com/FedericoCeratto/bottle-cork/issues/27#issuecomment-14352754
.

@FedericoCeratto
Owner

I did some experiments around building a more formal Backend interface and allowing configurable backends.
I'd like to hear you opinions on how users/roles/etc should be accessed and modified:

  1. Keep the current dictionary-like interface (e.g. users['tom']). This requires the backends to implement dictionary accessors.
  2. Remove the dictionaries and use explicit methods e.g. fetch_user(name='tom'), update_user(...), delete_user(...). This would require a lot of code (and unit tests) changes. This could make the backends a bit more readable.

Any strong feeling about the two options? Thanks

@jcdyer
jcdyer commented Mar 4, 2013

Ideally, it would be easy both to use the backend and to write a new backend. Maybe allow dictionary access like is currently done, but implement the accessors in a superclass to map Backend.get_user() and Backend.set_user() to Backend.user['foo']. Might require some trickery in using descriptors to make it work. I'll give it a little thought, and see if I can come up with a demo implementation of what I mean.

@lllama
lllama commented Mar 13, 2013

I'd like to vote that any SQL backend API does not add a requirement for SQLAlchemy or any other package. Backends can, of course, use these as they see fit but Cork shouldn't add this to its requirements.

@FedericoCeratto
Owner

Sounds reasonable. Each backend should import its own dependencies only if it's being used.
The backends should be declared as extras in http://peak.telecommunity.com/DevCenter/setuptools#declaring-extras-optional-features-with-their-own-dependencies

@jcdyer
jcdyer commented Mar 13, 2013

So I figured out a good way of having the backend be database aware, and still being able to get dict-style access on the attributes. I wrote it up here: Notes on the Descriptor Protocol. See the classes AttributeDict and AttributeDictObject for the implementation, and UserNotes for how to use it.

As far as dependency management, I would vote to create separate packages for backends that have external dependencies. So to use cork with SQLAlchemy, I would do something like:

$ pip install bottle-cork bottle-cork-sqlalchemy
... (bottle-cork-sqlalchemy has sqlalchemy (and bottle-cork?) as dependencies)

Then you would use it by doing

>>> import cork, cork_sqlalchemy
>>> auth = cork.Cork(backend=cork_sqlalchemy.SQLAlchemyBackend, ...)
@FedericoCeratto
Owner

A little wiki page to keep track of the backends:
https://github.com/FedericoCeratto/bottle-cork/wiki/Backends

This was referenced Mar 22, 2013
@dineshbvadhia

Concur with IIIama to support a pure sqlite backend without SQLAlchemy etc. Preference would be to prioritize sqlite as it can easily solve many db problems and is part of the Python standard library. Best.

@eseifert

It's great to have configurable backends now. I updated my experimental sqlalchemy backend (https://gist.github.com/eseifert/4651239) to implement the new interface.

I also suggest that a pop method gets added to base_backend.Table because it's required for the backends interface.

@FedericoCeratto
Owner

eseifert: can I start integrating your backend? If so, can I have your name and email address for the licensing?

@pbkdf3
pbkdf3 commented Mar 28, 2013

I would like to implement an ldap backend. However, this doesn't fit the current backend model very well.
In particular, I propose the api be changed so that the backend is asked to validate passwords, rather than Cork assuming the backend can produce a hash to it's specifications. This probably requires additional changes as well, pushing more functionality from the core out into the backend.

Any thoughts?

@FedericoCeratto
Owner

SQLAlchemy backend released in v0.7

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment