New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SQL backend #27

Closed
eseifert opened this Issue Jan 27, 2013 · 13 comments

Comments

Projects
None yet
7 participants
@eseifert

eseifert commented Jan 27, 2013

It would be nice to have a storage backend for Cork that uses an SQL database. In addition to the actual backend code some changes would be necessary to define the backend that should be used for Cork.

I implemented a very raw version of an SQL backend based on SQLAlchemy: https://gist.github.com/4651239. It's not optimized or well tested at the moment, but it works.

At the moment the implementation uses an additional layer (SqlRowProxy, SqlDict, and SqlColumnDict) to emulate Python dictionaries for Cork's users, roles, or pending_registrations. The classes of this layer are used to translate dictionary access to SQLAlchemy calls.

Comments are very welcome.

@ghost ghost assigned FedericoCeratto Jan 28, 2013

@jcdyer

This comment has been minimized.

Show comment
Hide comment
@jcdyer

jcdyer Mar 3, 2013

I think more important than the creation of the SQL backend itself is allowing the user to pass in a custom Backend class when they create a new Cork() instance, so that additional backends can be created separate from the Cork project, whether it's based on SQL or MongoDB or unix passwd file or any other type of data store.

Something like:

class Cork(object):
    def __init__(self, backend=JsonBackend, **kwargs):
        self._store = self.backend(
            kwargs['directory'],
            kwargs['users_fname'],
            kwargs['role_fname'],
            kwargs['pending_reg_fname'],
            kwargs['initialize'])

And maybe expand the docs to describe the interface to the backend more clearly to help people who want to write their own.

jcdyer commented Mar 3, 2013

I think more important than the creation of the SQL backend itself is allowing the user to pass in a custom Backend class when they create a new Cork() instance, so that additional backends can be created separate from the Cork project, whether it's based on SQL or MongoDB or unix passwd file or any other type of data store.

Something like:

class Cork(object):
    def __init__(self, backend=JsonBackend, **kwargs):
        self._store = self.backend(
            kwargs['directory'],
            kwargs['users_fname'],
            kwargs['role_fname'],
            kwargs['pending_reg_fname'],
            kwargs['initialize'])

And maybe expand the docs to describe the interface to the backend more clearly to help people who want to write their own.

@23min

This comment has been minimized.

Show comment
Hide comment
@23min

23min Mar 4, 2013

Contributor

See https://github.com/23min/bottle-cork/blob/mongobackend-v1/cork/cork.py

I was basically forced to implement that in order to make mongo backend
pluggable.

On Sun, Mar 3, 2013 at 8:13 PM, Cliff Dyer notifications@github.com wrote:

I think more important than the creation of the SQL backend itself is
allowing the user to pass in a custom Backend class when they create a new
Cork() instance, so that additional backends can be created separate from
the Cork project, whether it's based on SQL or MongoDB or unix passwd file
or any other type of data store.

Something like:

class Cork(object):
def init(self, backend=JsonBackend, **kwargs):
self._store = self.backend(
kwargs['directory'],
kwargs['users_fname'],
kwargs['role_fname'],
kwargs['pending_reg_fname'],
kwargs['initialize'])

And maybe expand the docs to describe the interface to the backend more
clearly to help people who want to write their own.


Reply to this email directly or view it on GitHubhttps://github.com//issues/27#issuecomment-14352754
.

Contributor

23min commented Mar 4, 2013

See https://github.com/23min/bottle-cork/blob/mongobackend-v1/cork/cork.py

I was basically forced to implement that in order to make mongo backend
pluggable.

On Sun, Mar 3, 2013 at 8:13 PM, Cliff Dyer notifications@github.com wrote:

I think more important than the creation of the SQL backend itself is
allowing the user to pass in a custom Backend class when they create a new
Cork() instance, so that additional backends can be created separate from
the Cork project, whether it's based on SQL or MongoDB or unix passwd file
or any other type of data store.

Something like:

class Cork(object):
def init(self, backend=JsonBackend, **kwargs):
self._store = self.backend(
kwargs['directory'],
kwargs['users_fname'],
kwargs['role_fname'],
kwargs['pending_reg_fname'],
kwargs['initialize'])

And maybe expand the docs to describe the interface to the backend more
clearly to help people who want to write their own.


Reply to this email directly or view it on GitHubhttps://github.com//issues/27#issuecomment-14352754
.

@FedericoCeratto

This comment has been minimized.

Show comment
Hide comment
@FedericoCeratto

FedericoCeratto Mar 4, 2013

Owner

I did some experiments around building a more formal Backend interface and allowing configurable backends.
I'd like to hear you opinions on how users/roles/etc should be accessed and modified:

  1. Keep the current dictionary-like interface (e.g. users['tom']). This requires the backends to implement dictionary accessors.
  2. Remove the dictionaries and use explicit methods e.g. fetch_user(name='tom'), update_user(...), delete_user(...). This would require a lot of code (and unit tests) changes. This could make the backends a bit more readable.

Any strong feeling about the two options? Thanks

Owner

FedericoCeratto commented Mar 4, 2013

I did some experiments around building a more formal Backend interface and allowing configurable backends.
I'd like to hear you opinions on how users/roles/etc should be accessed and modified:

  1. Keep the current dictionary-like interface (e.g. users['tom']). This requires the backends to implement dictionary accessors.
  2. Remove the dictionaries and use explicit methods e.g. fetch_user(name='tom'), update_user(...), delete_user(...). This would require a lot of code (and unit tests) changes. This could make the backends a bit more readable.

Any strong feeling about the two options? Thanks

@jcdyer

This comment has been minimized.

Show comment
Hide comment
@jcdyer

jcdyer Mar 4, 2013

Ideally, it would be easy both to use the backend and to write a new backend. Maybe allow dictionary access like is currently done, but implement the accessors in a superclass to map Backend.get_user() and Backend.set_user() to Backend.user['foo']. Might require some trickery in using descriptors to make it work. I'll give it a little thought, and see if I can come up with a demo implementation of what I mean.

jcdyer commented Mar 4, 2013

Ideally, it would be easy both to use the backend and to write a new backend. Maybe allow dictionary access like is currently done, but implement the accessors in a superclass to map Backend.get_user() and Backend.set_user() to Backend.user['foo']. Might require some trickery in using descriptors to make it work. I'll give it a little thought, and see if I can come up with a demo implementation of what I mean.

@lllama

This comment has been minimized.

Show comment
Hide comment
@lllama

lllama Mar 13, 2013

I'd like to vote that any SQL backend API does not add a requirement for SQLAlchemy or any other package. Backends can, of course, use these as they see fit but Cork shouldn't add this to its requirements.

lllama commented Mar 13, 2013

I'd like to vote that any SQL backend API does not add a requirement for SQLAlchemy or any other package. Backends can, of course, use these as they see fit but Cork shouldn't add this to its requirements.

@FedericoCeratto

This comment has been minimized.

Show comment
Hide comment
@FedericoCeratto

FedericoCeratto Mar 13, 2013

Owner

Sounds reasonable. Each backend should import its own dependencies only if it's being used.
The backends should be declared as extras in http://peak.telecommunity.com/DevCenter/setuptools#declaring-extras-optional-features-with-their-own-dependencies

Owner

FedericoCeratto commented Mar 13, 2013

Sounds reasonable. Each backend should import its own dependencies only if it's being used.
The backends should be declared as extras in http://peak.telecommunity.com/DevCenter/setuptools#declaring-extras-optional-features-with-their-own-dependencies

@jcdyer

This comment has been minimized.

Show comment
Hide comment
@jcdyer

jcdyer Mar 13, 2013

So I figured out a good way of having the backend be database aware, and still being able to get dict-style access on the attributes. I wrote it up here: Notes on the Descriptor Protocol. See the classes AttributeDict and AttributeDictObject for the implementation, and UserNotes for how to use it.

As far as dependency management, I would vote to create separate packages for backends that have external dependencies. So to use cork with SQLAlchemy, I would do something like:

$ pip install bottle-cork bottle-cork-sqlalchemy
... (bottle-cork-sqlalchemy has sqlalchemy (and bottle-cork?) as dependencies)

Then you would use it by doing

>>> import cork, cork_sqlalchemy
>>> auth = cork.Cork(backend=cork_sqlalchemy.SQLAlchemyBackend, ...)

jcdyer commented Mar 13, 2013

So I figured out a good way of having the backend be database aware, and still being able to get dict-style access on the attributes. I wrote it up here: Notes on the Descriptor Protocol. See the classes AttributeDict and AttributeDictObject for the implementation, and UserNotes for how to use it.

As far as dependency management, I would vote to create separate packages for backends that have external dependencies. So to use cork with SQLAlchemy, I would do something like:

$ pip install bottle-cork bottle-cork-sqlalchemy
... (bottle-cork-sqlalchemy has sqlalchemy (and bottle-cork?) as dependencies)

Then you would use it by doing

>>> import cork, cork_sqlalchemy
>>> auth = cork.Cork(backend=cork_sqlalchemy.SQLAlchemyBackend, ...)
@FedericoCeratto

This comment has been minimized.

Show comment
Hide comment
@FedericoCeratto

FedericoCeratto Mar 14, 2013

Owner

A little wiki page to keep track of the backends:
https://github.com/FedericoCeratto/bottle-cork/wiki/Backends

Owner

FedericoCeratto commented Mar 14, 2013

A little wiki page to keep track of the backends:
https://github.com/FedericoCeratto/bottle-cork/wiki/Backends

@dineshbvadhia

This comment has been minimized.

Show comment
Hide comment
@dineshbvadhia

dineshbvadhia Mar 23, 2013

Concur with IIIama to support a pure sqlite backend without SQLAlchemy etc. Preference would be to prioritize sqlite as it can easily solve many db problems and is part of the Python standard library. Best.

dineshbvadhia commented Mar 23, 2013

Concur with IIIama to support a pure sqlite backend without SQLAlchemy etc. Preference would be to prioritize sqlite as it can easily solve many db problems and is part of the Python standard library. Best.

@eseifert

This comment has been minimized.

Show comment
Hide comment
@eseifert

eseifert Mar 24, 2013

It's great to have configurable backends now. I updated my experimental sqlalchemy backend (https://gist.github.com/eseifert/4651239) to implement the new interface.

I also suggest that a pop method gets added to base_backend.Table because it's required for the backends interface.

eseifert commented Mar 24, 2013

It's great to have configurable backends now. I updated my experimental sqlalchemy backend (https://gist.github.com/eseifert/4651239) to implement the new interface.

I also suggest that a pop method gets added to base_backend.Table because it's required for the backends interface.

@FedericoCeratto

This comment has been minimized.

Show comment
Hide comment
@FedericoCeratto

FedericoCeratto Mar 24, 2013

Owner

eseifert: can I start integrating your backend? If so, can I have your name and email address for the licensing?

Owner

FedericoCeratto commented Mar 24, 2013

eseifert: can I start integrating your backend? If so, can I have your name and email address for the licensing?

@pbkdf3

This comment has been minimized.

Show comment
Hide comment
@pbkdf3

pbkdf3 Mar 28, 2013

I would like to implement an ldap backend. However, this doesn't fit the current backend model very well.
In particular, I propose the api be changed so that the backend is asked to validate passwords, rather than Cork assuming the backend can produce a hash to it's specifications. This probably requires additional changes as well, pushing more functionality from the core out into the backend.

Any thoughts?

pbkdf3 commented Mar 28, 2013

I would like to implement an ldap backend. However, this doesn't fit the current backend model very well.
In particular, I propose the api be changed so that the backend is asked to validate passwords, rather than Cork assuming the backend can produce a hash to it's specifications. This probably requires additional changes as well, pushing more functionality from the core out into the backend.

Any thoughts?

@FedericoCeratto

This comment has been minimized.

Show comment
Hide comment
@FedericoCeratto

FedericoCeratto Apr 8, 2013

Owner

SQLAlchemy backend released in v0.7

Owner

FedericoCeratto commented Apr 8, 2013

SQLAlchemy backend released in v0.7

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment