[Tornado with pony]TransactionError: db_session is required when working with the database #126

Closed
jie opened this Issue Apr 12, 2015 · 4 comments

Projects

None yet

2 participants

@jie
jie commented Apr 12, 2015
    @coroutine
    @authenticated
    @pony.orm.db_session
    def post(self, *args, **kwargs):
        image_uid = yield self.get_image_uid()
        image = Image(
            uid=image_uid,
            image_type=self.ImageType,
            user=User[self.current_user.id]
        )
        image.save()
        self.genReturn({'message': 'success'})

but raise exception:

  File "/Users/zhouyang/MyCode/github/starcircus/core/modules/base_handler.py", line 345, in post_operate
    user=User[self.current_user.id]
  File "<auto generated wrapper of __getitem__() function>", line 2, in __getitem__
  File "/usr/local/lib/python2.7/site-packages/pony/utils.py", line 88, in cut_traceback
    return func(*args, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/pony/orm/core.py", line 2939, in __getitem__
    return entity._find_one_(kwargs)
  File "/usr/local/lib/python2.7/site-packages/pony/orm/core.py", line 3039, in _find_one_
    obj, unique = entity._find_in_cache_(pkval, avdict, for_update)
  File "/usr/local/lib/python2.7/site-packages/pony/orm/core.py", line 3044, in _find_in_cache_
    cache = entity._database_._get_cache()
  File "/usr/local/lib/python2.7/site-packages/pony/orm/core.py", line 508, in _get_cache
    ): throw(TransactionError, 'db_session is required when working with the database')
  File "/usr/local/lib/python2.7/site-packages/pony/utils.py", line 126, in throw
    raise exc
TransactionError: db_session is required when working with the database
@kozlovsky
Contributor

Thanks for the reporting!

The @db_session decorator can be applied to a normal functions only. It ensures that the queries are execute inside a single transaction. This is important to guarantee that the data returned from the database are consistent and that all modifications will be performed in the "all-or-nothing" style.

The post function in your code is not actually a function, but a generator. The generator execution is not atomic - it can be suspended at some point of time and resumed later. But the database transaction cannot be suspended, because it holds database locks. Because of this it is not correct to wrap entire generator function with @db_session.

Instead of this you can use @db_session as a context manager. You need to apply it to atomic parts of the generator, that is, the parts between yield invocations. I think that your code should look like this:

    @coroutine
    @authenticated
    def post(self, *args, **kwargs):
        image_uid = yield self.get_image_uid()
        with @pony.orm.db_session:
            image = Image(
                uid=image_uid,
                image_type=self.ImageType,
                user=User[self.current_user.id]
            )
            image.save()
        self.genReturn({'message': 'success'})

I think that we need to check if the function decorated by @db_session is usual function or generator function and raise appropriate exception with a clear message.

@kozlovsky
Contributor

Also, I'm wonder what is image.save() method. In Pony ORM it is not necessary to save individual objects, all modified objects will be saved automatically upon exit of db_session or if a programmer manually called flush() or commit() functions.

@jie
jie commented Apr 13, 2015

yes,save method call flush and commit
Thanks

@kozlovsky kozlovsky closed this in 6717eab Jan 11, 2016
@kozlovsky kozlovsky added a commit that referenced this issue Jan 11, 2016
@kozlovsky kozlovsky Pony ORM Release 0.6.2 (2015-01-11)
The documentation was moved from this repo to a separate one at https://github.com/ponyorm/pony-doc
The compiled version can be found at https://docs.ponyorm.com

# New features

* Python 3.5 support
* #132, #145: raw_sql() function was added
* #126: Ability to use @db_session with generator functions
* #116: Add support to select by UUID
* Ability to get string SQL statement using the Query.get_sql() method
* New function delete(gen) and Query.delete(bulk=False)
* Now it is possible to override Entity.__init__() and declare custom entity methods

# Backward incompatible changes

* Normalizing table names for symmetric relationships
* Autostrip - automatically remove leading and trailing characters

# Bugfixes

* #87: Pony fails with pymysql installed as MySQLdb
* #118: Pony should reconnect if previous connection was created before process was forked
* #121: Unable to update value of unique attribute
* #122: AssertionError when changing part of a composite key
* #127: a workaround for incorrect pysqlite locking behavior
* #136: Cascade delete does not work correctly for one-to-one relationships
* #141, #143: remove restriction on adding new methods to entities
* #142: Entity.select_random() AssertionError
* #147: Add 'atom_expr' symbol handling for Python 3.5 grammar
ab09f64
@kozlovsky kozlovsky added this to the 0.6.2 milestone Jan 11, 2016
@kozlovsky
Contributor

Hi, in release 0.6.2 we added support of generators to @db_session decorator:

https://blog.ponyorm.com/2016/01/11/pony-orm-release-0-6-2/

Previously the @db_session decorator was used for decorating functions that work with the database. But it didn't work correctly when was applied to generators (functions that return value using the yield expression). Starting with this release you can use the @db_session decorator for generators too.

With regular functions, the @db_session decorator works as a scope. When your program leaves the db_session scope, Pony finishes the transaction by performing commit (or rollback) and clears the db_session cache.

In case of a generator, the program can reenter the generator code for several times. In this case, when your program leaves the generator code, the db_session is not over, but suspended and Pony doesn't clear the cache. In the same time, we don't know if the program will come back to this generator code again. That is why you have to explicitly commit or rollback current transaction before the program leaves the generator on yield. On regular functions Pony calls commit() or rollback() automatically on leaving the @db_session scope.

In essence, here is the difference with using @db_session with generator functions:

  1. You have to call commit() or rollback() before the yield expression explicitly.
  2. Pony doesn't clear the transaction cache, so you can continue using loaded objects when coming back to the same generator.
  3. With a generator function, the @db_session can be used only as a decorator, not a context manager. This is because in Python the context manager cannot understand that it was left on yield.
  4. The @db_session parameters, such as retry or serializable cannot be used with generator functions. The only parameter that can be used in this case is immediate.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment