Skip to content

"db_session required" exception when working with aync def coroutines #380

Closed

Description

@amireldor

Hey there. I'm having an issue that reminds #126 which should've been resolved. I'm also using tornado. I've managed to reproduce it with this minimal code:

https://gist.github.com/amireldor/3c7f1701761d6c92855b868e5df07af0

I wrap a coroutine (async def) with db_session but an exception is thrown when doing a database action inside the coroutine. Regular wrapped functions or with blocks are fine.

This is Python 3.6.5 and pony 0.7.6; getting "pony.orm.core.TransactionError: db_session is required when working with the database".

Activity

  1. ranisalt commented on Sep 17, 2018

    @ranisalt

    I am having the same issue on Python 3.7. I work around it using with inside async def functions, but the extra level of indentation is undesirable.

  2. kozlovsky commented on Sep 22, 2018

    @kozlovsky
    Member

    Thanks for reporting, it should work now!

  3. ranisalt commented on Sep 22, 2018

    @ranisalt

    It does not. The following code:

    class AuthCheckHandler(BaseHandler):
        @orm.db_session
        async def get(self):
            user = self.current_user
            if user:
                self.finish({
                    'user': user.to_dict(only=['first_name', 'last_name']),
                })

    throws

        Traceback (most recent call last):
          File "/home/rsa/swartz/.venv/lib/python3.7/site-packages/pony/orm/core.py", line 544, in new_gen_func
            output = wrapped_interact(iterator)
          File "/home/rsa/swartz/.venv/lib/python3.7/site-packages/pony/orm/core.py", line 531, in wrapped_interact
            rollback_and_reraise(sys.exc_info())
          File "/home/rsa/swartz/.venv/lib/python3.7/site-packages/pony/orm/core.py", line 327, in rollback_and_reraise
            reraise(*exc_info)
          File "/home/rsa/swartz/.venv/lib/python3.7/site-packages/pony/utils/utils.py", line 118, in reraise
            try: raise exc.with_traceback(tb)
          File "/home/rsa/swartz/.venv/lib/python3.7/site-packages/pony/orm/core.py", line 521, in wrapped_interact
            output = interact(iterator, input, exc_info)
          File "/home/rsa/swartz/.venv/lib/python3.7/site-packages/pony/orm/core.py", line 494, in interact
            return next(iterator) if input is None else iterator.send(input)
        StopIteration
        
        The above exception was the direct cause of the following exception:
        
        Traceback (most recent call last):
          File "/home/rsa/swartz/.venv/lib/python3.7/site-packages/tornado/web.py", line 1592, in _execute
            result = yield result
          File "/home/rsa/swartz/.venv/lib/python3.7/site-packages/tornado/gen.py", line 1133, in run
            value = future.result()
        RuntimeError: generator raised StopIteration
    

    I guess db_session decorator is consuming the return value in a way that Tornado can't handle.

  4. ranisalt commented on Sep 24, 2018

    @ranisalt

    Here's a different, longer traceback of the same issue:

        Traceback (most recent call last):
          File "/home/rsa/swartz/.venv/lib/python3.7/site-packages/tornado/web.py", line 1592, in _execute
            result = yield result
          File "/home/rsa/swartz/.venv/lib/python3.7/site-packages/tornado/gen.py", line 1133, in run
            value = future.result()
          File "/home/rsa/swartz/.venv/lib/python3.7/site-packages/pony/orm/core.py", line 544, in new_gen_func
            output = wrapped_interact(iterator)
          File "/home/rsa/swartz/.venv/lib/python3.7/site-packages/pony/orm/core.py", line 531, in wrapped_interact
            rollback_and_reraise(sys.exc_info())
          File "/home/rsa/swartz/.venv/lib/python3.7/site-packages/pony/orm/core.py", line 327, in rollback_and_reraise
            reraise(*exc_info)
          File "/home/rsa/swartz/.venv/lib/python3.7/site-packages/pony/utils/utils.py", line 118, in reraise
            try: raise exc.with_traceback(tb)
          File "/home/rsa/swartz/.venv/lib/python3.7/site-packages/pony/orm/core.py", line 525, in wrapped_interact
            'You need to manually commit() changes before exiting from the generator')
          File "/home/rsa/swartz/.venv/lib/python3.7/site-packages/pony/utils/utils.py", line 129, in throw
            raise exc
        pony.orm.core.TransactionError: You need to manually commit() changes before exiting from the generator
    
  5. sashaaero commented on Oct 20, 2018

    @sashaaero
    Member

    This should solve the last problem. Please respond if it doesn't.

  6. ranisalt commented on Oct 20, 2018

    @ranisalt

    It does not:

    Traceback (most recent call last):
    File "/home/rsa/swartz/.venv/lib/python3.7/site-packages/tornado/web.py", line 1592, in _execute
      result = yield result
    File "/home/rsa/swartz/.venv/lib/python3.7/site-packages/tornado/gen.py", line 1133, in run
      value = future.result()
    File "/home/rsa/swartz/.venv/lib/python3.7/site-packages/pony/orm/core.py", line 544, in new_gen_func
      output = wrapped_interact(iterator)
    File "/home/rsa/swartz/.venv/lib/python3.7/site-packages/pony/orm/core.py", line 511, in wrapped_interact
      '@db_session-wrapped generator cannot be used inside another db_session')
    File "/home/rsa/swartz/.venv/lib/python3.7/site-packages/pony/utils/utils.py", line 129, in throw
      raise exc
    pony.orm.core.TransactionError: @db_session-wrapped generator cannot be used inside another db_session
    
  7. sashaaero commented on Oct 20, 2018

    @sashaaero
    Member

    Can you provide an example?

  8. kozlovsky commented on Oct 20, 2018

    @kozlovsky
    Member

    What exception says is true: "@db_session-wrapped generator cannot be used inside another db_session"

  9. kozlovsky commented on Oct 20, 2018

    @kozlovsky
    Member

    Well, actually we can temporarily switch outer db session to generator db session on resuming generator, but I'm not sure is it correct usage or not

  10. ranisalt commented on Oct 20, 2018

    @ranisalt

    Can you provide an example?

    I will try to isolate the issue in a gist

  11. kozlovsky commented on Oct 20, 2018

    @kozlovsky
    Member

    In my understanding, it looks incorrect when someone directly calls async (db_session-decorated) function from a normal (db_session-decorated) function. There should be some dispatcher from which async functions are actually called

  12. ranisalt commented on Oct 20, 2018

    @ranisalt

    Ok, @kozlovsky answer helped me find the issue. While my code works fine, tests are also wrapped with db_session so there is a nested db_session function call. Something like this:

    @db_session
    async def get():
      u = self.db.User[self.id]
      do_something(u)
    
    @db_session
    def test_get(http_client):
      res = http_client.get('https://localhost/path/to/get()')
    

    So it opens another db_session while the previous was already open.

    Is there a way to make db_session reentrant or at least a known way to work around this scenario?

  13. added a commit that references this issue on Jan 17, 2019
  14. trelljaret commented on Jul 13, 2020

    @trelljaret

    Is async support added finally?

  15. afkLuna commented on Sep 4, 2020

    @afkLuna

    Is it working now?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions