Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PostgreSQL: server closed the connection unexpectedly #357

Closed
kovaxur opened this Issue Jun 26, 2018 · 3 comments

Comments

Projects
None yet
2 participants
@kovaxur
Copy link

kovaxur commented Jun 26, 2018

Hi,
I started to use ponyorm for my project, I found the following problem:

Steps to reproduce:

  1. I start my application, everything is fine, I can work with the database properly
    (2. I can do stress tests all night, everything goes well)
  2. If I don't do anything for like 10-15 minutes and then I try to execute something, I get the following exception message:

server closed the connection unexpectedly
This probably means the server terminated abnormally
before or while processing the request.

Trace:
File \"<string>\", line 2, in __iter__\n","  File \"/usr/lib/python3.6/site-packages/pony/utils/utils.py\", line 58, in cut_traceback\n    return func(*args, **kwargs)\n","  
File \"/usr/lib/python3.6/site-packages/pony/orm/core.py\", line 5506, in __iter__\n    return iter(query._fetch())\n","  
File \"/usr/lib/python3.6/site-packages/pony/orm/core.py\", line 5343, in _fetch\n    cursor = database._exec_sql(sql, arguments)\n","  
File \"/usr/lib/python3.6/site-packages/pony/orm/core.py\", line 820, in _exec_sql\n    connection = cache.reconnect(e)\n","  
File \"/usr/lib/python3.6/site-packages/pony/orm/core.py\", line 1625, in reconnect\n    if not provider.should_reconnect(exc): reraise(*sys.exc_info())\n","  
File \"/usr/lib/python3.6/site-packages/pony/utils/utils.py\", line 87, in reraise\n    try: raise exc.with_traceback(tb)\n"," 
File \"/usr/lib/python3.6/site-packages/pony/orm/core.py\", line 818, in _exec_sql\n    try: new_id = provider.execute(cursor, sql, arguments, returning_id)\n","  File \"<string>\", line 2, in execute\n","  
File \"/usr/lib/python3.6/site-packages/pony/orm/dbapiprovider.py\", line 61, in wrap_dbapi_exceptions\n    raise OperationalError(e)\n"

Versions:

pony.__version__
'0.7.3'
psycopg2.__version__
'2.7.5 (dt dec pq3 ext lo64)'
python 3.6

Maybe this is rather an architectural problem in my code, however I used the same model as in the examples.

Thanks

@kovaxur

This comment has been minimized.

Copy link
Author

kovaxur commented Jun 26, 2018

I did some debugging, so if I don't do anything, the connection probably times out after a while, in core.py line 819, an exception is catched (cursor's closed attribute is True)

        try: new_id = provider.execute(cursor, sql, arguments, returning_id)
        except Exception as e:
            connection = cache.reconnect(e)
            cursor = connection.cursor()
            if local.debug: log_sql(sql, arguments)
            t = time()
            new_id = provider.execute(cursor, sql, arguments, returning_id)

image

The exception is caught, and now it tries to reconnect:
The exception is re-raised during the reconnect:

 def reconnect(cache, exc):
        provider = cache.database.provider
        if exc is not None:
            exc = getattr(exc, 'original_exc', exc)
            if not provider.should_reconnect(exc): reraise(*sys.exc_info())
@kovaxur

This comment has been minimized.

Copy link
Author

kovaxur commented Jun 26, 2018

The should_reconnect functions:
image

exc.pgcode = None
exc.pgerror = 'server closed the connection unexpectedly
	This probably means the server terminated abnormally
	before or while processing the request.'
exc.cursor = (not none)

@kozlovsky kozlovsky closed this in f763694 Jun 26, 2018

@kozlovsky

This comment has been minimized.

Copy link
Member

kozlovsky commented Jun 26, 2018

Thanks for reporting, now Pony will again reconnect to PostgreSQL database automatically after the server closes the connection.

By the way, it looks strange that your server drops connection after 10-15 minutes, maybe there is some misconfiguration in your database connection settings.

kozlovsky added a commit that referenced this issue Jul 23, 2018

Pony ORM Release 0.7.4 (2018-07-23)
# Major features

* Hybrid methods and properties added: https://docs.ponyorm.com/entities.html#hybrid-methods-and-properties
* Allow to base queries on another queries: `select(x.a for x in prev_query if x.b)`
* Added support of Python 3.7
* Added support of PyPy
* `group_concat()` aggregate function added
* pony.flask subpackage added for integration with Flask

# Other features

* `distinct` option added to aggregate functions
* Support of explicit casting to `float` and `bool` in queries

# Improvements

* Apply @cut_traceback decorator only when pony.MODE is 'INTERACTIVE'

# Bugfixes

* In SQLite3 `LIKE` is case sensitive now
* #249: Fix incorrect mixin used for Timedelta
* #251: correct dealing with qualified table names
* #301: Fix aggregation over JSON Column
* #306: Support of frozenset constants added
* #308: Fixed an error when assigning JSON attribute value to the same attribute: obj.json_attr = obj.json_attr
* #313: Fix missed retry on exception raised during db_session.__exit__
* #314: Fix AttributeError: 'NoneType' object has no attribute 'seeds'
* #315: Fix attribute lifting for JSON attributes
* #321: Fix KeyError on obj.delete()
* #325: duplicating percentage sign in raw SQL queries without parameters
* #331: Overriding __len__ in entity fails
* #336: entity declaration serialization
* #357: reconnect after PostgreSQL server closed the connection unexpectedly
* Fix Python implementation of between() function and rename arguments: between(a, x, y) -> between(x, a, b)
* Fix retry handling: in PostgreSQL and Oracle an error can be raised during commit
* Fix optimistic update checks for composite foreign keys
* Don't raise OptimisticCheckError if db_session is not optimistic
* Handling incorrect datetime values in MySQL
* Improved ImportError exception messages when MySQLdb, pymysql, psycopg2 or psycopg2cffi driver was not found
* desc() function fixed to allow reverse its effect by calling desc(desc(x))
* __contains__ method should check if objects belong to the same db_session
* Fix pony.MODE detection; mod_wsgi detection according to official doc
* A lot of inner fixes
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.