Skip to content
Closed
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 17 additions & 11 deletions datajoint/connection.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
This module hosts the Connection class that manages the connection to the mysql database,
and the `conn` function that provides access to a persistent connection in datajoint.
"""

import warnings
from contextlib import contextmanager
import pymysql as client
import logging
Expand Down Expand Up @@ -59,7 +59,7 @@ def __init__(self, host, user, passwd, init_fun=None):
port = int(port)
else:
port = config['database.port']
self.conn_info = dict(host=host, port=port, user=user, passwd=passwd)
self.conn_info = dict(host=host, port=port, user=user, passwd=passwd, max_allowed_packet=1024**3) # 1073741824
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How is this used?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I'm also curious what this does? I'm guessing that you have encountered a problem with a large query hitting the packet size limit?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When I encountered the broken pipe issues in pymysql, I found that pymysql has its own max_allowed_packet for some reason that does not reflect the servers max_allowed_packet. Therefore I set it to the maximally allowed value by MySQL to let the server settings decide which packet sizes are accepted. Alternatively, a cleaner solution would be to put that value in dj.config.

self._conn = client.connect(init_command=init_fun, **self.conn_info)
if self.is_connected:
logger.info("Connected {user}@{host}:{port}".format(**self.conn_info))
Expand Down Expand Up @@ -161,19 +161,25 @@ def commit_transaction(self):
def transaction(self):
"""
Context manager for transactions. Opens an transaction and closes it after the with statement.
If an error is caught during the transaction, the commits are automatically rolled back. All
errors are raised again.
Only starts a transaction if there is not one going on already (MySQL does not support nested
transactions). If an error is caught during the transaction, the commits are automatically
rolled back. All errors are raised again.

Example:
>>> import datajoint as dj
>>> with dj.conn().transaction as conn:
>>> # transaction is open here
"""
try:
self.start_transaction()
yield self
except:
self.cancel_transaction()
raise
if not self.in_transaction:
try:
self.start_transaction()
yield self
except:
self.cancel_transaction()
raise
else:
self.commit_transaction()
else:
self.commit_transaction()
warnings.warn("""Connection is in a transaction already. MySQL does not support nested transaction. This
transaction call will be ignored. """)
yield self