Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Loading…

close the cursor as soon as possible #27

Closed
wants to merge 1 commit into from

2 participants

@peterbe

The reason for this change is quite complex and something I've only been able to "test" (there are no unit tests) in MySQL but then again MySQL is the only one famous for having this annoying problem (http://dev.mysql.com/doc/refman/5.0/en/gone-away.html)

Basically, what happens is that even though nothing goes wrong I get this on stdout::

    Exception _mysql_exceptions.OperationalError: (2013, 'Lost connection to MySQL server during query')
    in <bound method Cursor.__del__ of <MySQLdb.cursors.Cursor object at 0x1025c1250>> ignored

After a lot of searching in the pdb trenches I think I figured out what it was.

We use the cursor to execute something (e.g. "ALTER TABLE ...") but we don't care about any of the results. Then, nashvegas proceeds with a bunch of other things that might take time (e.g. executing post sync signals) and once all of that is done the objects are torn down and collected and when it does this, the cursor.del method is executed just before the python process shuts down and returns to the OS. Inside cursor.__del__ (see code below) it first exhausts what's been fetched and deletes it but because this happens such a long time since the cursor was executed you get that dreaded error (it's actually trapped as just a warning my MySQLdb).

The relevant code in MySQLdb/cursor.py:

    def __del__(self):
        self.close()
        self.errorhandler = None
        self._result = None

    def close(self):
        """Close the cursor. No further queries will be possible."""
        if not self.connection: return
        while self.nextset(): pass
        self.connection = None

    def nextset(self):
        if self._executed:
            self.fetchall()         
        del self.messages[:]

    db = self._get_db()
        nr = db.next_result()    # this is where MySQLdb barfs the exception 
        if nr == -1:
            return None
        self._do_get_result()
        self._post_get_result()
        self._warning_check()
        return 1

Like I said the exception is swallowed but we've experienced other weird undeterministic exceptions and errors with nashvegas and MySQL and I think it might help to at least use the dbapi properly as that makes things less likely to cause trouble when wrapped in Django's transaction magic clothing.

@paltman
Owner

Just to be clear the only code change here is line 187 where you close the cursor, right?

That's right.

@paltman
Owner

I really appreciate the pull request and the detailed write up and explanation. I don't currently use MySQL anywhere but I do remember seeing this exception from time to time when I used to use MySQL in other environments (still Django but not nashvegas). I'll make some time in the coming week or so to test this out on my existing project (Postgres) just to make sure it doesn't regress and then I'll pull and do a release.

Thanks!

@peterbe

Thanks. yeah, don't ask me why we're using MySQL when there is Postgres :)

Most of our problems we have with nashvegas + mysql are ultra weird and incredibly hard to reproduce. I think two titans are fighting for the attention of transaction management (django 1 and mysql 1) and somewhere there things go wrong sometimes.

Check out this one for example: http://jhonjairoroa87.blogspot.com/2011/03/mysql-operationalerror-error-on-rename.html

@paltman
Owner

So does this solve #26 as well? They seem like the same error message.

I apologize for the delay in dealing with this but will try my best to get your fix merged, tested, released this week.

@peterbe

I honestly don't know if it solves #26 because the error we get there is non-deterministic. It's not unlikely it'll fix it because I suspect lingering cursors might be the reason.

@peterbe

Any progress on looking at the pull request? It's a shame that it's not possible to make a reproduce-able case we can prove with a unit test but I think that it at least is more complete this way.

@paltman
Owner

I merged this manually and tagged/released 0.6.4 (now on pypi)

@paltman paltman closed this
@peterbe

Thanks! When time allows I'm going to upgrade us and see if it removes some of the weirdness problems we've had with mysql.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Commits on Aug 19, 2011
  1. @peterbe
This page is out of date. Refresh to see the latest.
Showing with 37 additions and 37 deletions.
  1. +37 −37 nashvegas/management/commands/upgradedb.py
View
74 nashvegas/management/commands/upgradedb.py
@@ -27,7 +27,7 @@ class MigrationError(Exception):
class Command(BaseCommand):
-
+
option_list = BaseCommand.option_list + (
make_option("-l", "--list", action = "store_true",
dest = "do_list", default = False,
@@ -56,26 +56,26 @@ class Command(BaseCommand):
help = "Upgrade database."
def _filter_down(self, stop_at=None):
-
+
if stop_at is None:
stop_at = float("inf")
-
+
applied = []
to_execute = []
scripts_in_directory = []
-
+
try:
already_applied = Migration.objects.all().order_by("migration_label")
-
+
for x in already_applied:
applied.append(x.migration_label)
-
+
in_directory = os.listdir(self.path)
in_directory = [migration for migration in in_directory if
not migration.startswith(".")]
in_directory.sort()
applied.sort()
-
+
for script in in_directory:
name, ext = os.path.splitext(script)
match = MIGRATION_NAME_RE.match(name)
@@ -85,7 +85,7 @@ def _filter_down(self, stop_at=None):
number = int(match.group(1))
if ext in [".sql", ".py"]:
scripts_in_directory.append((number, script))
-
+
for number, script in scripts_in_directory:
if script not in applied and number <= stop_at:
to_execute.append(script)
@@ -99,13 +99,13 @@ def _get_rev(self, fpath):
Get an SCM verion number. Try svn and git.
"""
rev = None
-
+
try:
cmd = ["git", "log", "-n1", "--pretty=format:\"%h\"", fpath]
rev = Popen(cmd, stdout=PIPE, stderr=PIPE).communicate()[0]
except:
pass
-
+
if not rev:
try:
cmd = ["svn", "info", fpath]
@@ -116,9 +116,9 @@ def _get_rev(self, fpath):
rev = tokens[1].strip()
except:
pass
-
+
return rev
-
+
def init_nashvegas(self):
# Copied from line 35 of django.core.management.commands.syncdb
# Import the 'management' module within each installed app, to register
@@ -139,7 +139,7 @@ def init_nashvegas(self):
msg = exc.args[0]
if not msg.startswith("No module named") or "management" not in msg:
raise
-
+
# @@@ make cleaner / check explicitly for model instead of looping over and doing string comparisons
connection = connections[self.db]
cursor = connection.cursor()
@@ -149,22 +149,22 @@ def init_nashvegas(self):
cursor.execute(s)
transaction.commit_unless_managed(using=self.db)
return
-
+
def create_migrations(self):
statements = get_sql_for_new_models(self.args)
if len(statements) > 0:
for s in statements:
print s
-
+
@transaction.commit_manually
def execute_migrations(self, show_traceback=False):
migrations = self._filter_down()
-
+
if not len(migrations):
sys.stdout.write("There are no migrations to apply.\n")
-
+
created_models = set()
-
+
try:
for migration in migrations:
migration_path = os.path.join(self.path, migration)
@@ -172,18 +172,19 @@ def execute_migrations(self, show_traceback=False):
lines = fp.readlines()
fp.close()
content = "".join(lines)
-
+
if migration_path.endswith(".sql"):
to_execute = "".join(
[l for l in lines if not l.startswith("### New Model: ")]
)
connection = connections[self.db]
cursor = connection.cursor()
-
+
sys.stdout.write("Executing %s... " % migration)
-
+
try:
cursor.execute(to_execute)
+ cursor.close()
except Exception:
sys.stdout.write("failed\n")
if show_traceback:
@@ -191,20 +192,20 @@ def execute_migrations(self, show_traceback=False):
raise MigrationError()
else:
sys.stdout.write("success\n")
-
+
for l in lines:
if l.startswith("### New Model: "):
created_models.add(
get_model(
*l.replace("### New Model: ", "").strip().split(".")
- )
+ )
)
elif migration_path.endswith(".py"):
sys.stdout.write("Executing %s... " % migration)
-
+
module = {}
execfile(migration_path, {}, module)
-
+
if "migrate" in module and callable(module["migrate"]):
try:
module["migrate"]()
@@ -215,7 +216,7 @@ def execute_migrations(self, show_traceback=False):
raise MigrationError()
else:
sys.stdout.write("success\n")
-
+
Migration.objects.create(
migration_label=migration,
content=content,
@@ -241,7 +242,7 @@ def execute_migrations(self, show_traceback=False):
raise
else:
transaction.commit(using=self.db)
-
+
def seed_migrations(self, stop_at=None):
# @@@ the command-line interface needs to be re-thinked
try:
@@ -263,18 +264,18 @@ def seed_migrations(self, stop_at=None):
print m.migration_label, "has been seeded"
else:
print m.migration_label, "was already applied."
-
-
+
+
def list_migrations(self):
migrations = self._filter_down()
if len(migrations) == 0:
print "There are no migrations to apply."
return
-
+
print "Migrations to Apply:"
for script in migrations:
print "\t%s" % script
-
+
def handle(self, *args, **options):
"""
Upgrades the database.
@@ -287,23 +288,22 @@ def handle(self, *args, **options):
self.do_create = options.get("do_create")
self.do_seed = options.get("do_seed")
self.args = args
-
+
self.path = options.get("path")
self.verbosity = int(options.get("verbosity", 1))
self.interactive = options.get("interactive")
self.db = options.get("database", DEFAULT_DB_ALIAS)
-
+
self.init_nashvegas()
if self.do_create:
self.create_migrations()
-
+
if self.do_execute:
self.execute_migrations(show_traceback=True)
-
+
if self.do_list:
self.list_migrations()
-
+
if self.do_seed:
self.seed_migrations()
-
Something went wrong with that request. Please try again.