Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

db.execute_write_fn(create_tables, block=True) hangs a thread if connection fails #935

Closed
simonw opened this issue Aug 15, 2020 · 3 comments
Labels

Comments

@simonw
Copy link
Owner

simonw commented Aug 15, 2020

Discovered in simonw/latest-datasette-with-all-plugins#3 (comment)

@simonw
Copy link
Owner Author

simonw commented Aug 15, 2020

The bug is here:

def _execute_writes(self):
# Infinite looping thread that protects the single write connection
# to this database
conn = self.connect(write=True)
while True:
task = self._write_queue.get()
try:
result = task.fn(conn)
except Exception as e:
print(e)
result = e
task.reply_queue.sync_q.put(result)

If conn = self.connect(write=True) raises an exception the entire server hangs, like this:

% datasette -i fixtures.db  --get /            
Exception in thread Thread-1:
Traceback (most recent call last):
  File "/usr/local/opt/python@3.8/Frameworks/Python.framework/Versions/3.8/lib/python3.8/threading.py", line 932, in _bootstrap_inner
    self.run()
  File "/usr/local/opt/python@3.8/Frameworks/Python.framework/Versions/3.8/lib/python3.8/threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "/Users/simon/.local/share/virtualenvs/latest-datasette-with-all-plugins-PJL_Xy9e/lib/python3.8/site-packages/datasette/database.py", line 92, in _execute_writes
    conn = self.connect(write=True)
  File "/Users/simon/.local/share/virtualenvs/latest-datasette-with-all-plugins-PJL_Xy9e/lib/python3.8/site-packages/datasette/database.py", line 55, in connect
    assert not (write and not self.is_mutable)
AssertionError

... server hangs here ...

@simonw simonw added the bug label Aug 15, 2020
@simonw
Copy link
Owner Author

simonw commented Aug 15, 2020

The easiest way to recreate this is to attempt a write against an immutable database, which triggers this assertion error:

def connect(self, write=False):
if self.is_memory:
return sqlite3.connect(":memory:")
# mode=ro or immutable=1?
if self.is_mutable:
qs = "?mode=ro"
else:
qs = "?immutable=1"
assert not (write and not self.is_mutable)

@simonw
Copy link
Owner Author

simonw commented Aug 15, 2020

This implementation seems to fix it, need to work out how to test though.

diff --git a/datasette/database.py b/datasette/database.py
index ffa7a79..7ba1456 100644
--- a/datasette/database.py
+++ b/datasette/database.py
@@ -89,14 +89,22 @@ class Database:
     def _execute_writes(self):
         # Infinite looping thread that protects the single write connection
         # to this database
-        conn = self.connect(write=True)
+        conn_exception = None
+        conn = None
+        try:
+            conn = self.connect(write=True)
+        except Exception as e:
+            conn_exception = e
         while True:
             task = self._write_queue.get()
-            try:
-                result = task.fn(conn)
-            except Exception as e:
-                print(e)
-                result = e
+            if conn_exception is not None:
+                result = conn_exception
+            else:
+                try:
+                    result = task.fn(conn)
+                except Exception as e:
+                    print(e)
+                    result = e
             task.reply_queue.sync_q.put(result)
 
     async def execute_fn(self, fn):

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant