Add check whether Terracotta is running in multithreaded context

[kiksekage]

I am seeing some weird behaviour when trying to use the RelationalMetaStore class, specifically the db connection handling. Since the connection is created once and yielded down the call stack it seems we might run into some issues when trying to do the commit or the rollback if any of the downstream calls errored out. We essentially end up with a lot of AttributeError: 'NoneType' object has no attribute 'rollback'/'commit', which seems weird to me since these are only set to None in the finally block in the top-most connect call.

I tried adding a little if self.connected: check before commiting or rollbacking which seems to reduce errors and smooth loading of the PNGs, but that seems a little icky since we are in a if not self.connected loop.

Am I looking at this wrong? Has anyone tried using the RelationalMetaStore implementation and seen/not seen this before? I tried thinking about a reproducible example but I am struggling to figure out how that could be done.

[dionhaefner]

Do you have a full traceback?

[kiksekage]

I forgot to mention that I am experimenting with deploying TC on Azure Functions, and im seeing these errors when trying to deploy and debug locally. I didnt think to try and serve from TC itself before now, but I can confirm that terracotta serve -d <SAME DB> works without hitches. Here is the traceback, even though it isnt so juicy:

Traceback (most recent call last):
  File "/home/thns/bathy/bss-tc/.venv/lib/python3.8/site-packages/flask/app.py", line 2525, in wsgi_app
    response = self.full_dispatch_request()
  File "/home/thns/bathy/bss-tc/.venv/lib/python3.8/site-packages/flask/app.py", line 1822, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/home/thns/bathy/bss-tc/.venv/lib/python3.8/site-packages/flask/app.py", line 1820, in full_dispatch_request
    rv = self.dispatch_request()
  File "/home/thns/bathy/bss-tc/.venv/lib/python3.8/site-packages/flask/app.py", line 1796, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
  File "/home/thns/bathy/bss-tc/misc/terracotta/terracotta/server/metadata.py", line 54, in get_metadata
    payload = metadata(parsed_keys)
  File "/usr/lib/python3.8/contextlib.py", line 75, in inner
    return func(*args, **kwds)
  File "/home/thns/bathy/bss-tc/misc/terracotta/terracotta/handlers/metadata.py", line 18, in metadata
    metadata = driver.get_metadata(keys)
  File "/home/thns/bathy/bss-tc/misc/terracotta/terracotta/drivers/base_classes.py", line 43, in inner
    return fun(self, *args, **kwargs)
  File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__
    next(self.gen)
  File "/home/thns/bathy/bss-tc/misc/terracotta/terracotta/drivers/relational_meta_store.py", line 150, in connect
    self.connection.commit()
AttributeError: 'NoneType' object has no attribute 'commit'

[dionhaefner]

Is the Azure environment using multithreading under the hood? I have a hard time seeing how this could occur without race conditions.

[kiksekage]

Lo and behold, that was the exact issue. Here is the setting for managing the thread count for any future reference, thanks @dionhaefner 🙏

[dionhaefner]

Glad it's solved. Let's be more defensive and add a check for this.