TypeError: can't apply this __setattr__ to DefaultMeta object

[MartinThoma]

I don't know where this comes from. I have a small project where all versions are pinned and the Flask app runs in a Docker container. I just made an unrelated changes with the same versions and get this error. Does this look familiar?

Expected Behavior

Just running without crashes

from flask_sqlalchemy import SQLAlchemy

db = SQLAlchemy()

Actual Behavior

Crashes

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.8/site-packages/flask_sqlalchemy/__init__.py", line 716, in __init__
    self.Model = self.make_declarative_base(model_class, metadata)
  File "/usr/local/lib/python3.8/site-packages/flask_sqlalchemy/__init__.py", line 798, in make_declarative_base
    model.query_class = self.Query
  File "/usr/local/lib/python3.8/site-packages/sqlalchemy/ext/declarative/api.py", line 79, in __setattr__
    _add_attribute(cls, key, value)
  File "/usr/local/lib/python3.8/site-packages/sqlalchemy/ext/declarative/base.py", line 802, in _add_attribute
    type.__setattr__(cls, key, value)
TypeError: can't apply this __setattr__ to DefaultMeta object

Environment

• Python version: Python 3.8.4 (Docker python:3.8-slim-buster)
• Flask-SQLAlchemy version: flask-sqlalchemy==2.4.1
• SQLAlchemy version: sqlalchemy==1.3.13

[MartinThoma]

Interestingly, this works fine:

$ virtualenv --python=python3.8 venv
$ source venv/bin/activate
$ pip install flask-sqlalchemy==2.4.1 sqlalchemy==1.3.13
$ python
Python 3.8.1 (default, Feb 13 2020, 10:26:36) 
[GCC 7.5.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from flask_sqlalchemy import SQLAlchemy; db = SQLAlchemy()

[MartinThoma]

With 3.8.4rc1 from pyenv I don't get that issue on my host. Only within the docker container. Even when I create an empty folder within that Docker container. Even when I create a venv in an empty folder in that container:

(venv) root@ea40bbfdf33b:/opt/app/foo# python
Python 3.8.4 (default, Jul 14 2020, 03:07:46) 
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from flask_sqlalchemy import SQLAlchemy
>>> db = SQLAlchemy()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/opt/app/foo/venv/lib/python3.8/site-packages/flask_sqlalchemy/__init__.py", line 716, in __init__
    self.Model = self.make_declarative_base(model_class, metadata)
  File "/opt/app/foo/venv/lib/python3.8/site-packages/flask_sqlalchemy/__init__.py", line 798, in make_declarative_base
    model.query_class = self.Query
  File "/opt/app/foo/venv/lib/python3.8/site-packages/sqlalchemy/ext/declarative/api.py", line 79, in __setattr__
    _add_attribute(cls, key, value)
  File "/opt/app/foo/venv/lib/python3.8/site-packages/sqlalchemy/ext/declarative/base.py", line 802, in _add_attribute
    type.__setattr__(cls, key, value)
TypeError: can't apply this __setattr__ to DefaultMeta object

[piotrgredowski]

I've just met similar problem.
It happens only on Docker image python:3.8-slim-buster which now resolves to python:3.8.4-slim-buster (due to yesterday's release).
Changing Docker image to python:3.8.3-slim-buster solved problem for now. But I want to upgrade to 3.8.4 so it will be great if this issue is solved.

[kam193]

This problem is probably caused by changes in Python 3.8.4 (see https://docs.python.org/3/whatsnew/changelog.html#core-and-builtins)

[taryodor]

This issue is caused by this PR from Python 3.8.4:
https://github.com/python/cpython/pull/21288/files

To be more specific, the problem happens here:
https://github.com/python/cpython/pull/21288/files#diff-c3cf251f16d5a03a9e7d4639f2d6f998R5952

[MartinThoma]

I just confirmed @piotrgredowski hint: python:3.8.3-slim-buster works

[MartinThoma]

@taryodor Is this change expected to break backwards compatibility? If not, did somebody inform people at cpython already?

[davidism]

I don't completely understand the reason referred to in the linked cpython change, so I'm not sure what the intended solution in our code is. In 2017, I made a change to construct the declarative base metaclass from some mixin classes to support customizing what features we add. SQLAlchemy is what overrides __setattr__ and uses type.__setattr__, which seems to be the thing that the change affects, but I don't think it's their issue either.

original cpython issue: https://bugs.python.org/issue39960
original cpython pr: python/cpython#21092

[davidism]

This comes from the line model.query_class = self.Query, which is assigning an extra attribute to the metaclass type, which is what the type.__setattr__ code is used for. It only happens when using our metaclass composed of mixins, if I use SQLAlchemy's DeclarativeMeta directly the assignment works. I have a feeling the cpython change needs to be adjusted, I'm really not sure what we could do differently in our code that wouldn't involve a massive rewrite.

Causes error:

from sqlalchemy.ext.declarative import declarative_base
from flask_sqlalchemy.model import DefaultMeta

Base = declarative_base(metaclass=DefaultMeta)
Base.test = True

Doesn't cause error:

from sqlalchemy.ext.declarative import declarative_base

Base = declarative_base()
Base.test = True

[davidism]

cc @zzzeek in case you see reports of this

[kam193]

I reported it in Python bug tracker: https://bugs.python.org/issue41295

But look carefully into report, and then check the flask-sqlalchemy code: the problem is caused, when a class inheritance from meta and non-metaclasses. And now, 2 months ago the base class for mixins included in DefaultMeta was changed from object to type. Conclusion: code from master works in Python 3.8.4, but the last released version is older.

You can check this by installing from master (pip install git+git://github.com/pallets/flask-sqlalchemy.git) and then run problematic code - should work also in 3.8.4.

Please release new version and problem (here) will be gone.

[davidism]

Oh nice, thanks for reporting and checking the development version. If someone backports the fix to the 2.x branch, I can make a quick bugfix release.

Applying flake8 and flake8-bugbear is what caused us to start subclassing type. Code linting to the rescue!

[kam193]

I created PR: #854

[davidism]

2.4.4 released: https://pypi.org/project/Flask-SQLAlchemy/2.4.4/

[MartinThoma]

Wow, you guys are amazing! Thank you for resolving the issue that quickly!

[kam193]

I hope we rescued some developers a few hours of stopped deployments ;)

[decaz]

Python 3.8.5 was released and this regression (https://bugs.python.org/issue41295) was resolved there: https://docs.python.org/release/3.8.5/whatsnew/changelog.html#python-3-8-5-final.