TypeError: can't apply this __setattr__ to DefaultMeta object

[MartinThoma]

I don't know where this comes from. I have a small project where all versions are pinned and the Flask app runs in a Docker container. I just made an unrelated changes with the same versions and get this error. Does this look familiar?

Expected Behavior

Just running without crashes

from flask_sqlalchemy import SQLAlchemy

db = SQLAlchemy()

Actual Behavior

Crashes

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.8/site-packages/flask_sqlalchemy/__init__.py", line 716, in __init__
    self.Model = self.make_declarative_base(model_class, metadata)
  File "/usr/local/lib/python3.8/site-packages/flask_sqlalchemy/__init__.py", line 798, in make_declarative_base
    model.query_class = self.Query
  File "/usr/local/lib/python3.8/site-packages/sqlalchemy/ext/declarative/api.py", line 79, in __setattr__
    _add_attribute(cls, key, value)
  File "/usr/local/lib/python3.8/site-packages/sqlalchemy/ext/declarative/base.py", line 802, in _add_attribute
    type.__setattr__(cls, key, value)
TypeError: can't apply this __setattr__ to DefaultMeta object

Environment

• Python version: Python 3.8.4 (Docker python:3.8-slim-buster)
• Flask-SQLAlchemy version: flask-sqlalchemy==2.4.1
• SQLAlchemy version: sqlalchemy==1.3.13

[MartinThoma]

Interestingly, this works fine:

$ virtualenv --python=python3.8 venv
$ source venv/bin/activate
$ pip install flask-sqlalchemy==2.4.1 sqlalchemy==1.3.13
$ python
Python 3.8.1 (default, Feb 13 2020, 10:26:36) 
[GCC 7.5.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from flask_sqlalchemy import SQLAlchemy; db = SQLAlchemy()

[MartinThoma]

With 3.8.4rc1 from pyenv I don't get that issue on my host. Only within the docker container. Even when I create an empty folder within that Docker container. Even when I create a venv in an empty folder in that container:

(venv) root@ea40bbfdf33b:/opt/app/foo# python
Python 3.8.4 (default, Jul 14 2020, 03:07:46) 
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from flask_sqlalchemy import SQLAlchemy
>>> db = SQLAlchemy()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/opt/app/foo/venv/lib/python3.8/site-packages/flask_sqlalchemy/__init__.py", line 716, in __init__
    self.Model = self.make_declarative_base(model_class, metadata)
  File "/opt/app/foo/venv/lib/python3.8/site-packages/flask_sqlalchemy/__init__.py", line 798, in make_declarative_base
    model.query_class = self.Query
  File "/opt/app/foo/venv/lib/python3.8/site-packages/sqlalchemy/ext/declarative/api.py", line 79, in __setattr__
    _add_attribute(cls, key, value)
  File "/opt/app/foo/venv/lib/python3.8/site-packages/sqlalchemy/ext/declarative/base.py", line 802, in _add_attribute
    type.__setattr__(cls, key, value)
TypeError: can't apply this __setattr__ to DefaultMeta object

[piotrgredowski]

I've just met similar problem.
It happens only on Docker image python:3.8-slim-buster which now resolves to python:3.8.4-slim-buster (due to yesterday's release).
Changing Docker image to python:3.8.3-slim-buster solved problem for now. But I want to upgrade to 3.8.4 so it will be great if this issue is solved.

[kam193]

This problem is probably caused by changes in Python 3.8.4 (see https://docs.python.org/3/whatsnew/changelog.html#core-and-builtins)

[taryodor]

This issue is caused by this PR from Python 3.8.4:
https://github.com/python/cpython/pull/21288/files

To be more specific, the problem happens here:
https://github.com/python/cpython/pull/21288/files#diff-c3cf251f16d5a03a9e7d4639f2d6f998R5952

[MartinThoma]

I just confirmed @piotrgredowski hint: python:3.8.3-slim-buster works

[MartinThoma]

@taryodor Is this change expected to break backwards compatibility? If not, did somebody inform people at cpython already?

[davidism]

The file was same for 10 years and just now, someone decided it worked fine for too long.

@taryodor while I appreciate that this change may have negatively affected you, and do encourage you to report an issue to cpython, please avoid commentary like above when doing so. They clearly did have a reason in mind when changing it, saying this doesn't add anything towards identifying or fixing the issue.

[davidism]

I don't completely understand the reason referred to in the linked cpython change, so I'm not sure what the intended solution in our code is. In 2017, I made a change to construct the declarative base metaclass from some mixin classes to support customizing what features we add. SQLAlchemy is what overrides __setattr__ and uses type.__setattr__, which seems to be the thing that the change affects, but I don't think it's their issue either.

original cpython issue: https://bugs.python.org/issue39960
original cpython pr: python/cpython#21092

[davidism]

This comes from the line model.query_class = self.Query, which is assigning an extra attribute to the metaclass type, which is what the type.__setattr__ code is used for. It only happens when using our metaclass composed of mixins, if I use SQLAlchemy's DeclarativeMeta directly the assignment works. I have a feeling the cpython change needs to be adjusted, I'm really not sure what we could do differently in our code that wouldn't involve a massive rewrite.

Causes error:

from sqlalchemy.ext.declarative import declarative_base
from flask_sqlalchemy.model import DefaultMeta

Base = declarative_base(metaclass=DefaultMeta)
Base.test = True

Doesn't cause error:

from sqlalchemy.ext.declarative import declarative_base

Base = declarative_base()
Base.test = True

[davidism]

cc @zzzeek in case you see reports of this

[kam193]

I reported it in Python bug tracker: https://bugs.python.org/issue41295

But look carefully into report, and then check the flask-sqlalchemy code: the problem is caused, when a class inheritance from meta and non-metaclasses. And now, 2 months ago the base class for mixins included in DefaultMeta was changed from object to type. Conclusion: code from master works in Python 3.8.4, but the last released version is older.

You can check this by installing from master (pip install git+git://github.com/pallets/flask-sqlalchemy.git) and then run problematic code - should work also in 3.8.4.

Please release new version and problem (here) will be gone.

[davidism]

Oh nice, thanks for reporting and checking the development version. If someone backports the fix to the 2.x branch, I can make a quick bugfix release.

Applying flake8 and flake8-bugbear is what caused us to start subclassing type. Code linting to the rescue!