0.15.0 creates an issue during initialization (python 3.8, aws beanstalk graviton) #261

inbarshani · 2023-04-05T19:53:23Z

The recent release of flask-marshmallow caused my processes in AWS beanstalk to fail with the following errors:
| 2023-04-05T22:49:58.290+03:00 | Apr 5 19:49:58 ip-172-31-46-77 web: self.SQLAlchemySchema.OPTIONS_CLASS.session = db.session
| 2023-04-05T22:49:58.290+03:00 | Apr 5 19:49:58 ip-172-31-46-77 web: AttributeError: '_SQLAlchemyState' object has no attribute 'session'

The issue is resolved once I locked version 0.14.0 in my pipfile:
flask-marshmallow = "==0.14.0"

This is what I think is relevant out of my app init code:

def create_app(config_object):
    app = Flask(__name__)

    CORS(app, supports_credentials=True)
    app.config.from_object(config_object)
    from scripts import scripts
    register_extensions(app)
    register_blueprints(app)
    register_instrumentations(app)

    return app

def register_extensions(app):
    """Register Flask extensions."""
    app.json_encoder = json_encoder
    app.json_decoder = json_decoder
    bcrypt.init_app(app)
    db.init_app(app)
    migrate.init_app(app, db)
    ma.init_app(app)
    mail.init_app(app)
    jwt.init_app(app)

Fuller call stack:

2023-04-05T22:49:58.289+03:00 | Apr 5 19:49:58 ip-172-31-46-77 web: [2023-04-05 19:49:58 +0000] [16846] [ERROR] Exception in worker process
-- | --
  | 2023-04-05T22:49:58.289+03:00 | Apr 5 19:49:58 ip-172-31-46-77 web: Traceback (most recent call last):
  | 2023-04-05T22:49:58.289+03:00 | Apr 5 19:49:58 ip-172-31-46-77 web: File "/var/app/venv/staging-LQM1lest/lib64/python3.8/site-packages/gunicorn/arbiter.py", line 589, in spawn_worker
  | 2023-04-05T22:49:58.289+03:00 | Apr 5 19:49:58 ip-172-31-46-77 web: worker.init_process()
  | 2023-04-05T22:49:58.289+03:00 | Apr 5 19:49:58 ip-172-31-46-77 web: File "/var/app/venv/staging-LQM1lest/lib64/python3.8/site-packages/gunicorn/workers/gthread.py", line 92, in init_process
  | 2023-04-05T22:49:58.289+03:00 | Apr 5 19:49:58 ip-172-31-46-77 web: super().init_process()
  | 2023-04-05T22:49:58.289+03:00 | Apr 5 19:49:58 ip-172-31-46-77 web: File "/var/app/venv/staging-LQM1lest/lib64/python3.8/site-packages/gunicorn/workers/base.py", line 134, in init_process
  | 2023-04-05T22:49:58.289+03:00 | Apr 5 19:49:58 ip-172-31-46-77 web: self.load_wsgi()
  | 2023-04-05T22:49:58.289+03:00 | Apr 5 19:49:58 ip-172-31-46-77 web: File "/var/app/venv/staging-LQM1lest/lib64/python3.8/site-packages/gunicorn/workers/base.py", line 146, in load_wsgi
  | 2023-04-05T22:49:58.289+03:00 | Apr 5 19:49:58 ip-172-31-46-77 web: self.wsgi = self.app.wsgi()
  | 2023-04-05T22:49:58.289+03:00 | Apr 5 19:49:58 ip-172-31-46-77 web: File "/var/app/venv/staging-LQM1lest/lib64/python3.8/site-packages/gunicorn/app/base.py", line 67, in wsgi
  | 2023-04-05T22:49:58.289+03:00 | Apr 5 19:49:58 ip-172-31-46-77 web: self.callable = self.load()
  | 2023-04-05T22:49:58.289+03:00 | Apr 5 19:49:58 ip-172-31-46-77 web: File "/var/app/venv/staging-LQM1lest/lib64/python3.8/site-packages/gunicorn/app/wsgiapp.py", line 58, in load
  | 2023-04-05T22:49:58.289+03:00 | Apr 5 19:49:58 ip-172-31-46-77 web: return self.load_wsgiapp()
  | 2023-04-05T22:49:58.290+03:00 | Apr 5 19:49:58 ip-172-31-46-77 web: File "/var/app/venv/staging-LQM1lest/lib64/python3.8/site-packages/gunicorn/app/wsgiapp.py", line 48, in load_wsgiapp
  | 2023-04-05T22:49:58.290+03:00 | Apr 5 19:49:58 ip-172-31-46-77 web: return util.import_app(self.app_uri)
  | 2023-04-05T22:49:58.290+03:00 | Apr 5 19:49:58 ip-172-31-46-77 web: File "/var/app/venv/staging-LQM1lest/lib64/python3.8/site-packages/gunicorn/util.py", line 359, in import_app
  | 2023-04-05T22:49:58.290+03:00 | Apr 5 19:49:58 ip-172-31-46-77 web: mod = importlib.import_module(module)
  | 2023-04-05T22:49:58.290+03:00 | Apr 5 19:49:58 ip-172-31-46-77 web: File "/usr/lib64/python3.8/importlib/__init__.py", line 127, in import_module
  | 2023-04-05T22:49:58.290+03:00 | Apr 5 19:49:58 ip-172-31-46-77 web: return _bootstrap._gcd_import(name[level:], package, level)
  | 2023-04-05T22:49:58.290+03:00 | Apr 5 19:49:58 ip-172-31-46-77 web: File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  | 2023-04-05T22:49:58.290+03:00 | Apr 5 19:49:58 ip-172-31-46-77 web: File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  | 2023-04-05T22:49:58.290+03:00 | Apr 5 19:49:58 ip-172-31-46-77 web: File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  | 2023-04-05T22:49:58.290+03:00 | Apr 5 19:49:58 ip-172-31-46-77 web: File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  | 2023-04-05T22:49:58.290+03:00 | Apr 5 19:49:58 ip-172-31-46-77 web: File "<frozen importlib._bootstrap_external>", line 843, in exec_module
  | 2023-04-05T22:49:58.290+03:00 | Apr 5 19:49:58 ip-172-31-46-77 web: File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  | 2023-04-05T22:49:58.290+03:00 | Apr 5 19:49:58 ip-172-31-46-77 web: File "/var/app/current/run.py", line 8, in <module>
  | 2023-04-05T22:49:58.290+03:00 | Apr 5 19:49:58 ip-172-31-46-77 web: application = create_app(Config)
  | 2023-04-05T22:49:58.290+03:00 | Apr 5 19:49:58 ip-172-31-46-77 web: File "/var/app/current/api/app.py", line 40, in create_app
  | 2023-04-05T22:49:58.290+03:00 | Apr 5 19:49:58 ip-172-31-46-77 web: register_extensions(app)
  | 2023-04-05T22:49:58.290+03:00 | Apr 5 19:49:58 ip-172-31-46-77 web: File "/var/app/current/api/app.py", line 67, in register_extensions
  | 2023-04-05T22:49:58.290+03:00 | Apr 5 19:49:58 ip-172-31-46-77 web: ma.init_app(app)
  | 2023-04-05T22:49:58.290+03:00 | Apr 5 19:49:58 ip-172-31-46-77 web: File "/var/app/venv/staging-LQM1lest/lib64/python3.8/site-packages/flask_marshmallow/__init__.py", line 116, in init_app
  | 2023-04-05T22:49:58.290+03:00 | Apr 5 19:49:58 ip-172-31-46-77 web: self.SQLAlchemySchema.OPTIONS_CLASS.session = db.session
  | 2023-04-05T22:49:58.290+03:00 | Apr 5 19:49:58 ip-172-31-46-77 web: AttributeError: '_SQLAlchemyState' object has no attribute 'session'
  | 2023-04-05T22:49:58.791+03:00 | Apr 5 19:49:58 ip-172-31-46-77 web: [2023-04-05 19:49:58 +0000] [16846] [INFO] Worker exiting (pid: 16846)
  | 2023-04-05T22:49:58.791+03:00 | Apr 5 19:49:58 ip-172-31-46-77 web: [2023-04-05 19:49:58 +0000] [16840] [INFO] Shutting down: Master

The text was updated successfully, but these errors were encountered:

Sillocan · 2023-04-06T00:00:31Z

Seeing the same issue on our end.

sirosen · 2023-04-06T02:31:04Z

Hi there, thanks for reporting this!
I want to note up front that 0.15.0 is the first release in a couple of years, so there's a lot of potential drift to account for, as other packages have updated over time. I'll be trying to understand and fix issues as quickly as I can, but it may take some time.

Could you also let me know the version numbers of the relevant packages you're using?
In particular, versions of marshmallow, flask-sqlalchemy (if applicable), marshmallow-sqlalchemy, and sqlalchemy strike me as relevant. It's best to work out how to reproduce the issue before I try making any adjustments to fix it.

ChriZzn · 2023-04-06T17:06:01Z

Hi @sirosen,

i have the same Problem, i think it is related with this Update: https://github.com/sqlalchemy/sqlalchemy/releases/tag/rel_2_0_9

Because this change got also Backported to v1.4.48.

Sillocan · 2023-04-06T18:12:41Z

marshmallow-3.19.0
flask_marshmallow-0.15.0
Flask_SQLAlchemy-2.5.1
marshmallow_sqlalchemy-0.29.0
SQLAlchemy-1.4.47

inbarshani · 2023-04-06T20:24:35Z

[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"

[packages]
atomicwrites = "*"
boto3="*"
autopep8 = "*"
colorama = "*"
croniter = "*"
firebase-admin = "*"
flask = "==2.0.*"
flask-cors = "*"
flask-mail = "*"
flask-marshmallow = "==0.14.0"
flask-migrate = "*"
flask-sqlalchemy = "==2.5.1"
hubspot3 = "*"
intuit-oauth = "*"
marshmallow-sqlalchemy = "*"
mock = "*"
pdfkit = "*"
psycopg2-binary = "==2.9.5"
pylint = "*"
pytest = "==6.2.0"
python-dotenv = "~=0.21.1"
pytz = "*"
requests = "*"
shapely = "==1.7.1"
soracom = "==0.0.1.post0"
sqlalchemy = "==1.4.46"
coverage-threshold = "==0.4.4"
coverage = "==6.3.2"
orjson = "~=3.7.7"
deepdiff = "~=5.8.1"
flask-jwt-extended = "~=4.4.4"
werkzeug = "~=2.0.1"
flask-bcrypt = "*"
pytest-cov = "~=4.0.0"
pytest-xdist = "~=3.2.1"
filelock = "==3.9.0"
sqlalchemycollector = "*"

[dev-packages]
flake8 = "*"

[requires]
python_version = "3.8"

sirosen · 2023-04-08T04:30:32Z

It seems very likely that most of these issues stem from the fact that support for flask-sqlalchemy<3 was dropped. This was not correctly documented in the changelog before (sorry!), which has been fixed in the repo and will update on readthedocs with the next release.

There are two comments here which indicate flask-sqlalchemy v2, which supports this guess.

I have pretty limited availability the next few days to respond or work on things, so I'm just going to brain dump some thoughts (hopefully informative and uncontroversial):

this is not a critical issue, since users can easily pin to v0.14.0 until they're ready to upgrade
there's no plan to support flask-sqlalchemy v2 in a future release. I would only consider it in extraordinary circumstances -- my intent is to get this library on healthy footing again, including reasonable lower bounds for dependencies
upgrading to flask-sqlalchemy v3 is a pretty minor matter for most apps -- I recently handled this at work and it was a chore but not a hard one -- and is my recommendation to most users
as a general recommendation, you should be pinning dependencies for any deployed production applications (pdm, poetry, pipenv, pip-tools, etc)

That last point is not meant to put blame on anyone seeing problems -- if someone is having a bad time because this broke their app I'm genuinely sorry -- but to help make sure that folks walk away with the right conclusions. If you're not pinning dependencies, you're going to experience breakage sooner or later.

sirosen · 2023-04-13T14:57:43Z

I've been able to reproduce the initial report using flask-sqlalchemy 2.5.1, and confirm that it's fixed with flask-sqlalchemy 3.0.3.
As such, I'm going to close this as a (fixed) documentation issue about ending support for flask-sqlalchemy 2.x

Here's my reproducer for reference:

import flask

from flask_marshmallow import Marshmallow
from flask_sqlalchemy import SQLAlchemy


db = SQLAlchemy()
ma = Marshmallow()


def create_app() -> flask.Flask:
    app = flask.Flask("extapp")
    app.config["SQLALCHEMY_DATABASE_URI"] = "sqlite:///:memory:"
    db.init_app(app)
    ma.init_app(app)
    return app


app = create_app()

sirosen added the bug label Apr 6, 2023

sirosen self-assigned this Apr 6, 2023

sirosen closed this as completed Apr 13, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

0.15.0 creates an issue during initialization (python 3.8, aws beanstalk graviton) #261

0.15.0 creates an issue during initialization (python 3.8, aws beanstalk graviton) #261

inbarshani commented Apr 5, 2023 •

edited

Loading

Sillocan commented Apr 6, 2023

sirosen commented Apr 6, 2023

ChriZzn commented Apr 6, 2023

Sillocan commented Apr 6, 2023

inbarshani commented Apr 6, 2023 •

edited

Loading

sirosen commented Apr 8, 2023

sirosen commented Apr 13, 2023

0.15.0 creates an issue during initialization (python 3.8, aws beanstalk graviton) #261

0.15.0 creates an issue during initialization (python 3.8, aws beanstalk graviton) #261

Comments

inbarshani commented Apr 5, 2023 • edited Loading

Sillocan commented Apr 6, 2023

sirosen commented Apr 6, 2023

ChriZzn commented Apr 6, 2023

Sillocan commented Apr 6, 2023

inbarshani commented Apr 6, 2023 • edited Loading

sirosen commented Apr 8, 2023

sirosen commented Apr 13, 2023

inbarshani commented Apr 5, 2023 •

edited

Loading

inbarshani commented Apr 6, 2023 •

edited

Loading