Issue: Fatal Python error: Cannot recover from stack overflow.

[imadmoussa1]

First check

• I added a very descriptive title to this issue.
• I used the GitHub search to find a similar issue and didn't find it.
• I searched the FastAPI documentation, with the integrated search.
• I already searched in Google "How to X in FastAPI" and didn't find any information.
• I already read and followed all the tutorials in the docs and didn't find an answer.
• I already checked if it is not related to FastAPI but to Pydantic.
• I already checked if it is not related to FastAPI but to Swagger UI.
• I already checked if it is not related to FastAPI but to ReDoc.
• After submitting this, I commit to one of:
  • Read open issues with questions until I find 2 issues where I can help someone and add a comment to help there.
  • I already hit the "watch" button in this repository to receive notifications and I commit to help at least 2 people that ask

Description

I have this issue on my QA server, but I was not able to replicate it locally.
My code based on the latest version of https://github.com/tiangolo/full-stack-fastapi-postgresql using the base class in CRUD,
this error happened on the update function

Fatal Python error: Cannot recover from stack overflow.

Thread 0x00007f615d0c0700 (most recent call first):
  File "/usr/local/lib/python3.7/concurrent/futures/thread.py", line 78 in _worker
  File "/usr/local/lib/python3.7/threading.py", line 870 in run
  File "/usr/local/lib/python3.7/threading.py", line 926 in _bootstrap_inner
  File "/usr/local/lib/python3.7/threading.py", line 890 in _bootstrap

Thread 0x00007f615d8c1700 (most recent call first):
  File "/usr/local/lib/python3.7/concurrent/futures/thread.py", line 78 in _worker
  File "/usr/local/lib/python3.7/threading.py", line 870 in run
  File "/usr/local/lib/python3.7/threading.py", line 926 in _bootstrap_inner
  File "/usr/local/lib/python3.7/threading.py", line 890 in _bootstrap

Current thread 0x00007f615e19f700 (most recent call first):
  File "<string>", line 1 in <lambda>
  File "/usr/local/lib/python3.7/site-packages/sqlalchemy/sql/operators.py", line 432 in __getitem__

Environment

• OS: [e.g. Linux / Windows / macOS]: ubuntu
• FastAPI Version [e.g. 0.3.0]: 0.61.0
• Python version: 3.7

Additional context

  def update(self, db: Session, *, db_obj: ModelType, obj_in: Union[UpdateSchemaType, Dict[str, Any]]) -> ModelType:
    obj_data = jsonable_encoder(db_obj)
    if isinstance(obj_in, dict):
      update_data = obj_in
    else:
      update_data = obj_in.dict(exclude_unset=True)
    for field in obj_data:
      if field in update_data:
        setattr(db_obj, field, update_data[field])
    db.add(db_obj)
    db.commit()
    db.refresh(db_obj)
    return db_obj

[Kludex]

This was caused by the new release (0.61.0) (jsonable_encoder).

The problem is the line that contains: obj_data = jsonable_encoder(db_obj)

Possible solution:

# Your function
  def update(self, db: Session, *, db_obj: ModelType, obj_in: Union[UpdateSchemaType, Dict[str, Any]]) -> ModelType:
    obj_data = db_obj.as_dict() # HERE!
    if isinstance(obj_in, dict):
      update_data = obj_in
    else:
      update_data = obj_in.dict(exclude_unset=True)
    for field in obj_data:
      if field in update_data:
        setattr(db_obj, field, update_data[field])
    db.add(db_obj)
    db.commit()
    db.refresh(db_obj)
    return db_obj

# SQLAlchemy Base object
@as_declarative()
class Base:
    def as_dict(self) -> dict:
        return {c.key: getattr(self, c.key) for c in inspect(self).mapper.column_attrs}

But I don't know why we have this error message, I need to check.

EDIT: As reminded by @imadmoussa1:

from sqlalchemy import inspect

[anh-collab]

I have the same FATAL errors when running normal select statement with databases/encode.

async def list_creators(
    skip: int = 0,
    limit: int = 0,
    manager_id: Optional[int] = None,
    mapping_group_id: Optional[int] = None,
):
    # basic select statement
    base_query = models.Creator.__table__.select()
    # if input manager_id
    # add where condition of mapping_groups with manager_id
    if manager_id:
        base_query = base_query.where(
            models.Creator.mapping_group_id.in_(
                select([column("mapping_group_id")])
                .select_from(models.MappingGroupMember.__table__)
                .where(models.MappingGroupMember.manager_id == manager_id)
            )
        )
    # if mapping_group_id
    # add where condition of mapping_groups with mapping_group_id
    if mapping_group_id:
        base_query = base_query.where(
            models.Creator.mapping_group_id == mapping_group_id
        )
    # add skip and limit
    query = base_query.offset(skip).limit(limit) if limit else base_query.offset(skip)
    return await database.fetch_all(query)

@router.get(
    "/list",
)
async def read_creators(
    skip: int = 0,
    limit: int = 100,
    manager_id: Optional[int] = None,
    mapping_group_id: Optional[int] = None,
):
    if manager_id and mapping_group_id:
        raise HTTPException(
            status_code=400,
            detail="Please do not provide manager_id and mapping_group_id at the same time",
        )
    return await crud.list_creators(
        skip=skip, limit=limit, manager_id=manager_id, mapping_group_id=mapping_group_id
    )

Any help would be appreciated. This did not happen before.

[duz-sg]

Same issue here, this crash does not give any trace info, took some time to trace the offending code, as outlined earlier:
https://github.com/tiangolo/full-stack-fastapi-postgresql/blob/master/%7B%7Bcookiecutter.project_slug%7D%7D/backend/app/app/crud/base.py#L49

[imadmoussa1]

@Kludex Thank you, Your solution fixed my problem.

[Kludex]

You're welcome! @imadmoussa1 Do you mind closing the issue then? 😎 🎉

[sv8083]

def as_dict(self) -> dict:
        return {c.key: getattr(self, c.key) for c in inspect(self).mapper.column_attrs}

Hi @Kludex
I applied the fix, but it throws me below error:

  File ".\crud\base.py", line 49, in update
    obj_data = db_obj.as_dict()
  File ".\db\base_class.py", line 15, in as_dict
    return {c.key: getattr(self, c.key) for c in inspect(self).mapper.column_attrs}
TypeError: 'module' object is not callable

[imadmoussa1]

Hi @sv8083 you should import the inspect from sqlalchemy

from sqlalchemy import inspect

[plankter]

Does it mean jsonable_encoder is broken? It is unclear why the issue was closed.

[imadmoussa1]

@plankter jsonable_encoder is not broken, I am using it in different functions, but on update function we had errors. fixed by replacing the jsonable_encoder with a custom function.

[duz-sg]

This fix does not help in my case, after applying the above fix, I still get the error message:

Fatal Python error: Cannot recover from stack overflow.

Thread 0x00007fa28e7bc700 (most recent call first):
  File "/usr/local/lib/python3.7/concurrent/futures/thread.py", line 78 in _worker
  File "/usr/local/lib/python3.7/threading.py", line 870 in run
  File "/usr/local/lib/python3.7/threading.py", line 926 in _bootstrap_inner
  File "/usr/local/lib/python3.7/threading.py", line 890 in _bootstrap

Thread 0x00007fa28effd700 (most recent call first):
  File "/usr/local/lib/python3.7/concurrent/futures/thread.py", line 78 in _worker
  File "/usr/local/lib/python3.7/threading.py", line 870 in run
  File "/usr/local/lib/python3.7/threading.py", line 926 in _bootstrap_inner
  File "/usr/local/lib/python3.7/threading.py", line 890 in _bootstrap

Current thread 0x00007fa28f7fe700 (most recent call first):
  File "/usr/local/lib/python3.7/site-packages/sqlalchemy/sql/type_api.py", line 67 in operate
  File "<string>", line 1 in <lambda>
  File "/usr/local/lib/python3.7/site-packages/sqlalchemy/sql/operators.py", line 432 in __getitem__
  File "/usr/local/lib/python3.7/site-packages/sqlalchemy/sql/elements.py", line 762 in operate

Update, fixed my issue:

    check_item = crud.item.get_by_user(
        db=db, id=item_id, user_id=current_user.id
    )
    pd_check_item = ItemCheck.from_orm(check_item) # Fix: convert from DB model to pydantic model 
    check_data = jsonable_encoder(pd_check_item, exclude_none=True) # Breaks if accept db model directly
    return check_data

It seems jsonable_encoder can no longer accept a DB model and convert the data, it only accepts a pydantic model.

[shawnwall]

Another approach I was trying locally was to use load a pydantic model from the orm model, get the dict(), then iterate that, such as:

obj_data = UserInDB.from_orm(db_obj).dict()

...now I know that doesnt perfectly fit into CRUDBase unless we refactor it to also hold onto a DBSchemaType in addition to CreateSchemaType and UpdateSchemaType

[l1725821492]

This fix does not help in my case, after applying the above fix, I still get the error message:

Fatal Python error: Cannot recover from stack overflow.

Thread 0x00007fa28e7bc700 (most recent call first):
  File "/usr/local/lib/python3.7/concurrent/futures/thread.py", line 78 in _worker
  File "/usr/local/lib/python3.7/threading.py", line 870 in run
  File "/usr/local/lib/python3.7/threading.py", line 926 in _bootstrap_inner
  File "/usr/local/lib/python3.7/threading.py", line 890 in _bootstrap

Thread 0x00007fa28effd700 (most recent call first):
  File "/usr/local/lib/python3.7/concurrent/futures/thread.py", line 78 in _worker
  File "/usr/local/lib/python3.7/threading.py", line 870 in run
  File "/usr/local/lib/python3.7/threading.py", line 926 in _bootstrap_inner
  File "/usr/local/lib/python3.7/threading.py", line 890 in _bootstrap

Current thread 0x00007fa28f7fe700 (most recent call first):
  File "/usr/local/lib/python3.7/site-packages/sqlalchemy/sql/type_api.py", line 67 in operate
  File "<string>", line 1 in <lambda>
  File "/usr/local/lib/python3.7/site-packages/sqlalchemy/sql/operators.py", line 432 in __getitem__
  File "/usr/local/lib/python3.7/site-packages/sqlalchemy/sql/elements.py", line 762 in operate

Update, fixed my issue:

    check_item = crud.item.get_by_user(
        db=db, id=item_id, user_id=current_user.id
    )
    pd_check_item = ItemCheck.from_orm(check_item) # Fix: convert from DB model to pydantic model 
    check_data = jsonable_encoder(pd_check_item, exclude_none=True) # Breaks if accept db model directly
    return check_data

It seems jsonable_encoder can no longer accept a DB model and convert the data, it only accepts a pydantic model.

This fix does not help in my case, after applying the above fix, I still get the error message:

Fatal Python error: Cannot recover from stack overflow.

Thread 0x00007fa28e7bc700 (most recent call first):
  File "/usr/local/lib/python3.7/concurrent/futures/thread.py", line 78 in _worker
  File "/usr/local/lib/python3.7/threading.py", line 870 in run
  File "/usr/local/lib/python3.7/threading.py", line 926 in _bootstrap_inner
  File "/usr/local/lib/python3.7/threading.py", line 890 in _bootstrap

Thread 0x00007fa28effd700 (most recent call first):
  File "/usr/local/lib/python3.7/concurrent/futures/thread.py", line 78 in _worker
  File "/usr/local/lib/python3.7/threading.py", line 870 in run
  File "/usr/local/lib/python3.7/threading.py", line 926 in _bootstrap_inner
  File "/usr/local/lib/python3.7/threading.py", line 890 in _bootstrap

Current thread 0x00007fa28f7fe700 (most recent call first):
  File "/usr/local/lib/python3.7/site-packages/sqlalchemy/sql/type_api.py", line 67 in operate
  File "<string>", line 1 in <lambda>
  File "/usr/local/lib/python3.7/site-packages/sqlalchemy/sql/operators.py", line 432 in __getitem__
  File "/usr/local/lib/python3.7/site-packages/sqlalchemy/sql/elements.py", line 762 in operate

Update, fixed my issue:

    check_item = crud.item.get_by_user(
        db=db, id=item_id, user_id=current_user.id
    )
    pd_check_item = ItemCheck.from_orm(check_item) # Fix: convert from DB model to pydantic model 
    check_data = jsonable_encoder(pd_check_item, exclude_none=True) # Breaks if accept db model directly
    return check_data

It seems jsonable_encoder can no longer accept a DB model and convert the data, it only accepts a pydantic model.

How to import ItemCheck