SQLAlchemy-Utils UUIDType `TypeError: string argument without an encoding` #635

NightSkySK · 2023-09-30T08:36:42Z

Checklist

The bug is reproducible against the latest release or master.
There are no similar issues or pull requests to fix it yet.

Describe the bug

Since I've changed company_uuid from str to UUID type I get following error once I want to get detailed view or edit record:

  File "C:\mSpaceLock\mSpaceLock-www\env\Lib\site-packages\sqladmin\helpers.py", line 234, in object_identifier_values
    values.append(get_column_python_type(pk)(part))
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: string argument without an encoding

SQLAlchemy model:

class Company(Base):
    company_uuid: Mapped[UUID] = mapped_column(
        UUIDType(binary=False),
        primary_key=True,
        index=True,
        default=uuid4,
    )
    company_name: Mapped[str] = mapped_column(String, unique=True, nullable=False)
    address: Mapped[Optional[str]]
    city: Mapped[Optional[str]]
    country: Mapped[Optional[str]] = mapped_column(CountryType)
    tax_id: Mapped[Optional[str]] = mapped_column(default=None)
    contact_details: Mapped[Optional[str]] = mapped_column(Text, default=None)
    is_active: Mapped[bool] = mapped_column(default=True)
    location: Mapped[List["Location"]] = relationship(
        Location,
        back_populates="company",
    )
    user: Mapped[List["User"]] = relationship(User, back_populates="company")

    def __str__(self) -> str:
        return self.company_name

from admin.views.registry import register_view
from db.models.companies import Company
from sqladmin import ModelView


@register_view
class CompanyAdmin(ModelView, model=Company):
    column_list = [
        Company.company_name,
        Company.address,
        Company.city,
        Company.country,
        Company.tax_id,
        Company.is_active,
        Company.contact_details,
    ]
    form_columns = [
        Company.company_name,
        Company.address,
        Company.city,
        Company.country,
        Company.tax_id,
        Company.is_active,
        Company.contact_details,
    ]
    name = "Company"
    name_plural = "Companies"
    icon = "fa-solid fa-building"

At first I thought that error is caused because of binary format UUID and added UUIDType(binary=False) but it didn't change anything also adding to main:

sys.stdin.reconfigure(encoding="utf-8")
sys.stdout.reconfigure(encoding="utf-8")

or PYTHONIOENCODING = 'uft-8' to environmental variables didn't help a bit.

Steps to reproduce the bug

Create model with primary key
Mapped[UUID] = mapped_column(UUIDType(binary=False), primary_key=True, index=True, default=uuid4)
Initialize database with initialdata.
Create ModelView for this table
Access http://127.0.0.1:8000/admin/{modelname}/list --> all looks ok
Try to View or Edit to show detailed view or form --> TypeError: string argument without an encoding

Expected behavior

No response

Actual behavior

No response

Debugging material

INFO:     127.0.0.1:50349 - "GET /company/details/9d97ad6c-6c62-4ea1-8e65-53b861b90601 HTTP/1.1" 500 Internal Server Error
ERROR:    Exception in ASGI application
Traceback (most recent call last):
  File "C:\mSpaceLock\mSpaceLock-www\env\Lib\site-packages\uvicorn\protocols\http\h11_impl.py", line 408, in run_asgi
    result = await app(  # type: ignore[func-returns-value]
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\mSpaceLock\mSpaceLock-www\env\Lib\site-packages\uvicorn\middleware\proxy_headers.py", line 84, in __call__
    return await self.app(scope, receive, send)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\mSpaceLock\mSpaceLock-www\env\Lib\site-packages\fastapi\applications.py", line 292, in __call__
    await super().__call__(scope, receive, send)
  File "C:\mSpaceLock\mSpaceLock-www\env\Lib\site-packages\starlette\applications.py", line 122, in __call__
    await self.middleware_stack(scope, receive, send)
  File "C:\mSpaceLock\mSpaceLock-www\env\Lib\site-packages\starlette\middleware\errors.py", line 184, in __call__
    raise exc
  File "C:\mSpaceLock\mSpaceLock-www\env\Lib\site-packages\starlette\middleware\errors.py", line 162, in __call__
    await self.app(scope, receive, _send)
  File "C:\mSpaceLock\mSpaceLock-www\env\Lib\site-packages\starlette\middleware\exceptions.py", line 79, in __call__
    raise exc
  File "C:\mSpaceLock\mSpaceLock-www\env\Lib\site-packages\starlette\middleware\exceptions.py", line 68, in __call__
    await self.app(scope, receive, sender)
  File "C:\mSpaceLock\mSpaceLock-www\env\Lib\site-packages\fastapi\middleware\asyncexitstack.py", line 20, in __call__
    raise e
  File "C:\mSpaceLock\mSpaceLock-www\env\Lib\site-packages\fastapi\middleware\asyncexitstack.py", line 17, in __call__
    await self.app(scope, receive, send)
  File "C:\mSpaceLock\mSpaceLock-www\env\Lib\site-packages\starlette\routing.py", line 718, in __call__
    await route.handle(scope, receive, send)
  File "C:\mSpaceLock\mSpaceLock-www\env\Lib\site-packages\starlette\routing.py", line 443, in handle
    await self.app(scope, receive, send)
  File "C:\mSpaceLock\mSpaceLock-www\env\Lib\site-packages\starlette\applications.py", line 122, in __call__
    await self.middleware_stack(scope, receive, send)
  File "C:\mSpaceLock\mSpaceLock-www\env\Lib\site-packages\starlette\middleware\errors.py", line 184, in __call__
    raise exc
  File "C:\mSpaceLock\mSpaceLock-www\env\Lib\site-packages\starlette\middleware\errors.py", line 162, in __call__
    await self.app(scope, receive, _send)
  File "C:\mSpaceLock\mSpaceLock-www\env\Lib\site-packages\starlette\middleware\sessions.py", line 86, in __call__
    await self.app(scope, receive, send_wrapper)
  File "C:\mSpaceLock\mSpaceLock-www\env\Lib\site-packages\starlette\middleware\exceptions.py", line 79, in __call__
    raise exc
  File "C:\mSpaceLock\mSpaceLock-www\env\Lib\site-packages\starlette\middleware\exceptions.py", line 68, in __call__
    await self.app(scope, receive, sender)
  File "C:\mSpaceLock\mSpaceLock-www\env\Lib\site-packages\starlette\routing.py", line 718, in __call__
    await route.handle(scope, receive, send)
  File "C:\mSpaceLock\mSpaceLock-www\env\Lib\site-packages\starlette\routing.py", line 276, in handle
    await self.app(scope, receive, send)
  File "C:\mSpaceLock\mSpaceLock-www\env\Lib\site-packages\starlette\routing.py", line 66, in app
    response = await func(request)
               ^^^^^^^^^^^^^^^^^^^
  File "C:\mSpaceLock\mSpaceLock-www\env\Lib\site-packages\sqladmin\authentication.py", line 66, in wrapper_decorator
    return await func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\mSpaceLock\mSpaceLock-www\env\Lib\site-packages\sqladmin\application.py", line 454, in details
    model = await model_view.get_object_for_details(request.path_params["pk"])
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\mSpaceLock\mSpaceLock-www\env\Lib\site-packages\sqladmin\models.py", line 834, in get_object_for_details
    stmt = self._stmt_by_identifier(value)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\mSpaceLock\mSpaceLock-www\env\Lib\site-packages\sqladmin\models.py", line 856, in _stmt_by_identifier
    values = object_identifier_values(identifier, self.model)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\mSpaceLock\mSpaceLock-www\env\Lib\site-packages\sqladmin\helpers.py", line 234, in object_identifier_values
    values.append(get_column_python_type(pk)(part))
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: string argument without an encoding

Environment

OS Windows 10
Python 3.11.5
pip.txt

fastapi 0.103.1
pydantic 2.3.0
pydantic_core 2.6.3
sqladmin 0.15.0
SQLAlchemy 2.0.21
SQLAlchemy-Utils 0.41.1

uvicorn 0.23.2
WTForms 3.0.1

Additional context

No response

The text was updated successfully, but these errors were encountered:

aminalaee · 2023-09-30T11:33:38Z

Where is the UUIDType coming from? I think it should be from sqlalchemy import Uuid probably.

NightSkySK · 2023-09-30T11:39:50Z

It's comming from SQLAlchemy-Utils

from sqlalchemy_utils import CountryType, UUIDType

I found in the change log that this tool should be working with SQLAdmin:

Add support for UUIDType from sqlalchemy_utils by @okapies in #183

Also there is such code in sqladmin/sqladmin
/forms.py

    @converts(
        "sqlalchemy.dialects.postgresql.base.UUID",
        "sqlalchemy.sql.sqltypes.UUID",
        "sqlalchemy.sql.sqltypes.Uuid",
        "sqlalchemy_utils.types.uuid.UUIDType",
    )
    def conv_uuid(
        self, model: type, prop: ColumnProperty, kwargs: Dict[str, Any]
    ) -> UnboundField:
        kwargs.setdefault("validators", [])
        kwargs["validators"].append(validators.UUID())
        return StringField(**kwargs)

BhuwanPandey · 2023-10-11T11:13:12Z

I think that this problem raised due to issue on sqlalchemy_utils package such as

In helper.py file of sqladmin there is method that check wheather impl attribute of column.type or not

def get_column_python_type(column: Column) -> type:
    try:
        if hasattr(column.type, "impl"):
            return column.type.impl.python_type
        
        return column.type.python_type
    except NotImplementedError:
        return str

On using sqlalchemy_utils for uuid handling ,Here we used UUIDType where binary=False is passed

Mapped[UUID] = mapped_column(UUIDType(binary=False), primary_key=True, index=True, default=uuid4)

whether, we passed binary= False or binary=True , UUIDType class of (python_type attribute ) always return bytes.

on this code
values.append(get_column_python_type(pk)(part))

Bytes class is called with uuid as argument ,where
get_column_python_type(pk) return Bytes class
part is uuid value

so that, these error occur on using with UUIDTypes.

But, Is there any way to bypass these scenario from sqladmin side.

aminalaee · 2023-10-11T12:11:59Z

I think you are correct in the debugging, but I think a good step is to compare https://sqlalchemy-utils.readthedocs.io/en/latest/_modules/sqlalchemy_utils/types/uuid.html#UUIDType with sqlalchemy UUID type because I don't see that one raising any issues.
My first understanding was that the sqlalchemy_utils UUID type should return different impl if binary status changes.

BhuwanPandey · 2023-10-11T12:38:16Z

I think you are correct in the debugging, but I think a good step is to compare https://sqlalchemy-utils.readthedocs.io/en/latest/_modules/sqlalchemy_utils/types/uuid.html#UUIDType with sqlalchemy UUID type because I don't see that one raising any issues. My first understanding was that the sqlalchemy_utils UUID type should return different impl if binary status changes.

Yes, I was thinking so, impl value should be different based on binary status.

I was expecting , it should be like

from pydantic import UUID4

class UUIDType(ScalarCoercible, types.TypeDecorator):
    """
    Stores a UUID in the database natively when it can and falls back to
    a BINARY(16) or a CHAR(32) when it can't.

    ::

        from sqlalchemy_utils import UUIDType
        import uuid

        class User(Base):
            __tablename__ = 'user'

            # Pass `binary=False` to fallback to CHAR instead of BINARY
            id = sa.Column(
                UUIDType(binary=False),
                primary_key=True,
                default=uuid.uuid4
            )
    """
        class UUIDChar(CHAR):
                python_type = UUID4  # type: 

    impl = types.BINARY(16)

    python_type = uuid.UUID

    cache_ok = True

    def __init__(self, binary=True, native=True):
        """
        :param binary: Whether to use a BINARY(16) or CHAR(32) fallback.
        """
        self.binary = binary
        self.native = native
        if not self.binary:
              self.impl =  self.UUIDChar

by changing like this, it works on my computer locally.
I take this reference from fastapi_users db sqlalchemy

aminalaee · 2023-10-11T12:42:09Z

Or even better it would be in load_dialect_impl which is specifically for this purpose to return impl dynamically. But I'm not really sure if this was changed in sqlalchemy_utils recently or even if it's a bug there.

BhuwanPandey · 2023-10-11T13:13:42Z

this wasnot changed in sqlalchemy_utils currently. Once I decide to push the changes and leave it for review. But, problem is that I havenot found pydantic was used on sqlalchemy_utils . currently i leave it by creating issue on sqlqlchemy_utils.

and, what do you think. How this code handle if binary = True in UUIDTypes.
means
get_column_python_type(pk) return bytes class

values.append(get_column_python_type(pk)(part))

I think these raise same error because for bytes data to decode , we have to call decode method of bytes class.

guilhermelou · 2023-10-12T21:14:07Z

Why are we returning first the impl.python_type and not python_type directly in the first place?

So, instead of this:

def get_column_python_type(column: Column) -> type:
    try:
        if hasattr(column.type, "impl"):
            return column.type.impl.python_type
        return column.type.python_type
    except NotImplementedError:
        return str

Perhaps doing something like this:

def get_column_python_type(column: Column) -> type:
    try:
        return column.type.python_type
    except NotImplementedError:
        if hasattr(column.type, "impl"):
            try:
                return column.type.impl.python_type
            except NotImplementedError:
                pass
        return str

aminalaee · 2023-11-09T16:39:10Z

I don't think that's the same problem, the error clearly is different and you are not using sqlalchemy_utils. I think you should open a different issue.

greatlaki · 2023-11-09T17:28:34Z

I don't think that's the same problem, the error clearly is different and you are not using sqlalchemy_utils. I think you should open a different issue.

Okay, I did it

breduin · 2024-02-28T15:35:19Z

Hi! I faced with a problem I want to edit or delete an instance of the User model in the admin panel, but I got that error: TypeError: UUID objects are immutable The User model from fastapi_users lib:

I've got the same problem. Have you succeeded?

Kumzy · 2024-05-06T09:25:10Z

Hello,
I wanted to know if there is any news regarding this issue? I am encountering the same thing at the moment.

Thanks

aminalaee · 2024-05-07T07:42:18Z

This seems like a good solution, implemented in #757

aminalaee changed the title ~~UUID class causing TypeError: string argument without an encoding in helpers.py: object_identifier_values~~ SQLAlchemy-Utils UUIDType TypeError: string argument without an encoding Sep 30, 2023

aminalaee mentioned this issue Sep 30, 2023

Fix object_identifier_values type casting #636

Closed

This comment was marked as off-topic.

Sign in to view

aminalaee closed this as completed May 7, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

SQLAlchemy-Utils UUIDType `TypeError: string argument without an encoding` #635

SQLAlchemy-Utils UUIDType `TypeError: string argument without an encoding` #635

NightSkySK commented Sep 30, 2023

aminalaee commented Sep 30, 2023

NightSkySK commented Sep 30, 2023 •

edited

Loading

BhuwanPandey commented Oct 11, 2023 •

edited

Loading

aminalaee commented Oct 11, 2023

BhuwanPandey commented Oct 11, 2023

aminalaee commented Oct 11, 2023

BhuwanPandey commented Oct 11, 2023

guilhermelou commented Oct 12, 2023

This comment was marked as off-topic.

aminalaee commented Nov 9, 2023

greatlaki commented Nov 9, 2023

breduin commented Feb 28, 2024

Kumzy commented May 6, 2024

aminalaee commented May 7, 2024

SQLAlchemy-Utils UUIDType TypeError: string argument without an encoding #635

SQLAlchemy-Utils UUIDType TypeError: string argument without an encoding #635

Comments

NightSkySK commented Sep 30, 2023

Checklist

Describe the bug

Steps to reproduce the bug

Expected behavior

Actual behavior

Debugging material

Environment

Additional context

aminalaee commented Sep 30, 2023

NightSkySK commented Sep 30, 2023 • edited Loading

BhuwanPandey commented Oct 11, 2023 • edited Loading

aminalaee commented Oct 11, 2023

BhuwanPandey commented Oct 11, 2023

aminalaee commented Oct 11, 2023

BhuwanPandey commented Oct 11, 2023

guilhermelou commented Oct 12, 2023

This comment was marked as off-topic.

aminalaee commented Nov 9, 2023

greatlaki commented Nov 9, 2023

breduin commented Feb 28, 2024

Kumzy commented May 6, 2024

aminalaee commented May 7, 2024

SQLAlchemy-Utils UUIDType `TypeError: string argument without an encoding` #635

SQLAlchemy-Utils UUIDType `TypeError: string argument without an encoding` #635

NightSkySK commented Sep 30, 2023 •

edited

Loading

BhuwanPandey commented Oct 11, 2023 •

edited

Loading