Inferring query results

[mehdigmira]

Hello,

Currently migrating a large code base which is largely type annotated to SQLAlchemy 2.0, we have a LOT of code such as this

from sqlalchemy import select
from sqlalchemy.orm import DeclarativeBase, Mapped, Session


def test(session: Session):
    class Model(DeclarativeBase):
        name: Mapped[str]

    model_name, = session.query(Model.name).one()
    model_names = [model_name for model_name, in session.query(Model.name)]

model_name and model_names will be inferred as Any, which defeats the point of having type checking.

I understand that to benefit from better inference we'd have to rewrite that as

    model_name, = session.query(Model.name).one().t
    model_names = [model_name for model_name, in session.query(Model.name).tuples()]

However, such usages are very hard to track down, and it's very hard to ensure across a large team that the first syntax is no longer used.

IMO, there are 2 options:

• either Row class should inherit from tuple to benefit from better inference upon unpacking
• or Row should not implement iter (typing-wise) so that someone that cares about type checking will get an error when using the first syntax

What do you think ?

[CaselIT]

Hi,

Currently migrating a large code base which is largely type annotated to SQLAlchemy 2.0, we have a LOT of code such as this ...

That raises an error, since that model definition is not correct.

model_name and model_names will be inferred as Any, which defeats the point of having type checking.

Assuming you have a correct model, you can use scalars() to get a sequence of strings.
Also if you use the v2 syntax you will not have that issue.

IMO, there are 2 options:

There is currently a limitation in the typing side that prevents Row to behave like a tuple. Tbe situation should hopefully be fixed at some point

[mehdigmira]

That raises an error, since that model definition is not correct.

what do you mean ? it's just a snippet to give an idea, i didn't add primary key and tablename boilerplate.

Assuming you have a correct model, you can use scalars() to get a sequence of strings.
Also if you use the v2 syntax you will not have that issue.

That is my point: tracking down all the usages in a large codebase and changing them + making sure no one uses the first syntax is very tedious, or impossible. Since no error is logged when type checking, linting or runnning tests.

There is currently a limitation in the typing side that prevents Row to behave like a tuple. Tbe situation should hopefully be fixed at some point

What do you mean ? I don't see any future PEP or issue that aims to allow classes to behave like tuples, do you ?

[zzzeek]

there is https://peps.python.org/pep-0646/ which came extremely close to doing this and for inexplicable reasons stopped about 90% of the way there.

[mehdigmira]

yes unfortunately PEP646 doesn't allow this :/

[zzzeek]

either Row class should inherit from tuple to benefit from better inference upon unpacking

that's a non-starter (edit: you can be absolutely sure this is the first place I went when trying to solve this problem. adding .t was absolutely not my preference. I dont recall the specific reasons this wasn't working out but they were pretty fundamental. tuple is a very rigid type in Python)

Row should not implement iter (typing-wise) so that someone that cares about type checking will get an error when using the first syntax

iterating a Row is entirely valid and this would raise typing errors for people actually doing simple things like tuple(row)

the solution here is that Python typing has to improve upon https://peps.python.org/pep-0646/ so that we can actually make Tuple-like objects that support all the typing features of Tuple.

[CaselIT]

using mypy with disallow_any_expr = true reports that error (among a boatload more)

[mehdigmira]

disallow_any_expr seems to disallow the usage of Any in the entire codebase. But Any is sometimes a legit type.

[CaselIT]

yes, sadly mypy seems to not have the usefult typescript setting "noImplicitAny" (at least I did not find it)

[mehdigmira]

But here the Any is explicit, since Row implements def __iter__(self) -> Iterator[Any]

[CaselIT]

I think in typescript that would error since you need to explicitly indicate that a variable is any. but I may be wrong

[mehdigmira]

On second thought @zzzeek @CaselIT I think it would be better to make Row inherit from tuple and benefit from PEP646 TypeVarTuple.
If there is some runtime reason why Row isn't a tuple, I don't think there is one typing-wise and we can do smth like the POC I tried here https://github.com/sqlalchemy/sqlalchemy/pull/10276/files#diff-657c0a613fba57dc7d5fec03e74fdaa213deb631ded96c52f71b8a3a50a187caR48

_TP = TypeVarTuple("_TP")

if TYPE_CHECKING:

    class _Row(BaseRow, tuple[Unpack[_TP]]):
        ...

else:

    class _Row(BaseRow, Generic[Unpack[_TP]]):
        ...  

class Row(_Row[Unpack[_TP]]):
    ... # code goes here

And we can now benefit from both PEP646 features as well as tuple unpacking magic, without having to ever call specific methods such as .t

With the POC above, pyright now infers

    item = session.query(Model.name).one()
    (name,) = item  # name is inferred as str !

What do you think ?

[CaselIT]

does mypy support pep 646? search, nope python/mypy#12280

At the moment this seems more or less dead in the water, until at least mypy supports it

[CaselIT]

At the moment this seems more or less dead in the water, until at least mypy supports it

unless this new feature keeps things the same as they are now in mypy that is

[mehdigmira]

yeah mypy is a bit of a pain point ^^
The development cycle is very slow. Pyright on the other side started supporting it a while ago.

[CaselIT]

sure, but from what I can gather mypy is the main typing tool used by the python community, so we realistically cannot make things worse for it to support a feature available only on pyright.

it's a bit of a bummer, but that's how things are

[mehdigmira]

Sure, but hopefully, mypy is not that far away from supporting this :) python/mypy@6f650cf