latest mypy 0.990 crashes with SQLAlchemy main #14027

zzzeek · 2022-11-07T18:53:56Z

running mypy or dmypy on SQLAlchemy main branch crashes reliably for the new 0.990 release . For the run below, I have SQLAlchemy checked out on the rel_2_0_0b3 branch so that the results should stay the same regardless of when we make new commits to main. Same issue reproduces with mypy git master.

[classic@photon3 tmp]$ virtualenv .venv
created virtual environment CPython3.10.0.final.0-64 in 234ms

[classic@photon3 tmp]$ .venv/bin/pip install mypy
Successfully installed mypy-0.990 mypy-extensions-0.4.3 tomli-2.0.1 typing-extensions-4.4.0

[classic@photon3 tmp]$ git clone https://github.com/sqlalchemy/sqlalchemy
Cloning into 'sqlalchemy'...

[classic@photon3 tmp]$ source .venv/bin/activate
(.venv) [classic@photon3 tmp]$ cd sqlalchemy/

(.venv) [classic@photon3 tmp]$ git checkout rel_2_0_0b3
HEAD is now at 35648a109 - 2.0.0b3


(.venv) [classic@photon3 sqlalchemy:HEAD]$ mypy --show-traceback lib/
lib/sqlalchemy/util/typing.py:371: error: Missing return statement  [empty-body]
lib/sqlalchemy/util/typing.py:392: error: Missing return statement  [empty-body]
lib/sqlalchemy/util/typing.py:395: error: Implicit return in function which does not return  [empty-body]
lib/sqlalchemy/util/typing.py:398: error: Implicit return in function which does not return  [empty-body]
lib/sqlalchemy/util/typing.py:414: error: Missing return statement  [empty-body]
lib/sqlalchemy/util/langhelpers.py:1451: error: Function "Callable[[type, Union[type, UnionType, Tuple[Union[type, UnionType, Tuple[Any, ...]], ...]]], bool]" could always be true in boolean context  [truthy-function]
lib/sqlalchemy/sql/visitors.py:556: error: Function "Callable[..., Any]" could always be true in boolean context  [truthy-function]
lib/sqlalchemy/sql/traversals.py:568: error: Function "Callable[..., Any]" could always be true in boolean context  [truthy-function]
lib/sqlalchemy/pool/base.py:730: error: Function "Callable[[Optional[DBAPIConnection], Optional[_ConnectionRecord], Pool, Optional[ReferenceType[_ConnectionFairy]], Union[bool, Literal['debug'], None], bool, Optional[_ConnectionFairy]], None]" could always be true in boolean context  [truthy-function]
lib/sqlalchemy/sql/compiler.py:1730: error: Function "Callable[[Any, Any], Any]" could always be true in boolean context  [truthy-function]
lib/sqlalchemy/engine/interfaces.py:1259: error: Missing return statement  [empty-body]
lib/sqlalchemy/engine/base.py:2950: error: Function "Callable[[], bool]" could always be true in boolean context  [truthy-function]
lib/sqlalchemy/orm/bulk_persistence.py:889: error: INTERNAL ERROR -- Please try using mypy master on GitHub:
https://mypy.readthedocs.io/en/stable/common_issues.html#using-a-development-mypy-build
Please report a bug at https://github.com/python/mypy/issues
version: 1.0.0+dev.39d35cdee1bd02a6fb071334273e1c1fb0893066
Traceback (most recent call last):
  File "/home/classic/tmp/.venv/bin/mypy", line 8, in <module>
    sys.exit(console_entry())
  File "/home/classic/tmp/.venv/lib/python3.10/site-packages/mypy/__main__.py", line 15, in console_entry
    main()
  File "/home/classic/tmp/.venv/lib/python3.10/site-packages/mypy/main.py", line 95, in main
    res, messages, blockers = run_build(sources, options, fscache, t0, stdout, stderr)
  File "/home/classic/tmp/.venv/lib/python3.10/site-packages/mypy/main.py", line 174, in run_build
    res = build.build(sources, options, None, flush_errors, fscache, stdout, stderr)
  File "/home/classic/tmp/.venv/lib/python3.10/site-packages/mypy/build.py", line 193, in build
    result = _build(
  File "/home/classic/tmp/.venv/lib/python3.10/site-packages/mypy/build.py", line 276, in _build
    graph = dispatch(sources, manager, stdout)
  File "/home/classic/tmp/.venv/lib/python3.10/site-packages/mypy/build.py", line 2903, in dispatch
    process_graph(graph, manager)
  File "/home/classic/tmp/.venv/lib/python3.10/site-packages/mypy/build.py", line 3287, in process_graph
    process_stale_scc(graph, scc, manager)
  File "/home/classic/tmp/.venv/lib/python3.10/site-packages/mypy/build.py", line 3388, in process_stale_scc
    graph[id].type_check_first_pass()
  File "/home/classic/tmp/.venv/lib/python3.10/site-packages/mypy/build.py", line 2309, in type_check_first_pass
    self.type_checker().check_first_pass()
  File "/home/classic/tmp/.venv/lib/python3.10/site-packages/mypy/checker.py", line 465, in check_first_pass
    self.accept(d)
  File "/home/classic/tmp/.venv/lib/python3.10/site-packages/mypy/checker.py", line 573, in accept
    stmt.accept(self)
  File "/home/classic/tmp/.venv/lib/python3.10/site-packages/mypy/nodes.py", line 1161, in accept
    return visitor.visit_class_def(self)
  File "/home/classic/tmp/.venv/lib/python3.10/site-packages/mypy/checker.py", line 2132, in visit_class_def
    self.accept(defn.defs)
  File "/home/classic/tmp/.venv/lib/python3.10/site-packages/mypy/checker.py", line 573, in accept
    stmt.accept(self)
  File "/home/classic/tmp/.venv/lib/python3.10/site-packages/mypy/nodes.py", line 1242, in accept
    return visitor.visit_block(self)
  File "/home/classic/tmp/.venv/lib/python3.10/site-packages/mypy/checker.py", line 2569, in visit_block
    self.accept(s)
  File "/home/classic/tmp/.venv/lib/python3.10/site-packages/mypy/checker.py", line 573, in accept
    stmt.accept(self)
  File "/home/classic/tmp/.venv/lib/python3.10/site-packages/mypy/nodes.py", line 934, in accept
    return visitor.visit_decorator(self)
  File "/home/classic/tmp/.venv/lib/python3.10/site-packages/mypy/checker.py", line 4533, in visit_decorator
    self.check_func_item(e.func, name=e.func.name)
  File "/home/classic/tmp/.venv/lib/python3.10/site-packages/mypy/checker.py", line 1011, in check_func_item
    self.check_func_def(defn, typ, name, allow_empty)
  File "/home/classic/tmp/.venv/lib/python3.10/site-packages/mypy/checker.py", line 1208, in check_func_def
    self.accept(item.body)
  File "/home/classic/tmp/.venv/lib/python3.10/site-packages/mypy/checker.py", line 573, in accept
    stmt.accept(self)
  File "/home/classic/tmp/.venv/lib/python3.10/site-packages/mypy/nodes.py", line 1242, in accept
    return visitor.visit_block(self)
  File "/home/classic/tmp/.venv/lib/python3.10/site-packages/mypy/checker.py", line 2569, in visit_block
    self.accept(s)
  File "/home/classic/tmp/.venv/lib/python3.10/site-packages/mypy/checker.py", line 573, in accept
    stmt.accept(self)
  File "/home/classic/tmp/.venv/lib/python3.10/site-packages/mypy/nodes.py", line 1510, in accept
    return visitor.visit_if_stmt(self)
  File "/home/classic/tmp/.venv/lib/python3.10/site-packages/mypy/checker.py", line 4185, in visit_if_stmt
    self.accept(s.else_body)
  File "/home/classic/tmp/.venv/lib/python3.10/site-packages/mypy/checker.py", line 573, in accept
    stmt.accept(self)
  File "/home/classic/tmp/.venv/lib/python3.10/site-packages/mypy/nodes.py", line 1242, in accept
    return visitor.visit_block(self)
  File "/home/classic/tmp/.venv/lib/python3.10/site-packages/mypy/checker.py", line 2569, in visit_block
    self.accept(s)
  File "/home/classic/tmp/.venv/lib/python3.10/site-packages/mypy/checker.py", line 573, in accept
    stmt.accept(self)
  File "/home/classic/tmp/.venv/lib/python3.10/site-packages/mypy/nodes.py", line 834, in accept
    return visitor.visit_func_def(self)
  File "/home/classic/tmp/.venv/lib/python3.10/site-packages/mypy/checker.py", line 938, in visit_func_def
    self._visit_func_def(defn)
  File "/home/classic/tmp/.venv/lib/python3.10/site-packages/mypy/checker.py", line 963, in _visit_func_def
    assert orig_type is not None, f"Error checking function redefinition {defn}"
AssertionError: Error checking function redefinition FuncDef:889(
  eval_condition
  Args(
    Var(obj))
  Block:889(
    ReturnStmt:890(
      NameExpr(True [builtins.True]))))
lib/sqlalchemy/orm/bulk_persistence.py:889: : note: use --pdb to drop into pdb

The text was updated successfully, but these errors were encountered:

hauntsaninja · 2022-11-07T19:00:04Z

Thanks for testing! I'll take a look at fixing this. I'll also add sqlalchemy main to mypy_primer

zzzeek · 2022-11-07T19:03:46Z

Thanks for testing! I'll take a look at fixing this. I'll also add sqlalchemy main to mypy_primer

this is a good idea, SQLAlchemy pushes the typing system very hard

zzzeek · 2022-11-09T14:27:52Z

this patch in SQLAlchemy replaces the problematic code that Mypy crashes on and the crash seems to be avoided

diff --git a/lib/sqlalchemy/orm/bulk_persistence.py b/lib/sqlalchemy/orm/bulk_persistence.py
index 651533db6a..1640554678 100644
--- a/lib/sqlalchemy/orm/bulk_persistence.py
+++ b/lib/sqlalchemy/orm/bulk_persistence.py
@@ -19,7 +19,7 @@ from typing import cast
 from typing import Dict
 from typing import Iterable
 from typing import Optional
-from typing import overload
+from typing import overload, Callable
 from typing import TYPE_CHECKING
 from typing import TypeVar
 from typing import Union
@@ -883,11 +883,15 @@ class BulkUDCompileState(ORMDMLState):
             crit += cls._adjust_for_extra_criteria(global_attributes, mapper)
 
         if crit:
-            eval_condition = evaluator_compiler.process(*crit)
+            eval_condition: Callable[..., bool] = evaluator_compiler.process(
+                *crit
+            )
         else:
-
-            def eval_condition(obj):
+            # workaround for both pylance complaining as well as
+            # mypy https://github.com/python/mypy/issues/14027
+            def _eval_condition(obj):
                 return True
+            eval_condition = _eval_condition
 
         return eval_condition

mypy introduces a crash we need to work around, also some new rules. It also has either a behavioral change regarding how output is rendered in relationship to files being within sys.path or not, so work around that for test_mypy_plugin_py3k.py References: python/mypy#14027 Change-Id: I689c7fe27dc52abee932de9e0fb23b2a2eba76fa

hauntsaninja · 2022-11-10T21:33:40Z

Thanks!

Here's a minimal repro:

def something(x):
    raise

def whatever() -> None:
    cond = ()
    if cond:
        foo = something(*cond)
    else:
        def foo(obj): ...

(will probably get time to actually try to fix over the weekend)

Further simplifies to:

def whatever() -> None:
    cond = ()
    if cond:
        foo = 1
    else:
        def foo(obj): ...

Fixes python#14027

Fixes #14027 (issue was surfaced by #13509)

zzzeek added the crash label Nov 7, 2022

hauntsaninja self-assigned this Nov 7, 2022

peterschutt mentioned this issue Nov 7, 2022

chore(deps): unpin mypy topsport-com-au/starlite-saqlalchemy#100

Closed

dchigarev mentioned this issue Nov 8, 2022

[CI] The latest mypy fails on current master modin-project/modin#5206

Closed

samuelcolvin mentioned this issue Nov 8, 2022

mypy 0.990 is 1000x slow on pydantic codebase vs 0.982 #14034

Closed

svalentin mentioned this issue Nov 8, 2022

Mypy 0.990 Release Planning #13871

Closed

hauntsaninja added a commit to hauntsaninja/mypy that referenced this issue Nov 11, 2022

Fix crash with function redefinition

4c4cf8f

Fixes python#14027

hauntsaninja mentioned this issue Nov 11, 2022

Fix crash with function redefinition #14064

Merged

hauntsaninja closed this as completed in #14064 Nov 11, 2022

hauntsaninja added a commit that referenced this issue Nov 11, 2022

Fix crash with function redefinition (#14064)

67855fa

Fixes #14027 (issue was surfaced by #13509)

hauntsaninja added a commit that referenced this issue Nov 11, 2022

Fix crash with function redefinition (#14064)

ab0ea1e

Fixes #14027 (issue was surfaced by #13509)

dchigarev mentioned this issue Dec 5, 2022

TEST-#5348: Instead of capping mypy, exclude 0.990. modin-project/modin#5349

Merged

7 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

latest mypy 0.990 crashes with SQLAlchemy main #14027

latest mypy 0.990 crashes with SQLAlchemy main #14027

zzzeek commented Nov 7, 2022 •

edited

hauntsaninja commented Nov 7, 2022

zzzeek commented Nov 7, 2022

zzzeek commented Nov 9, 2022

hauntsaninja commented Nov 10, 2022 •

edited

latest mypy 0.990 crashes with SQLAlchemy main #14027

latest mypy 0.990 crashes with SQLAlchemy main #14027

Comments

zzzeek commented Nov 7, 2022 • edited

hauntsaninja commented Nov 7, 2022

zzzeek commented Nov 7, 2022

zzzeek commented Nov 9, 2022

hauntsaninja commented Nov 10, 2022 • edited

zzzeek commented Nov 7, 2022 •

edited

hauntsaninja commented Nov 10, 2022 •

edited