Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

mypy failure with --no-implicit-reexport #897

Closed
philpep opened this issue Aug 31, 2021 · 16 comments
Closed

mypy failure with --no-implicit-reexport #897

philpep opened this issue Aug 31, 2021 · 16 comments
Assignees
Labels
bug Something isn't working pep 484 typing related issues

Comments

@philpep
Copy link

philpep commented Aug 31, 2021

Describe the bug

Using mypy with --no-implicit-reexport (which is part of --strict) generate a failure since alembic 1.7

Expected behavior

No AssertionError

To Reproduce

$ cat << EOF > t.py
import alembic
EOF
$ mypy --no-implicit-reexport t.py
Traceback (most recent call last):
  File "[...]/bin/mypy", line 8, in <module>
    sys.exit(console_entry())
  File "[...]/lib/python3.9/site-packages/mypy/__main__.py", line 11, in console_entry
    main(None, sys.stdout, sys.stderr)
  File "mypy/main.py", line 87, in main
  File "mypy/main.py", line 165, in run_build
  File "mypy/build.py", line 179, in build
  File "mypy/build.py", line 254, in _build
  File "mypy/build.py", line 2697, in dispatch
  File "mypy/build.py", line 3021, in process_graph
  File "mypy/build.py", line 3138, in process_stale_scc
  File "mypy/build.py", line 2288, in write_cache
  File "mypy/build.py", line 1475, in write_cache
  File "mypy/nodes.py", line 313, in serialize
  File "mypy/nodes.py", line 3149, in serialize
  File "mypy/nodes.py", line 3083, in serialize
AssertionError: Definition of alembic.runtime.migration.EnvironmentContext is unexpectedly incomplete

Versions.

  • OS: Debian 11
  • Python: 3.9
  • Alembic: 1.7.1
  • SQLAlchemy: 1.4.23
  • mypy: 0.910

Note it might be a mypy bug (since this is an AssertionError), but I think this looks related to some weirdness in alembic typing code.

Thanks!

@philpep philpep added the requires triage New issue that requires categorization label Aug 31, 2021
@CaselIT CaselIT added bug Something isn't working help wanted Extra attention is needed mypy and removed requires triage New issue that requires categorization labels Aug 31, 2021
@CaselIT
Copy link
Member

CaselIT commented Aug 31, 2021

Hi,

Thanks for reporting. I'm not sure what we should change.
I'm not sure if empty re-export files https://github.com/sqlalchemy/alembic/blob/master/alembic/environment.py and https://github.com/sqlalchemy/alembic/blob/master/alembic/migration.py are to blame.

Running it without --no-implicit-reexport works for you?

@philpep
Copy link
Author

philpep commented Aug 31, 2021

Yes, running mypy without --no-implicit-reexport or --strict doesn't trigger the bug.

Seems when TYPE_CHECKING is enabled there's an import loop between alembic.runtime.migration and alembic.runtime.environment but I'm not sure this is related to the bug since commenting the lines still trigger the bug.
I also tried to comment re-export imports from alembic.environment and alembic.migration and this does not have much effect.

@philpep
Copy link
Author

philpep commented Aug 31, 2021

A workaround for mypy config might be:

[mypy-alembic.*]
no_implicit_reexport = false

@zzzeek
Copy link
Member

zzzeek commented Aug 31, 2021

IMO mypy devs should look at this. maybe @bryanforbes has insight

@taybin
Copy link

taybin commented Aug 31, 2021

I'm not sure if this is relevant to this particular issue, but with no_implicit_reexport set to false, mypy still complains that:

alembic/env.py:9: error: Module has no attribute "config"; maybe "configure"?

@CaselIT
Copy link
Member

CaselIT commented Sep 2, 2021

I'm not sure if this is relevant to this particular issue, but with no_implicit_reexport set to false, mypy still complains that:

alembic/env.py:9: error: Module has no attribute "config"; maybe "configure"?

this is explicetly skipped by the stub generator here https://github.com/sqlalchemy/alembic/blob/master/tools/write_pyi.py#L22 since it has no documentation. I guess I can try fiddle with it to see if that's the problem.

@CaselIT
Copy link
Member

CaselIT commented Sep 6, 2021

the issue seems to be in caused by the presence of context.pyi, and in particular it seems a circular import error.

To reproduce the error it's sufficient having as context.pyi:

# context.pyi
from .migration import MigrationContext

I think this is a mypy bug

@zzzeek
Copy link
Member

zzzeek commented Sep 13, 2021

that import seems wrong, it should come from runtime to avoid the "import *"

@zzzeek
Copy link
Member

zzzeek commented Sep 13, 2021

im not really able to trigger the crash at the moment. if you still are, can you try this:

diff --git a/alembic/context.pyi b/alembic/context.pyi
index d789a22..5b9dd70 100644
--- a/alembic/context.pyi
+++ b/alembic/context.pyi
@@ -13,7 +13,7 @@ if TYPE_CHECKING:
     from sqlalchemy.engine.base import Connection
     from sqlalchemy.sql.schema import MetaData
 
-    from .migration import MigrationContext
+    from .runtime.migration import MigrationContext
     from .runtime.migration import _ProxyTransaction
 
 ### end imports ###

@zzzeek zzzeek removed the help wanted Extra attention is needed label Sep 13, 2021
@sqla-tester
Copy link
Collaborator

Mike Bayer has proposed a fix for this issue in the master branch:

use in-package imports https://gerrit.sqlalchemy.org/c/sqlalchemy/alembic/+/3077

@CaselIT
Copy link
Member

CaselIT commented Sep 13, 2021

You need to test a file that imports alembic. This:

# file.py
import alembic

is enough. Then run mypy file.py --no-implicit-reexport

@CaselIT
Copy link
Member

CaselIT commented Sep 13, 2021

that change does fixes it. Still there is a bug in mypy

@zzzeek
Copy link
Member

zzzeek commented Sep 13, 2021

heres the output when i do that:

$ .tox/mypy/bin/mypy t.py --no-implicit-reexport --no-incremental
alembic/util/langhelpers.py:23: error: Module "alembic.util.compat" does not explicitly export attribute "inspect_getfullargspec"; implicit reexport disabled  [attr-defined]
alembic/util/__init__.py:4: error: Module "alembic.util.langhelpers" does not explicitly export attribute "asbool"; implicit reexport disabled  [attr-defined]
alembic/util/__init__.py:7: error: Module "alembic.util.langhelpers" does not explicitly export attribute "immutabledict"; implicit reexport disabled  [attr-defined]
alembic/util/__init__.py:8: error: Module "alembic.util.langhelpers" does not explicitly export attribute "memoized_property"; implicit reexport disabled  [attr-defined]
alembic/util/__init__.py:11: error: Module "alembic.util.langhelpers" does not explicitly export attribute "to_list"; implicit reexport disabled  [attr-defined]
alembic/util/__init__.py:13: error: Module "alembic.util.langhelpers" does not explicitly export attribute "unique_list"; implicit reexport disabled  [attr-defined]
alembic/ddl/base.py:29: error: Module "alembic.util.sqla_compat" does not explicitly export attribute "Computed"; implicit reexport disabled  [attr-defined]
alembic/ddl/base.py:30: error: Module "alembic.util.sqla_compat" does not explicitly export attribute "Identity"; implicit reexport disabled  [attr-defined]
alembic/operations/batch.py:31: error: Module "alembic.util.sqla_compat" does not explicitly export attribute "_select"; implicit reexport disabled  [attr-defined]
alembic/context.pyi:16: error: Module "alembic.migration" does not explicitly export attribute "MigrationContext"; implicit reexport disabled  [attr-defined]
alembic/script/revision.py:125: error: Module has no attribute "memoized_property"  [attr-defined]
alembic/script/revision.py:138: error: Module has no attribute "memoized_property"  [attr-defined]
alembic/script/revision.py:151: error: Module has no attribute "memoized_property"  [attr-defined]
alembic/script/revision.py:161: error: Module has no attribute "memoized_property"  [attr-defined]
alembic/script/revision.py:171: error: Module has no attribute "memoized_property"  [attr-defined]
alembic/script/revision.py:193: error: Module has no attribute "warn"  [attr-defined]
alembic/script/revision.py:221: error: Module has no attribute "warn"  [attr-defined]
alembic/script/revision.py:359: error: Module has no attribute "to_tuple"  [attr-defined]
alembic/script/revision.py:418: error: Module has no attribute "warn"  [attr-defined]
alembic/script/revision.py:438: error: Module has no attribute "warn"  [attr-defined]
alembic/script/revision.py:704: error: Module has no attribute "to_tuple"  [attr-defined]
alembic/script/revision.py:765: error: Module has no attribute "to_tuple"  [attr-defined]
alembic/script/revision.py:1124: error: Module has no attribute "to_tuple"  [attr-defined]
alembic/script/revision.py:1129: error: Module has no attribute "to_tuple"  [attr-defined]
alembic/script/revision.py:1138: error: Module has no attribute "warn"  [attr-defined]
alembic/script/revision.py:1195: error: Module has no attribute "to_tuple"  [attr-defined]
alembic/script/revision.py:1530: error: Module has no attribute "to_tuple"  [attr-defined]
alembic/script/revision.py:1532: error: Module has no attribute "to_tuple"  [attr-defined]
alembic/script/revision.py:1541: error: Module has no attribute "to_tuple"  [attr-defined]
alembic/script/revision.py:1559: error: Module has no attribute "dedupe_tuple"  [attr-defined]
alembic/script/revision.py:1560: error: Module has no attribute "to_tuple"  [attr-defined]
alembic/script/revision.py:1570: error: Module has no attribute "dedupe_tuple"  [attr-defined]
alembic/script/revision.py:1571: error: Module has no attribute "to_tuple"  [attr-defined]
alembic/script/revision.py:1577: error: Module has no attribute "to_tuple"  [attr-defined]
alembic/operations/base.py:20: error: Module "alembic.util.compat" does not explicitly export attribute "inspect_formatargspec"; implicit reexport disabled  [attr-defined]
alembic/operations/base.py:21: error: Module "alembic.util.compat" does not explicitly export attribute "inspect_getfullargspec"; implicit reexport disabled  [attr-defined]
alembic/operations/base.py:37: error: Name "util.ModuleClsProxy" is not defined  [name-defined]
alembic/operations/base.py:68: error: Module has no attribute "Dispatcher"  [attr-defined]
alembic/operations/ops.py:69: error: Module has no attribute "memoized_property"  [attr-defined]
alembic/operations/ops.py:89: error: Module has no attribute "Dispatcher"  [attr-defined]
alembic/autogenerate/render.py:50: error: Module "alembic.util.sqla_compat" does not explicitly export attribute "Computed"; implicit reexport disabled  [attr-defined]
alembic/autogenerate/render.py:51: error: Module "alembic.util.sqla_compat" does not explicitly export attribute "Identity"; implicit reexport disabled  [attr-defined]
alembic/autogenerate/render.py:92: error: Module has no attribute "Dispatcher"  [attr-defined]
alembic/autogenerate/render.py:127: error: Module has no attribute "to_list"  [attr-defined]
alembic/autogenerate/render.py:914: error: Module has no attribute "Dispatcher"  [attr-defined]
alembic/autogenerate/render.py:925: error: Module has no attribute "warn"  [attr-defined]
alembic/autogenerate/rewriter.py:50: error: Module has no attribute "Dispatcher"  [attr-defined]
alembic/autogenerate/rewriter.py:55: error: Module has no attribute "Dispatcher"  [attr-defined]
alembic/autogenerate/rewriter.py:131: error: Module has no attribute "to_list"  [attr-defined]
alembic/autogenerate/compare.py:21: error: Module "alembic.ddl.base" does not explicitly export attribute "_fk_spec"; implicit reexport disabled  [attr-defined]
alembic/autogenerate/compare.py:57: error: Module has no attribute "Dispatcher"  [attr-defined]
alembic/autogenerate/compare.py:1088: error: Module has no attribute "warn"  [attr-defined]
alembic/autogenerate/api.py:295: error: Module has no attribute "CommandError"  [attr-defined]
alembic/autogenerate/api.py:313: error: Module has no attribute "CommandError"  [attr-defined]
alembic/autogenerate/api.py:342: error: Module has no attribute "memoized_property"  [attr-defined]
alembic/autogenerate/api.py:425: error: Module has no attribute "memoized_property"  [attr-defined]
alembic/autogenerate/api.py:440: error: Module has no attribute "memoized_property"  [attr-defined]
alembic/autogenerate/api.py:535: error: Module has no attribute "CommandError"  [attr-defined]
alembic/autogenerate/api.py:541: error: Module has no attribute "CommandError"  [attr-defined]
alembic/autogenerate/api.py:586: error: Module has no attribute "rev_id"  [attr-defined]
alembic/ddl/impl.py:117: error: Module has no attribute "CommandError"  [attr-defined]
alembic/ddl/impl.py:163: error: Module has no attribute "immutabledict"  [attr-defined]
alembic/ddl/impl.py:225: error: Module has no attribute "warn"  [attr-defined]
alembic/ddl/sqlite.py:55: error: Module has no attribute "Computed"  [attr-defined]
alembic/ddl/sqlite.py:74: error: Module has no attribute "warn"  [attr-defined]
alembic/ddl/postgresql.py:28: error: Module "alembic.ddl.base" does not explicitly export attribute "compiles"; implicit reexport disabled  [attr-defined]
alembic/ddl/postgresql.py:155: error: Module has no attribute "CommandError"  [attr-defined]
alembic/ddl/postgresql.py:397: error: Module has no attribute "Identity"  [attr-defined]
alembic/ddl/mysql.py:326: error: Module has no attribute "CommandError"  [attr-defined]
alembic/ddl/mssql.py:110: error: Module has no attribute "CommandError"  [attr-defined]
alembic/runtime/migration.py:193: error: Module has no attribute "DefaultImpl"  [attr-defined]
alembic/runtime/migration.py:479: error: Module has no attribute "CommandError"  [attr-defined]
alembic/runtime/migration.py:512: error: Module has no attribute "to_list"  [attr-defined]
alembic/runtime/migration.py:515: error: Module has no attribute "to_tuple"  [attr-defined]
alembic/runtime/migration.py:518: error: Module has no attribute "CommandError"  [attr-defined]
alembic/runtime/migration.py:586: error: Module has no attribute "CommandError"  [attr-defined]
alembic/runtime/migration.py:768: error: Module has no attribute "CommandError"  [attr-defined]
alembic/runtime/migration.py:793: error: Module has no attribute "CommandError"  [attr-defined]
alembic/runtime/migration.py:910: error: Module has no attribute "to_tuple"  [attr-defined]
alembic/runtime/migration.py:918: error: Module has no attribute "to_tuple"  [attr-defined]
alembic/runtime/migration.py:1008: error: Module has no attribute "format_as_comma"  [attr-defined]
alembic/runtime/migration.py:1009: error: Module has no attribute "format_as_comma"  [attr-defined]
alembic/runtime/migration.py:1252: error: Module has no attribute "to_tuple"  [attr-defined]
alembic/runtime/migration.py:1253: error: Module has no attribute "to_tuple"  [attr-defined]
alembic/script/write_hooks.py:54: error: Module has no attribute "CommandError"  [attr-defined]
alembic/script/write_hooks.py:80: error: Module has no attribute "CommandError"  [attr-defined]
alembic/script/write_hooks.py:84: error: Module has no attribute "status"  [attr-defined]
alembic/script/base.py:91: error: Module has no attribute "CommandError"  [attr-defined]
alembic/script/base.py:101: error: Module has no attribute "CommandError"  [attr-defined]
alembic/script/base.py:105: error: Module has no attribute "memoized_property"  [attr-defined]
alembic/script/base.py:130: error: Module has no attribute "warn"  [attr-defined]
alembic/script/base.py:152: error: Module has no attribute "CommandError"  [attr-defined]
alembic/script/base.py:205: error: Module has no attribute "coerce_resource_to_filename"  [attr-defined]
alembic/script/base.py:239: error: Module has no attribute "CommandError"  [attr-defined]
alembic/script/base.py:251: error: Module has no attribute "format_as_comma"  [attr-defined]
alembic/script/base.py:253: error: Module has no attribute "CommandError"  [attr-defined]
alembic/script/base.py:259: error: Module has no attribute "CommandError"  [attr-defined]
alembic/script/base.py:261: error: Module has no attribute "CommandError"  [attr-defined]
alembic/script/base.py:406: error: Module has no attribute "CommandError"  [attr-defined]
alembic/script/base.py:476: error: Module has no attribute "to_tuple"  [attr-defined]
alembic/script/base.py:483: error: Module has no attribute "unique_list"  [attr-defined]
alembic/script/base.py:563: error: Module has no attribute "load_python_file"  [attr-defined]
alembic/script/base.py:570: error: Module has no attribute "status"  [attr-defined]
alembic/script/base.py:572: error: Module has no attribute "template_to_file"  [attr-defined]
alembic/script/base.py:580: error: Module has no attribute "status"  [attr-defined]
alembic/script/base.py:587: error: Module has no attribute "status"  [attr-defined]
alembic/script/base.py:602: error: Module has no attribute "CommandError"  [attr-defined]
alembic/script/base.py:650: error: Module has no attribute "CommandError"  [attr-defined]
alembic/script/base.py:662: error: Module has no attribute "CommandError"  [attr-defined]
alembic/script/base.py:674: error: Module has no attribute "CommandError"  [attr-defined]
alembic/script/base.py:686: error: Module has no attribute "CommandError"  [attr-defined]
alembic/script/base.py:699: error: Module has no attribute "CommandError"  [attr-defined]
alembic/script/base.py:713: error: Module has no attribute "to_list"  [attr-defined]
alembic/script/base.py:724: error: Module has no attribute "to_tuple"  [attr-defined]
alembic/script/base.py:729: error: Module has no attribute "format_as_comma"  [attr-defined]
alembic/script/base.py:741: error: Module has no attribute "CommandError"  [attr-defined]
alembic/script/base.py:745: error: Module has no attribute "CommandError"  [attr-defined]
alembic/script/base.py:796: error: Module has no attribute "to_tuple"  [attr-defined]
alembic/script/base.py:799: error: Module has no attribute "to_tuple"  [attr-defined]
alembic/script/base.py:850: error: Module has no attribute "format_as_comma"  [attr-defined]
alembic/script/base.py:855: error: Module has no attribute "format_as_comma"  [attr-defined]
alembic/script/base.py:860: error: Module has no attribute "format_as_comma"  [attr-defined]
alembic/script/base.py:893: error: Module has no attribute "format_as_comma"  [attr-defined]
alembic/script/base.py:900: error: Module has no attribute "format_as_comma"  [attr-defined]
alembic/script/base.py:937: error: Module has no attribute "format_as_comma"  [attr-defined]
alembic/script/base.py:999: error: Module has no attribute "load_python_file"  [attr-defined]
alembic/script/base.py:1006: error: Module has no attribute "CommandError"  [attr-defined]
alembic/runtime/environment.py:29: error: Name "util.ModuleClsProxy" is not defined  [name-defined]
alembic/runtime/environment.py:219: error: Module has no attribute "CommandError"  [attr-defined]
alembic/runtime/environment.py:808: error: Module has no attribute "to_tuple"  [attr-defined]
alembic/command.py:62: error: Module has no attribute "CommandError"  [attr-defined]
alembic/command.py:68: error: Module has no attribute "CommandError"  [attr-defined]
alembic/command.py:71: error: Module has no attribute "status"  [attr-defined]
alembic/command.py:78: error: Module has no attribute "status"  [attr-defined]
alembic/command.py:91: error: Module has no attribute "msg"  [attr-defined]
alembic/command.py:105: error: Module has no attribute "status"  [attr-defined]
alembic/command.py:108: error: Module has no attribute "msg"  [attr-defined]
alembic/command.py:185: error: Module has no attribute "RevisionContext"  [attr-defined]
alembic/command.py:192: error: Module has no attribute "asbool"  [attr-defined]
alembic/command.py:198: error: Module has no attribute "CommandError"  [attr-defined]
alembic/command.py:213: error: Module has no attribute "CommandError"  [attr-defined]
alembic/command.py:271: error: Module has no attribute "rev_id"  [attr-defined]
alembic/command.py:305: error: Module has no attribute "CommandError"  [attr-defined]
alembic/command.py:347: error: Module has no attribute "CommandError"  [attr-defined]
alembic/command.py:350: error: Module has no attribute "CommandError"  [attr-defined]
alembic/command.py:416: error: Module has no attribute "CommandError"  [attr-defined]
alembic/command.py:424: error: Module has no attribute "asbool"  [attr-defined]
alembic/command.py:584: error: Module has no attribute "to_list"  [attr-defined]
alembic/command.py:598: error: Module has no attribute "to_list"  [attr-defined]
alembic/command.py:609: error: Module has no attribute "to_tuple"  [attr-defined]
alembic/command.py:641: error: Module has no attribute "CommandError"  [attr-defined]
alembic/command.py:645: error: Module has no attribute "open_in_editor"  [attr-defined]
alembic/config.py:99: error: Name "util.immutabledict" is not defined  [name-defined]
alembic/config.py:99: error: Module has no attribute "immutabledict"  [attr-defined]
alembic/config.py:135: error: Module has no attribute "memoized_property"  [attr-defined]
alembic/config.py:174: error: Module has no attribute "write_outstream"  [attr-defined]
alembic/config.py:176: error: Module has no attribute "memoized_property"  [attr-defined]
alembic/config.py:282: error: Module has no attribute "CommandError"  [attr-defined]
alembic/config.py:523: error: Module has no attribute "inspect_getfullargspec"  [attr-defined]
alembic/config.py:564: error: Module has no attribute "CommandError"  [attr-defined]
alembic/config.py:568: error: Module has no attribute "err"  [attr-defined]
Found 161 errors in 23 files (checked 1 source file)

I have no idea why i am seeing all that or what those errors are

@CaselIT
Copy link
Member

CaselIT commented Sep 13, 2021

that's some of the strict mode of mypy. with --strict some additional errors are also added

@zzzeek
Copy link
Member

zzzeek commented Sep 13, 2021

Note no assertion error though. The other day I think this did reproduce for me so I don't know what changed

@CaselIT
Copy link
Member

CaselIT commented Sep 13, 2021

yes, I could reproduce the mypy assertion error before this patch. with the patch is seems ok

@zzzeek zzzeek added the pep 484 typing related issues label Jul 7, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working pep 484 typing related issues
Projects
None yet
Development

No branches or pull requests

5 participants