New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Hide imported names in stubs unless 'as id' is used (PEP 484) #3706

Merged
merged 16 commits into from Jul 31, 2017

Conversation

Projects
None yet
3 participants
@ilevkivskyi
Collaborator

ilevkivskyi commented Jul 12, 2017

Fixes #2927

PEP 484 specifies that only names imported as from mod import name as name should be re-exported. However, mypy didn't follow this rule so that this code passed without errors but obviously fails at runtime:

from collections import TypeVar
from weakref import Generic
from getopt import List

T = TypeVar('T')

class C(Generic[T], List[int]):
    ...

This PR makes all these errors. CI will fail since there are several dozens of errors in typeshed, I am going to make a PR to typeshed soon.

Ivan Levkivskyi added some commits Jul 12, 2017

@gvanrossum

This comment has been minimized.

Show comment
Hide comment
@gvanrossum

gvanrossum Jul 12, 2017

Member

I'll test this against Dropbox codebases.

Member

gvanrossum commented Jul 12, 2017

I'll test this against Dropbox codebases.

@ilevkivskyi

This comment has been minimized.

Show comment
Hide comment
@ilevkivskyi

ilevkivskyi Jul 12, 2017

Collaborator

It looks like some of the typeshed failures are due to a bug in my PR, will investigate now.

Collaborator

ilevkivskyi commented Jul 12, 2017

It looks like some of the typeshed failures are due to a bug in my PR, will investigate now.

@gvanrossum

This comment has been minimized.

Show comment
Hide comment
@gvanrossum

gvanrossum Jul 12, 2017

Member

Yeah, I get this for example:

mypy -2 -c 'import xml.sax'
typeshed/stdlib/2and3/xml/sax/__init__.pyi:27: error: Name 'xml.sax.xmlreader.XMLReader' is not defined
typeshed/stdlib/2and3/xml/sax/__init__.pyi:29: error: Name 'xml.sax.handler.ContentHandler' is not defined
typeshed/stdlib/2and3/xml/sax/__init__.pyi:30: error: Name 'xml.sax.handler.ErrorHandler' is not defined
typeshed/stdlib/2and3/xml/sax/__init__.pyi:32: error: Name 'xml.sax.handler.ContentHandler' is not defined
typeshed/stdlib/2and3/xml/sax/__init__.pyi:33: error: Name 'xml.sax.handler.ErrorHandler' is not defined
typeshed/stdlib/2and3/xml/sax/__init__.pyi:35: error: Name 'xml.sax.xmlreader.XMLReader' is not defined
Member

gvanrossum commented Jul 12, 2017

Yeah, I get this for example:

mypy -2 -c 'import xml.sax'
typeshed/stdlib/2and3/xml/sax/__init__.pyi:27: error: Name 'xml.sax.xmlreader.XMLReader' is not defined
typeshed/stdlib/2and3/xml/sax/__init__.pyi:29: error: Name 'xml.sax.handler.ContentHandler' is not defined
typeshed/stdlib/2and3/xml/sax/__init__.pyi:30: error: Name 'xml.sax.handler.ErrorHandler' is not defined
typeshed/stdlib/2and3/xml/sax/__init__.pyi:32: error: Name 'xml.sax.handler.ContentHandler' is not defined
typeshed/stdlib/2and3/xml/sax/__init__.pyi:33: error: Name 'xml.sax.handler.ErrorHandler' is not defined
typeshed/stdlib/2and3/xml/sax/__init__.pyi:35: error: Name 'xml.sax.xmlreader.XMLReader' is not defined
Ivan Levkivskyi
@ilevkivskyi

This comment has been minimized.

Show comment
Hide comment
@ilevkivskyi

ilevkivskyi Jul 12, 2017

Collaborator

After I fixed the bug in this PR, it looks like there are only two errors in typeshed:

error: Module 'json' has no attribute 'JSONDecodeError'
error: Module 'asyncio.futures' has no attribute 'Awaitable'

Both seem to be legitimate errors.

Collaborator

ilevkivskyi commented Jul 12, 2017

After I fixed the bug in this PR, it looks like there are only two errors in typeshed:

error: Module 'json' has no attribute 'JSONDecodeError'
error: Module 'asyncio.futures' has no attribute 'Awaitable'

Both seem to be legitimate errors.

@ilevkivskyi

This comment has been minimized.

Show comment
Hide comment
@ilevkivskyi

ilevkivskyi Jul 12, 2017

Collaborator

only two errors in typeshed

Actually two in stdlib plus a dozen or so in third party stubs.

Collaborator

ilevkivskyi commented Jul 12, 2017

only two errors in typeshed

Actually two in stdlib plus a dozen or so in third party stubs.

@gvanrossum

This comment has been minimized.

Show comment
Hide comment
@gvanrossum

gvanrossum Jul 12, 2017

Member

So do we need to fix all the typeshed errors first? Also I can't re-test this with the Dropbox codebases until you've fixed the merge conflict.

Member

gvanrossum commented Jul 12, 2017

So do we need to fix all the typeshed errors first? Also I can't re-test this with the Dropbox codebases until you've fixed the merge conflict.

@ilevkivskyi

This comment has been minimized.

Show comment
Hide comment
@ilevkivskyi

ilevkivskyi Jul 13, 2017

Collaborator

@gvanrossum I fixed the merge conflict and made a PR to typeshed python/typeshed#1484

Collaborator

ilevkivskyi commented Jul 13, 2017

@gvanrossum I fixed the merge conflict and made a PR to typeshed python/typeshed#1484

@gvanrossum

This comment has been minimized.

Show comment
Hide comment
@gvanrossum

gvanrossum Jul 13, 2017

Member

Please hold off on this until after the release of 0.521 (or until Jukka vets it).

Member

gvanrossum commented Jul 13, 2017

Please hold off on this until after the release of 0.521 (or until Jukka vets it).

@ilevkivskyi

This comment has been minimized.

Show comment
Hide comment
@ilevkivskyi

ilevkivskyi Jul 13, 2017

Collaborator

Please hold off on this until after the release of 0.521 (or until Jukka vets it).

NP, I don't think this is urgent. But it would be great to test this with internal codebases (maybe there are more errors in stubs).

Collaborator

ilevkivskyi commented Jul 13, 2017

Please hold off on this until after the release of 0.521 (or until Jukka vets it).

NP, I don't think this is urgent. But it would be great to test this with internal codebases (maybe there are more errors in stubs).

Ivan Levkivskyi added some commits Jul 13, 2017

@ilevkivskyi

This comment has been minimized.

Show comment
Hide comment
@ilevkivskyi

ilevkivskyi Jul 13, 2017

Collaborator

OK, I have fixed the logic as discussed in python/typeshed#1484, this should be now ready for review and testing with internal codebases.

Collaborator

ilevkivskyi commented Jul 13, 2017

OK, I have fixed the logic as discussed in python/typeshed#1484, this should be now ready for review and testing with internal codebases.

Ivan Levkivskyi added some commits Jul 15, 2017

Ivan Levkivskyi
Ivan Levkivskyi
@ilevkivskyi

This comment has been minimized.

Show comment
Hide comment
@ilevkivskyi

ilevkivskyi Jul 15, 2017

Collaborator

I added one more test and simplified the PR, still I cannot do this with only one flag module_public since it is used for other things like __all__, import * and _private_names. It is possible to have a single flag with three values HIDDEN, PRIVATE, PUBLIC = 0, 1, 2 instead of two bool flags module_public and module_hidden but this will induce some code churn.

Also typeshed is synced, so that this ready to be reviewed and merged.

Collaborator

ilevkivskyi commented Jul 15, 2017

I added one more test and simplified the PR, still I cannot do this with only one flag module_public since it is used for other things like __all__, import * and _private_names. It is possible to have a single flag with three values HIDDEN, PRIVATE, PUBLIC = 0, 1, 2 instead of two bool flags module_public and module_hidden but this will induce some code churn.

Also typeshed is synced, so that this ready to be reviewed and merged.

@ethanhs

One style nit, otherwise looks good to me.

Show outdated Hide outdated mypy/semanal.py
@ethanhs

Thanks! Looks good.

@ilevkivskyi

This comment has been minimized.

Show comment
Hide comment
@ilevkivskyi

ilevkivskyi Jul 31, 2017

Collaborator

@gvanrossum Do you think Ethan's review is enough, or we need another one? (This is quite a minor change, so that I would merge this soon to avoid more merge conflicts.)

Collaborator

ilevkivskyi commented Jul 31, 2017

@gvanrossum Do you think Ethan's review is enough, or we need another one? (This is quite a minor change, so that I would merge this soon to avoid more merge conflicts.)

@gvanrossum

This comment has been minimized.

Show comment
Hide comment
@gvanrossum

gvanrossum Jul 31, 2017

Member

I'm taking a look now.

Member

gvanrossum commented Jul 31, 2017

I'm taking a look now.

@gvanrossum

LGTM. I'm pressed for time, can you merge it yourself?

@ilevkivskyi ilevkivskyi merged commit c87e413 into python:master Jul 31, 2017

2 checks passed

continuous-integration/appveyor/pr AppVeyor build succeeded
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details

@ilevkivskyi ilevkivskyi deleted the ilevkivskyi:no-re-export-from-stubs branch Jul 31, 2017

gvanrossum added a commit to python/typeshed that referenced this pull request Aug 3, 2017

Re-export code from _ast into ast. (#1515)
After mypy [started hiding](python/mypy#3706) imported names in stubs unless `from ... import ...` is used, we found an error with stubs of ast module. 

It looks like ast module should re-export everything in `_ast` and according to PEP 484, it can do that by doing `from _ast import *`, so this is what this PR does.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment