Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding Python types #120

Closed
sbdchd opened this issue Aug 10, 2020 · 4 comments
Closed

Adding Python types #120

sbdchd opened this issue Aug 10, 2020 · 4 comments

Comments

@sbdchd
Copy link

sbdchd commented Aug 10, 2020

Was wondering your thoughts on adding types directly to this project to support mypy and similar tools.

Otherwise, would you be okay with me (or someone else) adding types for this package to the python/typeshed?

related: https://www.python.org/dev/peps/pep-0561/
related: https://www.python.org/dev/peps/pep-0484/

@indygreg
Copy link
Owner

I am 100% in favor of adding type annotations. And since main dropped support for Python 2, we can even add them inline.

However, we can't simply add type annotations to C/Rust extensions. So we may have to do something sneaky to trick mypy and other tools to recognize the annotations.

I also have a strong preference for adding type annotations to this repo, as I don't want to worry about syncing them with a 3rd party repo.

What's the state of the art for defining type annotations for code implemented by C extensions?

@sbdchd
Copy link
Author

sbdchd commented Aug 11, 2020

Great! Having the types in the repo is definitely better than having to keep them in sync with the typeshed.

As for typing native extensions, I think we'd need to define type stubs aka .pyi files along with the py.typed file mentioned in https://www.python.org/dev/peps/pep-0561/#packaging-type-information

sbdchd added a commit to chdsbd/kodiak that referenced this issue Aug 11, 2020
Now we explicitly ignore specific imports instead of a blanket ignore
on all missing types.

Upgraded databases and pytest to get their latest versions which include
types, for the rest we're ignoring them explicitly.
see #485

- arrow

  doesn't have types but we can replace it with std library calls

- requests_async

  deprecated in favor of httpx, so we'd need to upgrade to get types.
  The API of httpx is a little different as well.

- structlog

  doesn't have types currently

  hynek/structlog#165

- rure

  No stubs yet

  davidblewett/rure-python#23

- pytest_mock

  types recently added to the typeshed so we'll need to wait for the next
  release of mypy.

  pytest-dev/pytest-mock#152

- markdown_html_finder

  Wrapper around some Rust code. Needs stubs to be added.

  chdsbd/markdown-html-finder#5

- asyncio_redis

  We're our own fork of it, but adding types would be rather manual.
  Granted we aren't using that much of the API surface.

  https://github.com/chdsbd/asyncio-redis

- zstandard

  No stubs yet, so we're ignoring for the time being

  indygreg/python-zstandard#120

- inflection

  Types, but I think they're misconfigured, once they're fixed we can
  unignore.

  jpvanhal/inflection#49

- django

  Need to setup django-stubs for this to work since django doesn't have
  any static types.

  https://github.com/TypedDjango/django-stubs

- dj_database_url

  Doesn't have any static types yet

  jazzband/dj-database-url#135

- responses

  No types yet

  getsentry/responses#339

- stripe

  Plan for types, but they don't exist currently

  stripe/stripe-python#650
kodiakhq bot pushed a commit to chdsbd/kodiak that referenced this issue Aug 11, 2020
Now we explicitly ignore specific imports instead of a blanket ignore
on all missing types.

Upgraded databases and pytest to get their latest versions which include
types, for the rest we're ignoring them explicitly.
see #485

- arrow

  doesn't have types but we can replace it with std library calls

- requests_async

  deprecated in favor of httpx, so we'd need to upgrade to get types.
  The API of httpx is a little different as well.

- structlog

  doesn't have types currently

  hynek/structlog#165

- rure

  No stubs yet

  davidblewett/rure-python#23

- pytest_mock

  types recently added to the typeshed so we'll need to wait for the next
  release of mypy.

  pytest-dev/pytest-mock#152

- markdown_html_finder

  Wrapper around some Rust code. Needs stubs to be added.

  chdsbd/markdown-html-finder#5

- asyncio_redis

  We're using our own fork of it, but adding types would be rather manual.
  Granted we aren't using that much of the API surface.

  https://github.com/chdsbd/asyncio-redis

- zstandard

  No stubs yet, so we're ignoring for the time being

  indygreg/python-zstandard#120

- inflection

  Types, but I think they're misconfigured, once they're fixed we can
  unignore.

  jpvanhal/inflection#49

- django

  Need to setup django-stubs for this to work since django doesn't have
  any static types.

  https://github.com/TypedDjango/django-stubs

- dj_database_url

  Doesn't have any static types yet

  jazzband/dj-database-url#135

- responses

  No types yet

  getsentry/responses#339

- stripe

  Plan for types, but they don't exist currently

  stripe/stripe-python#650
kodiakhq bot pushed a commit to chdsbd/kodiak that referenced this issue Nov 13, 2020
Added basic type stubs for some of the smaller libraries the bot and
web_api use.

We put the types in the `typings` folder which we configure in the Mypy, 
but is also the [default path that pyright looks at](https://github.com/microsoft/pyright/blob/b9e61e9f2ae6179284fed8e7b534e5dcb46c8f8b/docs/configuration.md)
so pylance should just work.

Libraries with missing types that remain:
- structlog, waiting on the release of its typed api
- requests_async, need to convert to httpx which has types
- asyncio_redis, maybe write stubs, or use a different client
- django, stubs exist, last time we tried it, there were a number of
  false positives
- stripe, no types currently, could probably write some minimal stubs
  rel: stripe/stripe-python#650


rel: indygreg/python-zstandard#120
rel: davidblewett/rure-python#23
rel: jazzband/dj-database-url#135
@ghost
Copy link

ghost commented Nov 29, 2020

What's the state of the art for defining type annotations for code implemented by C extensions?

Type annotations can be put in __init__.pyi file.
And add a 0-byte file py.typed, it indicates that the module supports type hints, this file is for mypy.

Here's an example:
https://github.com/animalize/pyzstd/tree/dev/src
(And add them in setup.py script)

On the current trend, maybe one day zstd will be included in Python standard library, so I wrote this module.

@indygreg
Copy link
Owner

I've committed a .pyi file for zstandard. Please file new issues if you run into any problems with the type annotations. I'm in the process of adding CI coverage for the type annotations to help ensure they are accurate.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants