Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement flake8-pyi #848

Closed
56 tasks done
JonathanPlasse opened this issue Nov 21, 2022 · 17 comments
Closed
56 tasks done

Implement flake8-pyi #848

JonathanPlasse opened this issue Nov 21, 2022 · 17 comments
Labels
plugin Implementing a known but unsupported plugin

Comments

@JonathanPlasse
Copy link
Contributor

JonathanPlasse commented Nov 21, 2022

flake8-pyi

Error codes

  • Y001 Names of TypeVars, ParamSpecs and TypeVarTuples in stubs should usually start with _. This makes sure you don't accidentally expose names internal to the stub.
  • Y002 If test must be a simple comparison against sys.platform or sys.version_info. Stub files support simple conditionals to indicate differences between Python versions or platforms, but type checkers only understand a limited subset of Python syntax, and this warning triggers on conditionals that type checkers will probably not understand.
  • Y003 Unrecognized sys.version_info check. Similar, but triggers on some comparisons involving version checks.
  • Y004 Version comparison must use only major and minor version. Type checkers like mypy don't know about patch versions of Python (e.g. 3.4.3 versus 3.4.4), only major and minor versions (3.3 versus 3.4). Therefore, version checks in stubs should only use the major and minor versions. If new functionality was introduced in a patch version, pretend that it was there all along.
  • Y005 Version comparison must be against a length-n tuple.
  • Y006 Use only < and >= for version comparisons. Comparisons involving > and <= may produce unintuitive results when tools do use the full sys.version_info tuple.
  • Y007 Unrecognized sys.platform check. Platform checks should be simple string comparisons.
  • Y008 Unrecognized platform. To prevent you from typos, we warn if you use a platform name outside a small set of known platforms (e.g. "linux" and "win32").
  • Y009 Empty body should contain ..., not pass. This is just a stylistic choice, but it's the one typeshed made.
  • Y010 Function body must contain only .... Stub files should not contain code, so function bodies should be empty.
  • Y011 All default values for typed function arguments must be .... Type checkers ignore the default value, so the default value is not useful information in a stub file.
  • Y012 Class body must not contain pass.
  • Y013 Non-empty class body must not contain ....
  • Y014 All default values for arguments must be .... A stronger version of Y011 that includes arguments without type annotations.
  • Y015 Attribute must not have a default value other than ....
  • Y016 Unions shouldn't contain duplicates, e.g. str | str is not allowed.
  • Y017 Stubs should not contain assignments with multiple targets or non-name targets.
  • Y018 A private TypeVar should be used at least once in the file in which it is defined.
  • Y019 Certain kinds of methods should use _typeshed.Self instead of defining custom TypeVars for their return annotation. This check currently applies for instance methods that return self, class methods that return an instance of cls, and __new__ methods.
  • Y020 Quoted annotations should never be used in stubs.
  • Y021 Docstrings should not be included in stubs.
  • Y022 Imports linting: use typing-module aliases to stdlib objects as little as possible (e.g. builtins.list over typing.List, collections.Counter over typing.Counter, etc.). (UP035)
  • Y023 Where there is no detriment to backwards compatibility, import objects such as ClassVar and NoReturn from typing rather than typing_extensions.
  • Y024 Use typing.NamedTuple instead of collections.namedtuple, as it allows for more precise type inference.
  • Y025 Always alias collections.abc.Set when importing it, so as to avoid confusion with builtins.set.
  • Y026 Type aliases should be explicitly demarcated with typing.TypeAlias.
  • Y027 Same as Y022. Unlike Y022, however, the imports disallowed with this error code are required if you wish to write Python 2-compatible stubs. Switch this error code off in your config file if you support Python 2.
  • Y028 Always use class-based syntax for typing.NamedTuple, instead of assignment-based syntax. (UP014)
  • Y029 It is almost always redundant to define __str__ or __repr__ in a stub file, as the signatures are almost always identical to object.__str__ and object.__repr__.
  • Y030 Union expressions should never have more than one Literal member, as Literal[1] | Literal[2] is semantically identical to Literal[1, 2].
  • Y031 TypedDicts should use class-based syntax instead of assignment-based syntax wherever possible. (In situations where this is not possible, such as if a field is a Python keyword or an invalid identifier, this error will not be raised.) (UP013)
  • Y032 The second argument of an __eq__ or __ne__ method should usually be annotated with object rather than Any.
  • Y033 Do not use type comments (e.g. x = ... # type: int) in stubs, even if the stub supports Python 2. Always use annotations instead (e.g. x: int).
  • Y034 Y034 detects common errors where certain methods are annotated as having a fixed return type, despite returning self at runtime. Such methods should be annotated with _typeshed.Self. This check looks for:

  1.  Any in-place BinOp dunder methods (__iadd__, __ior__, etc.) that do not return Self.
  2.  __new__, __enter__ and __aenter__ methods that return the class's name unparameterised.
  3.  __iter__ methods that return Iterator, even if the class inherits directly from Iterator.
  4.  __aiter__ methods that return AsyncIterator, even if the class inherits directly from AsyncIterator.

This check excludes methods decorated with @overload or @AbstractMethod.

  • Y035 __all__ and __match_args__ in a stub file should always have values, as these special variables in a .pyi file have identical semantics to __all__ and __match_args__ in a .py file. E.g. write __all__ = ["foo", "bar"] instead of __all__: list[str].
  • Y036 Y036 detects common errors in __exit__ and __aexit__ methods. For example, the first argument in an __exit__ method should either be annotated with object or type[BaseException] | None.
  • Y037 Use PEP 604 syntax instead of typing.Union and typing.Optional. E.g. use str | int instead of Union[str, int], and use str | None instead of Optional[str]. (UP007)
  • Y038 Use from collections.abc import Set as AbstractSet instead of from typing import AbstractSet. Like Y027, this error code should be switched off in your config file if your stubs support Python 2. (UP035)
  • Y039 Use str instead of typing.Text. This error code is incompatible with stubs supporting Python 2. (Implemented as UP019.)
  • Y040 Never explicitly inherit from object, as all classes implicitly inherit from object in Python 3. This error code is incompatible with stubs supporting Python 2. (Implemented as UP004.)
  • Y041 Y041 detects redundant numeric unions. For example, PEP 484 specifies that type checkers should treat int as an implicit subtype of float, so int is redundant in the union int | float. In the same way, int is redundant in the union int | complex, and float is redundant in the union float | complex.
  • Y042 Type alias names should use CamelCase rather than snake_case
  • Y043 Do not use names ending in "T" for private type aliases. (The "T" suffix implies that an object is a TypeVar.)
  • Y044 from __future__ import annotations has no effect in stub files, since type checkers automatically treat stubs as having those semantics.
  • Y045 __iter__ methods should never return Iterable[T], as they should always return some kind of iterator.
  • Y046 A private Protocol should be used at least once in the file in which it is defined.
  • Y047 A private TypeAlias should be used at least once in the file in which it is defined.
  • Y048 Function bodies should contain exactly one statement. (Note that if a function body includes a docstring, the docstring counts as a "statement".)
  • Y049 A private TypedDict should be used at least once in the file in which it is defined.
  • Y050 Prefer typing_extensions.Never over typing.NoReturn for argument annotations. This is a purely stylistic choice in the name of readability.
  • Y051 Y051 detect redundant unions between Literal types and builtin supertypes. For example, Literal[5] is redundant in the union int | Literal[5], and Literal[True] is redundant in the union Literal[True] | bool.
  • Y052 Y052 disallows assignments to constant values where the assignment does not have a type annotation. For example, x = 0 in the global namespace is ambiguous in a stub, as there are four different types that could be inferred for the variable x: int, Final[int], Literal[0], or Final[Literal[0]]. Enum members are excluded from this check, as are various special assignments such as __all__ and __match_args__.
  • Y053 Only string and bytes literals <=50 characters long are permitted.
  • Y054 Only numeric literals with a string representation <=10 characters long are permitted.
  • Y055 Unions of the form type[X] \| type[Y] can be simplified to type[X \| Y]. Similarly, Union[type[X], type[Y]] can be simplified to type[Union[X, Y]].
  • Y056 Do not call methods such as .append(), .extend() or .remove() on __all__. Different type checkers have varying levels of support for calling these methods on __all__. Use += instead, which is known to be supported by all major type checkers.
@SigureMo
Copy link
Contributor

Flake8-pyi currently only works with .pyi files (PyCQA/flake8-pyi#118), it would be great if Ruff could support .py files.

Flake8-pyi author list some rules in qutebrowser/qutebrowser#7098 (comment) that can be easily apply to .py files

In addition, there are some rules that overlap with pyupgrade, such as Y037 and U007.

@charliermarsh charliermarsh added the rule Implementing or modifying a lint rule label Nov 21, 2022
@charliermarsh charliermarsh added plugin Implementing a known but unsupported plugin and removed rule Implementing or modifying a lint rule labels Dec 31, 2022
charliermarsh pushed a commit that referenced this issue Feb 10, 2023
Add basic scaffold for [flake8-pyi](https://github.com/PyCQA/flake8-pyi) and the first rule, Y001

rel: #848
@charliermarsh
Copy link
Member

These are being implemented under the PYI prefix (we tend to avoid single-letter prefixes like Y, since they lead to conflicts).

sbdchd added a commit to sbdchd/ruff that referenced this issue Feb 26, 2023
PYI009 and PYI010 are very similar, always use `...` in function and
class bodies in stubs.

PYI021 bans doc strings in stubs.

rel: astral-sh#848
@sbdchd
Copy link
Contributor

sbdchd commented Feb 26, 2023

Since opening this issue Y014 and Y011 have changed their behavior to allow some defaults, do you think we should mirror the new behavior?

PyCQA/flake8-pyi#326
PyCQA/flake8-pyi#316

@charliermarsh
Copy link
Member

@sbdchd - Yeah I'd vote to mirror their behavior.

charliermarsh pushed a commit that referenced this issue Feb 26, 2023
PYI009 and PYI010 are very similar, always use `...` in function and class bodies in stubs.

PYI021 bans doc strings in stubs.

I think all of these rules should be relatively straightforward to implement auto fixes for but can do that later once we get all the other rules added.

rel: #848
@fschulze
Copy link

fschulze commented Mar 6, 2023

I second @SigureMo, it would be great if some rules like Y006 would apply to *.py files as well.

konstin pushed a commit that referenced this issue Jul 19, 2023
## Summary

Implements PYI036 from `flake8-pyi`. See [original
code](https://github.com/PyCQA/flake8-pyi/blob/main/pyi.py#L1585)

## Test Plan

- Updated snapshots
- Checked against manual runs of flake8

ref: #848
konstin pushed a commit that referenced this issue Jul 19, 2023
## Summary

Implements PYI041 from flake8-pyi. See [original
code](https://github.com/PyCQA/flake8-pyi/blob/2a86db8271dc500247a8a69419536240334731cf/pyi.py#L1283).

This check only applies to function parameters in order to avoid issues
with mypy. See PyCQA/flake8-pyi#299.

ref: #848

## Test Plan

Snapshots, manual runs of flake8.
@qdegraaf
Copy link
Contributor

I'm implementing PYI017

charliermarsh pushed a commit that referenced this issue Jul 20, 2023
## Summary
Checks for `typehint.TypeAlias` annotation in type aliases. See
[original
source](https://github.com/PyCQA/flake8-pyi/blob/main/pyi.py#L1085).
```
$ flake8 --select Y026 crates/ruff/resources/test/fixtures/flake8_pyi/PYI026.pyi
crates/ruff/resources/test/fixtures/flake8_pyi/PYI026.pyi:4:1: Y026 Use typing_extensions.TypeAlias for type aliases, e.g. "NewAny: TypeAlias = Any"
crates/ruff/resources/test/fixtures/flake8_pyi/PYI026.pyi:5:1: Y026 Use typing_extensions.TypeAlias for type aliases, e.g. "OptinalStr: TypeAlias = typing.Optional[str]"
crates/ruff/resources/test/fixtures/flake8_pyi/PYI026.pyi:6:1: Y026 Use typing_extensions.TypeAlias for type aliases, e.g. "Foo: TypeAlias = Literal['foo']"
crates/ruff/resources/test/fixtures/flake8_pyi/PYI026.pyi:7:1: Y026 Use typing_extensions.TypeAlias for type aliases, e.g. "IntOrStr: TypeAlias = int | str"
crates/ruff/resources/test/fixtures/flake8_pyi/PYI026.pyi:8:1: Y026 Use typing_extensions.TypeAlias for type aliases, e.g. "AliasNone: TypeAlias = None"
```

```
$ ./target/debug/ruff --select PYI026 crates/ruff/resources/test/fixtures/flake8_pyi/PYI026.pyi --no-cache
crates/ruff/resources/test/fixtures/flake8_pyi/PYI026.pyi:4:1: PYI026 Use `typing.TypeAlias` for type aliases in `NewAny`, e.g. "NewAny: typing.TypeAlias = Any"
crates/ruff/resources/test/fixtures/flake8_pyi/PYI026.pyi:5:1: PYI026 Use `typing.TypeAlias` for type aliases in `OptinalStr`, e.g. "OptinalStr: typing.TypeAlias = typing.Optional[str]"
crates/ruff/resources/test/fixtures/flake8_pyi/PYI026.pyi:6:1: PYI026 Use `typing.TypeAlias` for type aliases in `Foo`, e.g. "Foo: typing.TypeAlias = Literal["foo"]"
crates/ruff/resources/test/fixtures/flake8_pyi/PYI026.pyi:7:1: PYI026 Use `typing.TypeAlias` for type aliases in `IntOrStr`, e.g. "IntOrStr: typing.TypeAlias = int | str"
crates/ruff/resources/test/fixtures/flake8_pyi/PYI026.pyi:8:1: PYI026 Use `typing.TypeAlias` for type aliases in `AliasNone`, e.g. "AliasNone: typing.TypeAlias = None"
Found 5 errors.
```

ref: #848 

## Test Plan

Snapshots, manual runs of flake8.
charliermarsh pushed a commit that referenced this issue Jul 20, 2023
## Summary

Implements `PYI017` or `Y017` from `flake8-pyi` plug-in. Mirrors
[upstream
implementation](https://github.com/PyCQA/flake8-pyi/blob/ceab86d16b822d12abae1d8087850d0bc66d2f52/pyi.py#L1039-L1048).
It checks for any assignment with more than 1 target or an assignment to
anything other than a name, and raises a violation for these in stub
files.

Couldn't find a clear and concise explanation for why this is to be
avoided and what is preferred for attribute cases like:

```python
a.b = int
```
So welcome some input there, to learn and to finish up the docs.

## Test Plan

Added test cases from upstream plug-in in a fixture (both `.py` and
`.pyi`). Added a few more.

## Issue link

Refers: #848
@LaBatata101
Copy link
Contributor

I'm implementing PYI056

charliermarsh pushed a commit that referenced this issue Jul 22, 2023
## Summary

Checks that `append`, `extend` and `remove` methods are not called on
`__all__`. See [original
implementation](https://github.com/PyCQA/flake8-pyi/blob/2a86db8271dc500247a8a69419536240334731cf/pyi.py#L1133-L1138).

```
$ flake8 --select Y026 crates/ruff/resources/test/fixtures/flake8_pyi/PYI056.pyi

crates/ruff/resources/test/fixtures/flake8_pyi/PYI056.pyi:3:1: Y056 Calling ".append()" on "__all__" may not be supported by all type checkers (use += instead)
crates/ruff/resources/test/fixtures/flake8_pyi/PYI056.pyi:4:1: Y056 Calling ".extend()" on "__all__" may not be supported by all type checkers (use += instead)
crates/ruff/resources/test/fixtures/flake8_pyi/PYI056.pyi:5:1: Y056 Calling ".remove()" on "__all__" may not be supported by all type checkers (use += instead)
```

```
$ ./target/debug/ruff --select PYI026 crates/ruff/resources/test/fixtures/flake8_pyi/PYI056.pyi --no-cache

crates/ruff/resources/test/fixtures/flake8_pyi/PYI056.pyi:3:1: PYI056 Calling ".append()" on "__all__" may not be supported by all type checkers (use += instead)
crates/ruff/resources/test/fixtures/flake8_pyi/PYI056.pyi:4:1: PYI056 Calling ".extend()" on "__all__" may not be supported by all type checkers (use += instead)
crates/ruff/resources/test/fixtures/flake8_pyi/PYI056.pyi:5:1: PYI056 Calling ".remove()" on "__all__" may not be supported by all type checkers (use += instead)
Found 3 errors.
```

ref #848

## Test Plan

Snapshots and manual runs of flake8.
@LaBatata101
Copy link
Contributor

I'm implementing PYI018 and PYI046, PYI047, PYI049 (which are similar to PYI018)

charliermarsh pushed a commit that referenced this issue Jul 26, 2023
## Summary

Check for unused private `TypeVar`. See [original
implementation](https://github.com/PyCQA/flake8-pyi/blob/2a86db8271dc500247a8a69419536240334731cf/pyi.py#L1958).

```
$ flake8 --select Y018 crates/ruff/resources/test/fixtures/flake8_pyi/PYI018.pyi

crates/ruff/resources/test/fixtures/flake8_pyi/PYI018.pyi:4:1: Y018 TypeVar "_T" is not used
crates/ruff/resources/test/fixtures/flake8_pyi/PYI018.pyi:5:1: Y018 TypeVar "_P" is not used
```

```
$ ./target/debug/ruff --select PYI018 crates/ruff/resources/test/fixtures/flake8_pyi/PYI018.pyi --no-cache

crates/ruff/resources/test/fixtures/flake8_pyi/PYI018.pyi:4:1: PYI018 TypeVar `_T` is never used
crates/ruff/resources/test/fixtures/flake8_pyi/PYI018.pyi:5:1: PYI018 TypeVar `_P` is never used
Found 2 errors.
```
In the file `unused_private_type_declaration.rs`, I'm planning to add
other rules that are similar to `PYI018` like the `PYI046`, `PYI047` and
`PYI049`.

ref #848

## Test Plan

Snapshots and manual runs of flake8.
charliermarsh pushed a commit that referenced this issue Jul 27, 2023
## Summary
Checks for the presence of unused private `typing.Protocol` definitions.

ref #848 

## Test Plan

Snapshots and manual runs of flake8.
This was referenced Jul 27, 2023
@LaBatata101
Copy link
Contributor

I'm implementing PYI051

charliermarsh pushed a commit that referenced this issue Jul 29, 2023
## Summary

Checks for the presence of unused private `typing.TypeAlias`
definitions.

ref #848 

## Test Plan

Snapshots and manual runs of flake8
charliermarsh pushed a commit that referenced this issue Jul 29, 2023
## Summary

Checks for the presence of unused private `typing.TypedDict`
definitions.

ref #848 

## Test Plan

Snapshots and manual runs of flake8
@qdegraaf
Copy link
Contributor

I'm implementing PYI019

@LaBatata101
Copy link
Contributor

I'm implementing PYI055

charliermarsh pushed a commit that referenced this issue Aug 2, 2023
## Summary
Checks for the presence of redundant `Literal` types and builtin super
types in an union. See [original
source](https://github.com/PyCQA/flake8-pyi/blob/2a86db8271dc500247a8a69419536240334731cf/pyi.py#L1261).

This implementation has a couple of differences from the original. The
first one is, we support the `complex` and `float` builtin types. The
second is, when reporting diagnostic for a `Literal` with multiple
members of the same type, we print the entire `Literal` while `flak8`
only prints the `Literal` with its first member.
For example:
```python
from typing import Literal

x: Literal[1, 2] | int
```  
Ruff will show `Literal[1, 2]` while flake8 only shows `Literal[1]`.

```shell
$ ruff crates/ruff/resources/test/fixtures/flake8_pyi/PYI051.pyi
crates/ruff/resources/test/fixtures/flake8_pyi/PYI051.pyi:4:18: PYI051 `Literal["foo"]` is redundant in an union with `str`
crates/ruff/resources/test/fixtures/flake8_pyi/PYI051.pyi:5:37: PYI051 `Literal[b"bar", b"foo"]` is redundant in an union with `bytes`
crates/ruff/resources/test/fixtures/flake8_pyi/PYI051.pyi:6:37: PYI051 `Literal[5]` is redundant in an union with `int`
crates/ruff/resources/test/fixtures/flake8_pyi/PYI051.pyi:6:67: PYI051 `Literal["foo"]` is redundant in an union with `str`
crates/ruff/resources/test/fixtures/flake8_pyi/PYI051.pyi:7:37: PYI051 `Literal[b"str_bytes"]` is redundant in an union with `bytes`
crates/ruff/resources/test/fixtures/flake8_pyi/PYI051.pyi:7:51: PYI051 `Literal[42]` is redundant in an union with `int`
crates/ruff/resources/test/fixtures/flake8_pyi/PYI051.pyi:9:31: PYI051 `Literal[1J]` is redundant in an union with `complex`
crates/ruff/resources/test/fixtures/flake8_pyi/PYI051.pyi:9:53: PYI051 `Literal[3.14]` is redundant in an union with `float`
Found 8 errors.
```

```shell
$ flake8 crates/ruff/resources/test/fixtures/flake8_pyi/PYI051.pyi
crates/ruff/resources/test/fixtures/flake8_pyi/PYI051.pyi:4:18: Y051 "Literal['foo']" is redundant in a union with "str"
crates/ruff/resources/test/fixtures/flake8_pyi/PYI051.pyi:5:37: Y051 "Literal[b'bar']" is redundant in a union with "bytes"
crates/ruff/resources/test/fixtures/flake8_pyi/PYI051.pyi:6:37: Y051 "Literal[5]" is redundant in a unionwith "int"
crates/ruff/resources/test/fixtures/flake8_pyi/PYI051.pyi:6:67: Y051 "Literal['foo']" is redundant in a union with "str"
crates/ruff/resources/test/fixtures/flake8_pyi/PYI051.pyi:7:37: Y051 "Literal[b'str_bytes']" is redundantin a union with "bytes"
crates/ruff/resources/test/fixtures/flake8_pyi/PYI051.pyi:7:51: Y051 "Literal[42]" is redundant in a union with "int"
```

While implementing this rule, I found a bug in the `is_unchecked_union`
check. This is the new check.


https://github.com/astral-sh/ruff/blob/1ab86bad35e5edd1ba4cd82b0eb4c78416509dac/crates/ruff/src/checkers/ast/analyze/expression.rs#L85-L102

The purpose of the check was to prevent rules from navigating through
nested `Union`s, as they already handle nested `Union`s. The way it was
implemented, this was not happening, the rules were getting executed
more than one time and sometimes were receiving expressions that were
not `Union`. For example, with the following code:
 ```python
  typing.Union[Literal[5], int, typing.Union[Literal["foo"], str]]
 ```

The rules were receiving the expressions in the following order:
- `typing.Union[Literal[5], int, typing.Union[Literal["foo"], str]]`
     - `Literal[5]`
     - `typing.Union[Literal["foo"], str]]`

This was causing `PYI030` to report redundant information, for example:
 ```python
typing.Union[Literal[5], int, typing.Union[Literal["foo"],
Literal["bar"]]]
 ```
This is the `PYI030` output for this code:
```shell
PYI030 Multiple literal members in a union. Use a single literal, e.g. `Literal[5, "foo", "bar"]`
YI030 Multiple literal members in a union. Use a single literal, e.g.`Literal[5, "foo"]`
```

If I haven't misinterpreted the rule, that looks incorrect. I didn't
have the time to check the `PYI016` rule.

The last thing is, I couldn't find a reason for the "Why is this bad?"
section for `PYI051`.

Ref: #848 

## Test Plan

Snapshots and manual runs of flake8.
\
@LaBatata101
Copy link
Contributor

I'm implementing PYI023

charliermarsh pushed a commit that referenced this issue Aug 4, 2023
…(UP035) (#6354)

## Summary
As of version
[23.1.0](https://github.com/PyCQA/flake8-pyi/blob/2a86db8271dc500247a8a69419536240334731cf/CHANGELOG.md?plain=1#L158-L160),
`flake8-pyi` remove the rule `Y027`.

The errors that resulted in `PYI027` are now being emitted by `PYI022`
(`UP035`).

ref: #848 

## Test Plan

Add new tests cases.
@charliermarsh
Copy link
Member

Done! Thanks everyone!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
plugin Implementing a known but unsupported plugin
Projects
None yet
Development

No branches or pull requests

9 participants