[`flake8-pyi`] Implement duplicate types in unions (`PYI016`) #3922

USER-5 · 2023-04-09T04:01:20Z

Addresses Y016 in #848:

Unions shouldn't contain duplicates, e.g. str | str is not allowed.

I hope I'm correct in assuming that it's always fixable - and that a suitable fix is to remove the last duplicates.

github-actions · 2023-04-09T04:17:29Z

PR Check Results

Ecosystem

✅ ecosystem check detected no changes.

Benchmark

Linux

group                                      main                                   pr
-----                                      ----                                   --
linter/all-rules/large/dataset.py          1.00     14.5±0.51ms     2.8 MB/sec    1.03     15.0±0.60ms     2.7 MB/sec
linter/all-rules/numpy/ctypeslib.py        1.03      3.9±0.15ms     4.3 MB/sec    1.00      3.8±0.15ms     4.4 MB/sec
linter/all-rules/numpy/globals.py          1.02   503.9±22.32µs     5.9 MB/sec    1.00   495.6±28.52µs     6.0 MB/sec
linter/all-rules/pydantic/types.py         1.03      6.5±0.31ms     3.9 MB/sec    1.00      6.3±0.28ms     4.1 MB/sec
linter/default-rules/large/dataset.py      1.01      7.7±0.31ms     5.3 MB/sec    1.00      7.6±0.27ms     5.4 MB/sec
linter/default-rules/numpy/ctypeslib.py    1.03  1746.5±88.77µs     9.5 MB/sec    1.00  1689.2±68.35µs     9.9 MB/sec
linter/default-rules/numpy/globals.py      1.05   198.0±10.30µs    14.9 MB/sec    1.00   188.4±12.23µs    15.7 MB/sec
linter/default-rules/pydantic/types.py     1.00      3.5±0.16ms     7.4 MB/sec    1.02      3.5±0.17ms     7.2 MB/sec

Windows

group                                      main                                   pr
-----                                      ----                                   --
linter/all-rules/large/dataset.py          1.01     15.9±0.08ms     2.6 MB/sec    1.00     15.7±0.09ms     2.6 MB/sec
linter/all-rules/numpy/ctypeslib.py        1.01      4.1±0.08ms     4.1 MB/sec    1.00      4.1±0.04ms     4.1 MB/sec
linter/all-rules/numpy/globals.py          1.02    507.2±9.71µs     5.8 MB/sec    1.00    496.0±6.24µs     5.9 MB/sec
linter/all-rules/pydantic/types.py         1.01      6.8±0.08ms     3.8 MB/sec    1.00      6.7±0.06ms     3.8 MB/sec
linter/default-rules/large/dataset.py      1.02      8.3±0.20ms     4.9 MB/sec    1.00      8.1±0.09ms     5.0 MB/sec
linter/default-rules/numpy/ctypeslib.py    1.01  1858.5±46.64µs     9.0 MB/sec    1.00  1833.2±38.63µs     9.1 MB/sec
linter/default-rules/numpy/globals.py      1.04    203.0±8.77µs    14.5 MB/sec    1.00    195.7±3.98µs    15.1 MB/sec
linter/default-rules/pydantic/types.py     1.04      4.0±0.11ms     6.4 MB/sec    1.00      3.8±0.13ms     6.7 MB/sec

USER-5 · 2023-04-09T06:24:58Z

crates/ruff/src/rules/flake8_pyi/rules/duplicates_in_union.rs

+    // right is the duplicate (since we will have already checked the others in other invocations)
+
+    // Collapse down the left side of the left expression into a vector of nodes
+    let mut left_nodes: Vec<ExprKind> = Vec::new();


I was using a HashSet here, but it looks like ExprKind doesn't implement HashCode & Equals

While large unions are not common, I'd definitely say they're possible. So an O(n^2) algorithm does seems problematic.

We have a thing to help with this -- check out ComparableExpr. You can convert from Expr to ComparableExpr to get location-agnostic hash and equality comparisons.

We use this (e.g.) for the "repeated keys in dictionary" checks (repeated_keys).

Turns out I'd over-complicated things - I've reduced this part down to use the ComparableExpr and FxHashSet (I assume we're using that over the regular HashSet for speed in this project?).

The violation code is recursive now, which I hope is ok.

bluetech · 2023-04-09T15:50:02Z

The field6 = 1 | 0 | 1 case seems like a false-positive to me. I think this lint should only trigger when the | occurs in a typing context. I haven't checked, but I bet Ruff is already able to recognize typing contexts because there's a lot of lints which depend on it.

(Would also be nice to emit this lint for Union[int, int] and x: "int | int" but that's extra)

charliermarsh · 2023-04-09T22:29:53Z

...but I bet Ruff is already able to recognize typing contexts because there's a lot of lints which depend on it.

Haven't reviewed fully yet but yes: we do have this ability. You probably want to check on checker.ctx.in_type_definition.

USER-5 · 2023-04-09T23:32:02Z

Is this a type-only lint? I originally had it that way but changed it because the description didn't mention types.

bluetech · 2023-04-10T12:25:51Z

Is this a type-only lint? I originally had it that way but changed it because the description didn't mention types.

IMO this should definitely be a typing lint. The | operator means something else in non-typing contexts, and this lint can be more useful if it's specific to typing (e.g. also handling Union and string annotations).

Relatedly, I believe flake8-pyi only runs on pyi files, but I don't see a reason to restrict this particular lint in Ruff only to pyi files.

USER-5 · 2023-04-10T12:32:22Z

IMO this should definitely be a typing lint. The | operator means something else in non-typing contexts, and this lint can be more useful if it's specific to typing (e.g. also handling Union and string annotations).

AFAIK in non-typing contexts it's a bitwise-or, which is equally pointless to have duplicates in (unless there are other uses of it), but I'm happy to change this to type-definitions only.

Relatedly, I believe flake8-pyi only runs on pyi files, but I don't see a reason to restrict this particular lint in Ruff only to pyi files.

Yeah, I believe there have been a few comments about enabling it for .py files. I didn't add anything in this branch to restrict it to only pyi files, so I assume that's being done somewhere else in the code.

bluetech · 2023-04-10T12:54:00Z

crates/ruff/src/rules/flake8_pyi/rules/duplicate_types_in_union.rs

+
+///PYI016
+pub fn duplicate_types_in_union<'a>(
+    mut seen_nodes: FxHashSet<ComparableExpr<'a>>,


You are passing and returning FxHashSet by value, which means it's (theoretically) copied each time you pass and return it. Consider instead taking a &mut FxHashSet<ComparableExpr<'a>> and returning nothing. This will eliminate the copies.

I had some issues with the recursive calls when I did this thanks to the borrow checker. But I'll give it another crack now that I've got lifetime annotations here.

charliermarsh · 2023-04-11T04:12:46Z

crates/ruff/src/rules/flake8_pyi/rules/duplicate_union_member.rs

+}
+
+/// PYI016
+pub fn duplicate_union_member(checker: &mut Checker, expr: &Expr) {


I tweaked this to use an inner helper, so that the caller doesn't have to worry or know about (e.g.) creating a hash set for internal use.

charliermarsh · 2023-04-11T04:12:57Z

crates/ruff/src/rules/flake8_pyi/rules/duplicate_union_member.rs

+impl AlwaysAutofixableViolation for DuplicateUnionMember {
+    #[derive_message_formats]
+    fn message(&self) -> String {
+        format!("Duplicate union member `{}`", self.duplicate_name)


Changed this to use the same message as flake8-pyi, I think.

crates/ruff/src/rules/flake8_pyi/rules/duplicate_union_member.rs

charliermarsh · 2023-04-11T04:15:13Z

crates/ruff/src/rules/flake8_pyi/rules/duplicate_union_member.rs

+                // SAFETY: impossible to have a duplicate without a `previous` node.
+                previous.unwrap().end_location.unwrap(),
+                expr.end_location.unwrap(),
+            ));


I think this can lead to syntax errors in the presence of parentheses. E.g., if you try to autofix this, we throw a syntax error:

field2: (str | int) | str # PYI016 Duplicate name in union

I'm guessing the issue is that the previous end_location won't include the parenthesis -- so we change to:

field2: (str | int # PYI016 Duplicate name in union

(I'm guessing here, I haven't looked at the erroneous source code.)

As an alternative strategy, we could use unparse_expr, and re-create the source expr, but with the duplicate expression removed. (That function takes an AST node and generates source code from the AST.) It'll be guaranteed to be correct and valid syntax, but we will lose trivia (comments, optional user-added parentheses) -- that seems ok here, since comments and such are pretty rare within these unions?

Hm, and there is also:

field: str | (str | int)

Which would presumably get corrected to

field: str | int)

In the above case, we would either want to:

delete the | character after the duplicate

reconstruct from unparse - removing all brackets in the process

delete as we do currently, then re-introduce a bracket at the correct location (this might be the easiest?)

~~I'm not sure what other "fix" tools I have at my disposal to help achieve this.~~

I just saw your edit about unparse_expr so I think I have a way forward.

I've hopefully fixed this issue, and I've added a couple test cases to the fixture for parentheses cases, which work as I'd expect:

field9: int | (int | str) # PYI016 Duplicate union member `int` field10: (str | int) | str # PYI016 Duplicate union member `str` # Gets auto fixed to field9: int | (str) # PYI016 Duplicate union member `int` field10: str | int # PYI016 Duplicate union member `str`

Cool, this looks good to me!

Yeah, the approach here makes sense. I guess I was suggesting taking the top-level expression (the thing we pass into duplicate_union_member) and recursively filtering to find the duplicated child, but what you settled on here seems better.

charliermarsh · 2023-04-11T04:15:37Z

Great work! Should be good to merge once we correct that one nuance in the autofix.

charliermarsh · 2023-04-11T04:16:31Z

crates/ruff/src/checkers/ast/mod.rs

+                op: Operator::BitOr,
+                ..
+            } => {
+                if self.is_stub {


Oh sorry -- I also changed this to .pyi only for consistency with the other flake8-pyi rules. I want to enable these for non-.pyi files, but I want to do it all at once for the plugin.

USER-5 · 2023-04-11T13:10:11Z

crates/ruff/src/rules/flake8_pyi/rules/duplicate_union_member.rs

+            let ExprKind::BinOp { left, right, .. } = &parent.node
+            else {
+                unreachable!();
+            };


I don't know if there's a better way to do this unwrapping.

charliermarsh · 2023-04-12T04:03:23Z

crates/ruff/src/checkers/ast/mod.rs

+                                    op: Operator::BitOr,
+                                    ..
+                                }
+                            )


@USER-5 - I loosened this check a bit, I think we just want to avoid flagging members that already would've been handled by the recursive check.

As-is, we weren't flagging cases like dict[str | str, int].

[![Mend Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com) This PR contains the following updates: | Package | Change | Age | Adoption | Passing | Confidence | |---|---|---|---|---|---| | [ruff](https://togithub.com/charliermarsh/ruff) | `^0.0.261` -> `^0.0.262` | [![age](https://badges.renovateapi.com/packages/pypi/ruff/0.0.262/age-slim)](https://docs.renovatebot.com/merge-confidence/) | [![adoption](https://badges.renovateapi.com/packages/pypi/ruff/0.0.262/adoption-slim)](https://docs.renovatebot.com/merge-confidence/) | [![passing](https://badges.renovateapi.com/packages/pypi/ruff/0.0.262/compatibility-slim/0.0.261)](https://docs.renovatebot.com/merge-confidence/) | [![confidence](https://badges.renovateapi.com/packages/pypi/ruff/0.0.262/confidence-slim/0.0.261)](https://docs.renovatebot.com/merge-confidence/) | --- ### Release Notes <details> <summary>charliermarsh/ruff</summary> ### [`v0.0.262`](https://togithub.com/charliermarsh/ruff/releases/tag/v0.0.262) [Compare Source](https://togithub.com/charliermarsh/ruff/compare/v0.0.261...v0.0.262)  #### What's Changed ##### Configuration - Allow users to extend the set of included files via `include` by [@charliermarsh](https://togithub.com/charliermarsh) in [astral-sh/ruff#3914 - Implement isort custom sections and ordering ([#2419](https://togithub.com/charliermarsh/ruff/issues/2419)) by [@hackedd](https://togithub.com/hackedd) in [astral-sh/ruff#3900 ##### Rules - \[`flake8-simplify`] Add autofix for `contextlib.suppress` (`SIM105`) by [@leiserfg](https://togithub.com/leiserfg) in [astral-sh/ruff#3915 - \[`flake8-bandit`] Ignore assert errors (S101) in `TYPE_CHECKING` blocks by [@charliermarsh](https://togithub.com/charliermarsh) in [astral-sh/ruff#3960 - \[`flake8-comprehensions`] Implement `unnecessary-literal-within-dict-call` (`C418`) by [@charliermarsh](https://togithub.com/charliermarsh) in [astral-sh/ruff#3969 - \[`ruff`] Add checks for mutable defaults `dataclass`es by [@mosauter](https://togithub.com/mosauter) in [astral-sh/ruff#3877 - \[`flake8-import-conventions`] Add a rule for `BannedImportAlias` by [@stancld](https://togithub.com/stancld) in [astral-sh/ruff#3926 - \[`flake8-pyi`] Implement duplicate types in unions (`PYI016`) by [@USER-5](https://togithub.com/USER-5) in [astral-sh/ruff#3922 - \[`flake8-bandit`] Implement flake8-bandit shell injection rules by [@robyoung](https://togithub.com/robyoung) in [astral-sh/ruff#3924 - \[`flake8-comprehensions`] Redirect `PIE802` to `C419` by [@charliermarsh](https://togithub.com/charliermarsh) in [astral-sh/ruff#3971 ##### Bug Fixes - Fix unicode handling in PLE2515 by [@konstin](https://togithub.com/konstin) in [astral-sh/ruff#3898 - Avoid adding required imports to stub files by [@charliermarsh](https://togithub.com/charliermarsh) in [astral-sh/ruff#3940 - Add 'or if cond' to `E712` message by [@charliermarsh](https://togithub.com/charliermarsh) in [astral-sh/ruff#3962 - Ignore argument assignments when enforcing `RET504` by [@charliermarsh](https://togithub.com/charliermarsh) in [astral-sh/ruff#4004 - Fix (doc-)line-too-long start location by [@MichaReiser](https://togithub.com/MichaReiser) in [astral-sh/ruff#4006 - Ignore stub file assignments to value-requiring targets by [@charliermarsh](https://togithub.com/charliermarsh) in [astral-sh/ruff#4030 - Allow legacy C and T selectors in JSON schema by [@charliermarsh](https://togithub.com/charliermarsh) in [astral-sh/ruff#3889 - Ignore `PLW2901` when using typing cast by [@dhruvmanila](https://togithub.com/dhruvmanila) in [astral-sh/ruff#3891 - Visit comprehension to detect group name usage/overrides by [@dhruvmanila](https://togithub.com/dhruvmanila) in [astral-sh/ruff#3887 - Ensure that tab characters aren't in multi-line strings before throwing a violation by [@evanrittenhouse](https://togithub.com/evanrittenhouse) in [astral-sh/ruff#3837 - Avoid N802 violations for `@override` methods by [@charliermarsh](https://togithub.com/charliermarsh) in [astral-sh/ruff#3912 - Check for arguments in inner/outer call for `C414` by [@dhruvmanila](https://togithub.com/dhruvmanila) in [astral-sh/ruff#3916 - Do not skip analysis if `*args` present for `F523` by [@dhruvmanila](https://togithub.com/dhruvmanila) in [astral-sh/ruff#3923 - Extend SIM105 to match also 'Ellipsis only' bodies in exception handlers by [@leiserfg](https://togithub.com/leiserfg) in [astral-sh/ruff#3925 - Support `pyright: ignore` comments by [@charliermarsh](https://togithub.com/charliermarsh) in [astral-sh/ruff#3941 - Tidy up some `pygrep-hooks` rules by [@charliermarsh](https://togithub.com/charliermarsh) in [astral-sh/ruff#3942 - Use identifier range for pytest rules by [@charliermarsh](https://togithub.com/charliermarsh) in [astral-sh/ruff#3948 - Allow `typing_extensions.TypeVar` assignments in `.pyi` files by [@charliermarsh](https://togithub.com/charliermarsh) in [astral-sh/ruff#3951 - Raise percent-format upgrade rule (`UP031`) for hanging modulos by [@charliermarsh](https://togithub.com/charliermarsh) in [astral-sh/ruff#3953 - Check for parenthesis in implicit str concat in `PT006` by [@dhruvmanila](https://togithub.com/dhruvmanila) in [astral-sh/ruff#3955 - Do not consider nested comment as part of code by [@dhruvmanila](https://togithub.com/dhruvmanila) in [astral-sh/ruff#3984 - Preserve type annotations when fixing `E731` by [@dhruvmanila](https://togithub.com/dhruvmanila) in [astral-sh/ruff#3983 - Remove autofix behavior for uncapitalized-environment-variables (`SIM112`) by [@charliermarsh](https://togithub.com/charliermarsh) in [astral-sh/ruff#3988 - Respect typing-modules when evaluating no-return functions by [@charliermarsh](https://togithub.com/charliermarsh) in [astral-sh/ruff#4001 - Avoid short-circuiting when detecting RET rules by [@charliermarsh](https://togithub.com/charliermarsh) in [astral-sh/ruff#4002 - Set non-empty range for indentation diagnostics by [@MichaReiser](https://togithub.com/MichaReiser) in [astral-sh/ruff#4005 - Ignore relative imports in `banned-api` rules by [@charliermarsh](https://togithub.com/charliermarsh) in [astral-sh/ruff#4024 - Support relative imports in `banned-api` enforcement by [@charliermarsh](https://togithub.com/charliermarsh) in [astral-sh/ruff#4025 - Treat non-future function annotations as required-at-runtime by [@charliermarsh](https://togithub.com/charliermarsh) in [astral-sh/ruff#4028 - Ignore certain flake8-pyi errors within function bodies by [@charliermarsh](https://togithub.com/charliermarsh) in [astral-sh/ruff#4029 #### New Contributors - [@tjkuson](https://togithub.com/tjkuson) made their first contribution in [astral-sh/ruff#3886 - [@mosauter](https://togithub.com/mosauter) made their first contribution in [astral-sh/ruff#3877 - [@stancld](https://togithub.com/stancld) made their first contribution in [astral-sh/ruff#3926 - [@USER-5](https://togithub.com/USER-5) made their first contribution in [astral-sh/ruff#3922 - [@robyoung](https://togithub.com/robyoung) made their first contribution in [astral-sh/ruff#3924 - [@hackedd](https://togithub.com/hackedd) made their first contribution in [astral-sh/ruff#3900 - [@justinchuby](https://togithub.com/justinchuby) made their first contribution in [astral-sh/ruff#3982 - [@mirecl](https://togithub.com/mirecl) made their first contribution in [astral-sh/ruff#4008 - [@Xemnas0](https://togithub.com/Xemnas0) made their first contribution in [astral-sh/ruff#4026 **Full Changelog**: astral-sh/ruff@v0.0.261...v0.0.262 </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Enabled. ♻ **Rebasing**: Whenever PR is behind base branch, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] If you want to rebase/retry this PR, check this box --- This PR has been generated by [Mend Renovate](https://www.mend.io/free-developer-tools/renovate/). View repository job log [here](https://app.renovatebot.com/dashboard#github/ixm-one/pytest-cmake-presets).  Signed-off-by: Renovate Bot <bot@renovateapp.com> Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

[![Mend Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com) This PR contains the following updates: | Package | Change | Age | Adoption | Passing | Confidence | |---|---|---|---|---|---| | [ruff](https://togithub.com/charliermarsh/ruff) | `==0.0.261` -> `==0.0.262` | [![age](https://badges.renovateapi.com/packages/pypi/ruff/0.0.262/age-slim)](https://docs.renovatebot.com/merge-confidence/) | [![adoption](https://badges.renovateapi.com/packages/pypi/ruff/0.0.262/adoption-slim)](https://docs.renovatebot.com/merge-confidence/) | [![passing](https://badges.renovateapi.com/packages/pypi/ruff/0.0.262/compatibility-slim/0.0.261)](https://docs.renovatebot.com/merge-confidence/) | [![confidence](https://badges.renovateapi.com/packages/pypi/ruff/0.0.262/confidence-slim/0.0.261)](https://docs.renovatebot.com/merge-confidence/) | --- ### Release Notes <details> <summary>charliermarsh/ruff</summary> ### [`v0.0.262`](https://togithub.com/charliermarsh/ruff/releases/tag/v0.0.262) [Compare Source](https://togithub.com/charliermarsh/ruff/compare/v0.0.261...v0.0.262)  #### What's Changed ##### Configuration - Allow users to extend the set of included files via `include` by [@charliermarsh](https://togithub.com/charliermarsh) in [astral-sh/ruff#3914 - Implement isort custom sections and ordering ([#2419](https://togithub.com/charliermarsh/ruff/issues/2419)) by [@hackedd](https://togithub.com/hackedd) in [astral-sh/ruff#3900 ##### Rules - \[`flake8-simplify`] Add autofix for `contextlib.suppress` (`SIM105`) by [@leiserfg](https://togithub.com/leiserfg) in [astral-sh/ruff#3915 - \[`flake8-bandit`] Ignore assert errors (S101) in `TYPE_CHECKING` blocks by [@charliermarsh](https://togithub.com/charliermarsh) in [astral-sh/ruff#3960 - \[`flake8-comprehensions`] Implement `unnecessary-literal-within-dict-call` (`C418`) by [@charliermarsh](https://togithub.com/charliermarsh) in [astral-sh/ruff#3969 - \[`ruff`] Add checks for mutable defaults `dataclass`es by [@mosauter](https://togithub.com/mosauter) in [astral-sh/ruff#3877 - \[`flake8-import-conventions`] Add a rule for `BannedImportAlias` by [@stancld](https://togithub.com/stancld) in [astral-sh/ruff#3926 - \[`flake8-pyi`] Implement duplicate types in unions (`PYI016`) by [@USER-5](https://togithub.com/USER-5) in [astral-sh/ruff#3922 - \[`flake8-bandit`] Implement flake8-bandit shell injection rules by [@robyoung](https://togithub.com/robyoung) in [astral-sh/ruff#3924 - \[`flake8-comprehensions`] Redirect `PIE802` to `C419` by [@charliermarsh](https://togithub.com/charliermarsh) in [astral-sh/ruff#3971 ##### Bug Fixes - Fix unicode handling in PLE2515 by [@konstin](https://togithub.com/konstin) in [astral-sh/ruff#3898 - Avoid adding required imports to stub files by [@charliermarsh](https://togithub.com/charliermarsh) in [astral-sh/ruff#3940 - Add 'or if cond' to `E712` message by [@charliermarsh](https://togithub.com/charliermarsh) in [astral-sh/ruff#3962 - Ignore argument assignments when enforcing `RET504` by [@charliermarsh](https://togithub.com/charliermarsh) in [astral-sh/ruff#4004 - Fix (doc-)line-too-long start location by [@MichaReiser](https://togithub.com/MichaReiser) in [astral-sh/ruff#4006 - Ignore stub file assignments to value-requiring targets by [@charliermarsh](https://togithub.com/charliermarsh) in [astral-sh/ruff#4030 - Allow legacy C and T selectors in JSON schema by [@charliermarsh](https://togithub.com/charliermarsh) in [astral-sh/ruff#3889 - Ignore `PLW2901` when using typing cast by [@dhruvmanila](https://togithub.com/dhruvmanila) in [astral-sh/ruff#3891 - Visit comprehension to detect group name usage/overrides by [@dhruvmanila](https://togithub.com/dhruvmanila) in [astral-sh/ruff#3887 - Ensure that tab characters aren't in multi-line strings before throwing a violation by [@evanrittenhouse](https://togithub.com/evanrittenhouse) in [astral-sh/ruff#3837 - Avoid N802 violations for `@override` methods by [@charliermarsh](https://togithub.com/charliermarsh) in [astral-sh/ruff#3912 - Check for arguments in inner/outer call for `C414` by [@dhruvmanila](https://togithub.com/dhruvmanila) in [astral-sh/ruff#3916 - Do not skip analysis if `*args` present for `F523` by [@dhruvmanila](https://togithub.com/dhruvmanila) in [astral-sh/ruff#3923 - Extend SIM105 to match also 'Ellipsis only' bodies in exception handlers by [@leiserfg](https://togithub.com/leiserfg) in [astral-sh/ruff#3925 - Support `pyright: ignore` comments by [@charliermarsh](https://togithub.com/charliermarsh) in [astral-sh/ruff#3941 - Tidy up some `pygrep-hooks` rules by [@charliermarsh](https://togithub.com/charliermarsh) in [astral-sh/ruff#3942 - Use identifier range for pytest rules by [@charliermarsh](https://togithub.com/charliermarsh) in [astral-sh/ruff#3948 - Allow `typing_extensions.TypeVar` assignments in `.pyi` files by [@charliermarsh](https://togithub.com/charliermarsh) in [astral-sh/ruff#3951 - Raise percent-format upgrade rule (`UP031`) for hanging modulos by [@charliermarsh](https://togithub.com/charliermarsh) in [astral-sh/ruff#3953 - Check for parenthesis in implicit str concat in `PT006` by [@dhruvmanila](https://togithub.com/dhruvmanila) in [astral-sh/ruff#3955 - Do not consider nested comment as part of code by [@dhruvmanila](https://togithub.com/dhruvmanila) in [astral-sh/ruff#3984 - Preserve type annotations when fixing `E731` by [@dhruvmanila](https://togithub.com/dhruvmanila) in [astral-sh/ruff#3983 - Remove autofix behavior for uncapitalized-environment-variables (`SIM112`) by [@charliermarsh](https://togithub.com/charliermarsh) in [astral-sh/ruff#3988 - Respect typing-modules when evaluating no-return functions by [@charliermarsh](https://togithub.com/charliermarsh) in [astral-sh/ruff#4001 - Avoid short-circuiting when detecting RET rules by [@charliermarsh](https://togithub.com/charliermarsh) in [astral-sh/ruff#4002 - Set non-empty range for indentation diagnostics by [@MichaReiser](https://togithub.com/MichaReiser) in [astral-sh/ruff#4005 - Ignore relative imports in `banned-api` rules by [@charliermarsh](https://togithub.com/charliermarsh) in [astral-sh/ruff#4024 - Support relative imports in `banned-api` enforcement by [@charliermarsh](https://togithub.com/charliermarsh) in [astral-sh/ruff#4025 - Treat non-future function annotations as required-at-runtime by [@charliermarsh](https://togithub.com/charliermarsh) in [astral-sh/ruff#4028 - Ignore certain flake8-pyi errors within function bodies by [@charliermarsh](https://togithub.com/charliermarsh) in [astral-sh/ruff#4029 #### New Contributors - [@tjkuson](https://togithub.com/tjkuson) made their first contribution in [astral-sh/ruff#3886 - [@mosauter](https://togithub.com/mosauter) made their first contribution in [astral-sh/ruff#3877 - [@stancld](https://togithub.com/stancld) made their first contribution in [astral-sh/ruff#3926 - [@USER-5](https://togithub.com/USER-5) made their first contribution in [astral-sh/ruff#3922 - [@robyoung](https://togithub.com/robyoung) made their first contribution in [astral-sh/ruff#3924 - [@hackedd](https://togithub.com/hackedd) made their first contribution in [astral-sh/ruff#3900 - [@justinchuby](https://togithub.com/justinchuby) made their first contribution in [astral-sh/ruff#3982 - [@mirecl](https://togithub.com/mirecl) made their first contribution in [astral-sh/ruff#4008 - [@Xemnas0](https://togithub.com/Xemnas0) made their first contribution in [astral-sh/ruff#4026 **Full Changelog**: astral-sh/ruff@v0.0.261...v0.0.262 </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Enabled. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] If you want to rebase/retry this PR, check this box --- This PR has been generated by [Mend Renovate](https://www.mend.io/free-developer-tools/renovate/). View repository job log [here](https://app.renovatebot.com/dashboard#github/allenporter/flux-local).  Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

[![Mend Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com) This PR contains the following updates: | Package | Change | Age | Adoption | Passing | Confidence | |---|---|---|---|---|---| | [ruff](https://togithub.com/charliermarsh/ruff) | `==0.0.261` -> `==0.0.262` | [![age](https://badges.renovateapi.com/packages/pypi/ruff/0.0.262/age-slim)](https://docs.renovatebot.com/merge-confidence/) | [![adoption](https://badges.renovateapi.com/packages/pypi/ruff/0.0.262/adoption-slim)](https://docs.renovatebot.com/merge-confidence/) | [![passing](https://badges.renovateapi.com/packages/pypi/ruff/0.0.262/compatibility-slim/0.0.261)](https://docs.renovatebot.com/merge-confidence/) | [![confidence](https://badges.renovateapi.com/packages/pypi/ruff/0.0.262/confidence-slim/0.0.261)](https://docs.renovatebot.com/merge-confidence/) | --- ### Release Notes <details> <summary>charliermarsh/ruff</summary> ### [`v0.0.262`](https://togithub.com/charliermarsh/ruff/releases/tag/v0.0.262) [Compare Source](https://togithub.com/charliermarsh/ruff/compare/v0.0.261...v0.0.262)  #### What's Changed ##### Configuration - Allow users to extend the set of included files via `include` by [@charliermarsh](https://togithub.com/charliermarsh) in [astral-sh/ruff#3914 - Implement isort custom sections and ordering ([#2419](https://togithub.com/charliermarsh/ruff/issues/2419)) by [@hackedd](https://togithub.com/hackedd) in [astral-sh/ruff#3900 ##### Rules - \[`flake8-simplify`] Add autofix for `contextlib.suppress` (`SIM105`) by [@leiserfg](https://togithub.com/leiserfg) in [astral-sh/ruff#3915 - \[`flake8-bandit`] Ignore assert errors (S101) in `TYPE_CHECKING` blocks by [@charliermarsh](https://togithub.com/charliermarsh) in [astral-sh/ruff#3960 - \[`flake8-comprehensions`] Implement `unnecessary-literal-within-dict-call` (`C418`) by [@charliermarsh](https://togithub.com/charliermarsh) in [astral-sh/ruff#3969 - \[`ruff`] Add checks for mutable defaults `dataclass`es by [@mosauter](https://togithub.com/mosauter) in [astral-sh/ruff#3877 - \[`flake8-import-conventions`] Add a rule for `BannedImportAlias` by [@stancld](https://togithub.com/stancld) in [astral-sh/ruff#3926 - \[`flake8-pyi`] Implement duplicate types in unions (`PYI016`) by [@USER-5](https://togithub.com/USER-5) in [astral-sh/ruff#3922 - \[`flake8-bandit`] Implement flake8-bandit shell injection rules by [@robyoung](https://togithub.com/robyoung) in [astral-sh/ruff#3924 - \[`flake8-comprehensions`] Redirect `PIE802` to `C419` by [@charliermarsh](https://togithub.com/charliermarsh) in [astral-sh/ruff#3971 ##### Bug Fixes - Fix unicode handling in PLE2515 by [@konstin](https://togithub.com/konstin) in [astral-sh/ruff#3898 - Avoid adding required imports to stub files by [@charliermarsh](https://togithub.com/charliermarsh) in [astral-sh/ruff#3940 - Add 'or if cond' to `E712` message by [@charliermarsh](https://togithub.com/charliermarsh) in [astral-sh/ruff#3962 - Ignore argument assignments when enforcing `RET504` by [@charliermarsh](https://togithub.com/charliermarsh) in [astral-sh/ruff#4004 - Fix (doc-)line-too-long start location by [@MichaReiser](https://togithub.com/MichaReiser) in [astral-sh/ruff#4006 - Ignore stub file assignments to value-requiring targets by [@charliermarsh](https://togithub.com/charliermarsh) in [astral-sh/ruff#4030 - Allow legacy C and T selectors in JSON schema by [@charliermarsh](https://togithub.com/charliermarsh) in [astral-sh/ruff#3889 - Ignore `PLW2901` when using typing cast by [@dhruvmanila](https://togithub.com/dhruvmanila) in [astral-sh/ruff#3891 - Visit comprehension to detect group name usage/overrides by [@dhruvmanila](https://togithub.com/dhruvmanila) in [astral-sh/ruff#3887 - Ensure that tab characters aren't in multi-line strings before throwing a violation by [@evanrittenhouse](https://togithub.com/evanrittenhouse) in [astral-sh/ruff#3837 - Avoid N802 violations for `@override` methods by [@charliermarsh](https://togithub.com/charliermarsh) in [astral-sh/ruff#3912 - Check for arguments in inner/outer call for `C414` by [@dhruvmanila](https://togithub.com/dhruvmanila) in [astral-sh/ruff#3916 - Do not skip analysis if `*args` present for `F523` by [@dhruvmanila](https://togithub.com/dhruvmanila) in [astral-sh/ruff#3923 - Extend SIM105 to match also 'Ellipsis only' bodies in exception handlers by [@leiserfg](https://togithub.com/leiserfg) in [astral-sh/ruff#3925 - Support `pyright: ignore` comments by [@charliermarsh](https://togithub.com/charliermarsh) in [astral-sh/ruff#3941 - Tidy up some `pygrep-hooks` rules by [@charliermarsh](https://togithub.com/charliermarsh) in [astral-sh/ruff#3942 - Use identifier range for pytest rules by [@charliermarsh](https://togithub.com/charliermarsh) in [astral-sh/ruff#3948 - Allow `typing_extensions.TypeVar` assignments in `.pyi` files by [@charliermarsh](https://togithub.com/charliermarsh) in [astral-sh/ruff#3951 - Raise percent-format upgrade rule (`UP031`) for hanging modulos by [@charliermarsh](https://togithub.com/charliermarsh) in [astral-sh/ruff#3953 - Check for parenthesis in implicit str concat in `PT006` by [@dhruvmanila](https://togithub.com/dhruvmanila) in [astral-sh/ruff#3955 - Do not consider nested comment as part of code by [@dhruvmanila](https://togithub.com/dhruvmanila) in [astral-sh/ruff#3984 - Preserve type annotations when fixing `E731` by [@dhruvmanila](https://togithub.com/dhruvmanila) in [astral-sh/ruff#3983 - Remove autofix behavior for uncapitalized-environment-variables (`SIM112`) by [@charliermarsh](https://togithub.com/charliermarsh) in [astral-sh/ruff#3988 - Respect typing-modules when evaluating no-return functions by [@charliermarsh](https://togithub.com/charliermarsh) in [astral-sh/ruff#4001 - Avoid short-circuiting when detecting RET rules by [@charliermarsh](https://togithub.com/charliermarsh) in [astral-sh/ruff#4002 - Set non-empty range for indentation diagnostics by [@MichaReiser](https://togithub.com/MichaReiser) in [astral-sh/ruff#4005 - Ignore relative imports in `banned-api` rules by [@charliermarsh](https://togithub.com/charliermarsh) in [astral-sh/ruff#4024 - Support relative imports in `banned-api` enforcement by [@charliermarsh](https://togithub.com/charliermarsh) in [astral-sh/ruff#4025 - Treat non-future function annotations as required-at-runtime by [@charliermarsh](https://togithub.com/charliermarsh) in [astral-sh/ruff#4028 - Ignore certain flake8-pyi errors within function bodies by [@charliermarsh](https://togithub.com/charliermarsh) in [astral-sh/ruff#4029 #### New Contributors - [@tjkuson](https://togithub.com/tjkuson) made their first contribution in [astral-sh/ruff#3886 - [@mosauter](https://togithub.com/mosauter) made their first contribution in [astral-sh/ruff#3877 - [@stancld](https://togithub.com/stancld) made their first contribution in [astral-sh/ruff#3926 - [@USER-5](https://togithub.com/USER-5) made their first contribution in [astral-sh/ruff#3922 - [@robyoung](https://togithub.com/robyoung) made their first contribution in [astral-sh/ruff#3924 - [@hackedd](https://togithub.com/hackedd) made their first contribution in [astral-sh/ruff#3900 - [@justinchuby](https://togithub.com/justinchuby) made their first contribution in [astral-sh/ruff#3982 - [@mirecl](https://togithub.com/mirecl) made their first contribution in [astral-sh/ruff#4008 - [@Xemnas0](https://togithub.com/Xemnas0) made their first contribution in [astral-sh/ruff#4026 **Full Changelog**: astral-sh/ruff@v0.0.261...v0.0.262 </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Enabled. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] If you want to rebase/retry this PR, check this box --- This PR has been generated by [Mend Renovate](https://www.mend.io/free-developer-tools/renovate/). View repository job log [here](https://app.renovatebot.com/dashboard#github/allenporter/pyrainbird).  Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

USER-5 added 5 commits April 9, 2023 08:48

PYI016 Simple case implementation

05282d6

Merge branch 'charliermarsh:main' into pyi016-duplicates-in-union

6e31ec7

Update PYI016 for all node types, chained unions

0cccb24

Change match to 'if let'

69a45bd

Fix invocations typo

9d22b5a

USER-5 commented Apr 9, 2023

View reviewed changes

USER-5 added 5 commits April 10, 2023 16:23

Only check type definitions

df965dc

Use more efficient algorithm

177578a

Remove redundant else statements

336a250

Remove test.py

03e7a97

Use set-based, recursive tree traversal

64aa831

Fix comment example

b11af21

bluetech reviewed Apr 10, 2023

View reviewed changes

USER-5 and others added 2 commits April 11, 2023 07:36

Use mutable reference instead of ownership

8f97c76

Use inner helper; use unparse_expr to get clear names

22fe136

charliermarsh reviewed Apr 11, 2023

View reviewed changes

crates/ruff/src/rules/flake8_pyi/rules/duplicate_union_member.rs Show resolved Hide resolved

charliermarsh reviewed Apr 11, 2023

View reviewed changes

Fix erroneous parentheses deletion

a71bb48

USER-5 commented Apr 11, 2023

View reviewed changes

Fix typo in fixture

37825d9

Minor tweaks

ef4d9ce

charliermarsh changed the title ~~PYI016 - Duplicates in union~~ [flake8-pyi] Implement duplicate types in unions (PYI016) Apr 12, 2023

charliermarsh enabled auto-merge (squash) April 12, 2023 03:48

charliermarsh disabled auto-merge April 12, 2023 03:49

charliermarsh added 3 commits April 11, 2023 23:50

Merge branch 'main' into pyi016-duplicates-in-union

5a5f633

Handle nested unions

d4a67a1

Merge branch 'main' into pyi016-duplicates-in-union

c6bd903

charliermarsh enabled auto-merge (squash) April 12, 2023 04:00

charliermarsh reviewed Apr 12, 2023

View reviewed changes

charliermarsh merged commit 8608414 into astral-sh:main Apr 12, 2023

p-linnane mentioned this pull request Apr 20, 2023

ruff 0.0.262 Homebrew/homebrew-core#128839

Merged

zanieb mentioned this pull request Jul 6, 2023

Implement PYI030: Unnecessary literal union #5570

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[`flake8-pyi`] Implement duplicate types in unions (`PYI016`) #3922

[`flake8-pyi`] Implement duplicate types in unions (`PYI016`) #3922

USER-5 commented Apr 9, 2023

github-actions bot commented Apr 9, 2023 •

edited

Loading

USER-5 Apr 9, 2023

bluetech Apr 9, 2023

charliermarsh Apr 9, 2023

charliermarsh Apr 9, 2023

USER-5 Apr 10, 2023 •

edited

Loading

bluetech commented Apr 9, 2023

charliermarsh commented Apr 9, 2023

USER-5 commented Apr 9, 2023

bluetech commented Apr 10, 2023

USER-5 commented Apr 10, 2023 •

edited

Loading

bluetech Apr 10, 2023

USER-5 Apr 10, 2023

charliermarsh Apr 11, 2023

charliermarsh Apr 11, 2023

charliermarsh Apr 11, 2023 •

edited

Loading

USER-5 Apr 11, 2023 •

edited

Loading

USER-5 Apr 11, 2023 •

edited

Loading

charliermarsh Apr 12, 2023

charliermarsh Apr 12, 2023

charliermarsh commented Apr 11, 2023

charliermarsh Apr 11, 2023

USER-5 Apr 11, 2023

charliermarsh Apr 12, 2023

charliermarsh Apr 12, 2023

[flake8-pyi] Implement duplicate types in unions (PYI016) #3922

[flake8-pyi] Implement duplicate types in unions (PYI016) #3922

Conversation

USER-5 commented Apr 9, 2023

github-actions bot commented Apr 9, 2023 • edited Loading

PR Check Results

Ecosystem

Benchmark

Linux

Windows

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

USER-5 Apr 10, 2023 • edited Loading

Choose a reason for hiding this comment

bluetech commented Apr 9, 2023

charliermarsh commented Apr 9, 2023

USER-5 commented Apr 9, 2023

bluetech commented Apr 10, 2023

USER-5 commented Apr 10, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

charliermarsh Apr 11, 2023 • edited Loading

Choose a reason for hiding this comment

USER-5 Apr 11, 2023 • edited Loading

Choose a reason for hiding this comment

USER-5 Apr 11, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

charliermarsh commented Apr 11, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

[`flake8-pyi`] Implement duplicate types in unions (`PYI016`) #3922

[`flake8-pyi`] Implement duplicate types in unions (`PYI016`) #3922

github-actions bot commented Apr 9, 2023 •

edited

Loading

USER-5 Apr 10, 2023 •

edited

Loading

USER-5 commented Apr 10, 2023 •

edited

Loading

charliermarsh Apr 11, 2023 •

edited

Loading

USER-5 Apr 11, 2023 •

edited

Loading

USER-5 Apr 11, 2023 •

edited

Loading