Respect .ipynb and .pyi sources when linting from stdin #6628
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
charliermarsh
commented
Aug 16, 2023
| message: format!( | ||
| "Expected a Jupyter Notebook (.{JUPYTER_NOTEBOOK_EXT} extension), \ | ||
| which must be internally stored as JSON, \ | ||
| but found a Python source file: {err}" |
There was a problem hiding this comment.
I decided to get rid of this. It's nice but kind of hard to preserve now that this method is agnostic to path-or-string (since it takes a reader).
charliermarsh
added
bug
PR Check ResultsEcosystem✅ ecosystem check detected no changes. BenchmarkLinuxWindows |
zanieb
approved these changes
Aug 16, 2023
MichaReiser
approved these changes
Aug 16, 2023
charliermarsh
force-pushed
the
charlie/jupyter
branch
from
ee92fff
to
7f35448
Compare
charliermarsh
added a commit
that referenced
this pull request
Aug 18, 2023
## Summary In working on #6628, I noticed that we clone the source code contents, potentially multiple times, prior to linting. The issue is that `SourceKind::Python` takes a `String`, so we first have to provide it with a `String`. In the stdin case, that means cloning. However, on top of this, we then have to clone `source_kind.contents()` because `SourceKind` gets mutated. So for stdin, we end up cloning twice. For non-stdin, we end up cloning once, but unnecessarily (since the _contents_ don't get mutated, only the kind). This PR removes the `String` from `source_kind`, instead requiring that we parse it out elsewhere. It reduces the number of clones down to 1 for Jupyter Notebooks, and zero otherwise.
renovate bot
added a commit
to allenporter/pyrainbird
that referenced
this pull request
Aug 19, 2023
[](https://renovatebot.com) This PR contains the following updates: | Package | Change | Age | Adoption | Passing | Confidence | |---|---|---|---|---|---| | [ruff](https://beta.ruff.rs/docs) ([source](https://togithub.com/astral-sh/ruff), [changelog](https://togithub.com/astral-sh/ruff/releases)) | `==0.0.284` -> `==0.0.285` | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | --- ### Release Notes <details> <summary>astral-sh/ruff (ruff)</summary> ### [`v0.0.285`](https://togithub.com/astral-sh/ruff/releases/tag/v0.0.285) [Compare Source](https://togithub.com/astral-sh/ruff/compare/v0.0.284...v0.0.285) #### What's Changed ##### New rules - \[`flake8-pytest-style`] Implement `pytest-unittest-raises-assertion` (`PT027`) by [@​harupy](https://togithub.com/harupy) in [astral-sh/ruff#6554 - \[`flake8-pytest-style`] Implement `pytest-duplicate-parametrize-test-cases` (`PT014`) by [@​harupy](https://togithub.com/harupy) in [astral-sh/ruff#6598 - \[`flake8-tidy-imports`] Implement `banned-module-level-imports` (`TID253`) by [@​durumu](https://togithub.com/durumu) in [astral-sh/ruff#6378 - \[`pylint`] Implement `bad-dunder-name` (`W3201`) (in the Ruff nursery) by [@​LaBatata101](https://togithub.com/LaBatata101) in [astral-sh/ruff#6486 - \[`pylint`] Implement `subprocess-run-check` (`W1510`) by [@​tjkuson](https://togithub.com/tjkuson) in [astral-sh/ruff#6487 - \[`ruff`] Implement `quadratic-list-summation` (`RUF017`) by [@​evanrittenhouse](https://togithub.com/evanrittenhouse) in [astral-sh/ruff#6489 ##### Rule changes - \[`flake8-bugbear`] Add autofix for `B006` by [@​qdegraaf](https://togithub.com/qdegraaf) in [astral-sh/ruff#6131 - \[`flake8-pyi`] Avoid applying `PYI055` to runtime-evaluated annotations by [@​charliermarsh](https://togithub.com/charliermarsh) in [astral-sh/ruff#6457 - \[`flake8-self`] Allow `os._exit` accesses in `SLF001` by [@​charliermarsh](https://togithub.com/charliermarsh) in [astral-sh/ruff#6490 - \[`perflint`] Ignore `PERF203` if `try` contains loop control flow statements by [@​evanrittenhouse](https://togithub.com/evanrittenhouse) in [astral-sh/ruff#6536 - \[`pylint`] Check for invalid format type specifiers in nested replacements for `PLE1300` by [@​zanieb](https://togithub.com/zanieb) in [astral-sh/ruff#6616 - \[`tryceratops`] Omit `NotImplementedError` from `TRY003` by [@​charliermarsh](https://togithub.com/charliermarsh) in [astral-sh/ruff#6568 ##### Settings - Respect `.ipynb` and `.pyi` sources when linting from stdin by [@​charliermarsh](https://togithub.com/charliermarsh) in [astral-sh/ruff#6628 - Support glob patterns for `raises_require_match_for` and `raises_require_match_for` by [@​harupy](https://togithub.com/harupy) in [astral-sh/ruff#6635 ##### Bug Fixes - Make `lambda-assignment` fix always-manual in class bodies by [@​charliermarsh](https://togithub.com/charliermarsh) in [astral-sh/ruff#6626 - Fix counting of message arguments when msg is provided as a keyword by [@​charliermarsh](https://togithub.com/charliermarsh) in [astral-sh/ruff#6456 - Add container types to `E721` by [@​charliermarsh](https://togithub.com/charliermarsh) in [astral-sh/ruff#6469 - Respect scoping rules when identifying builtins by [@​charliermarsh](https://togithub.com/charliermarsh) in [astral-sh/ruff#6468 - Respect tab width in line-length heuristic by [@​charliermarsh](https://togithub.com/charliermarsh) in [astral-sh/ruff#6491 - Respect dummy-variable-rgx for unused bound exceptions by [@​charliermarsh](https://togithub.com/charliermarsh) in [astral-sh/ruff#6492 - Fix detection of top-level imports with newlines in `E402` by [@​charliermarsh](https://togithub.com/charliermarsh) in [astral-sh/ruff#6526 - Allow if-expression with dual string arms in `invalid-envvar-value` by [@​charliermarsh](https://togithub.com/charliermarsh) in [astral-sh/ruff#6538 - Add deprecated unittest assertions to PT009 by [@​charliermarsh](https://togithub.com/charliermarsh) in [astral-sh/ruff#6572 - Avoid unused argument rules when functions call `locals()` by [@​charliermarsh](https://togithub.com/charliermarsh) in [astral-sh/ruff#6578 - Allow top-level `await` in Jupyter notebooks by [@​charliermarsh](https://togithub.com/charliermarsh) in [astral-sh/ruff#6607 - Don't detect `pandas#values` for stores, deletes, or class accesses by [@​charliermarsh](https://togithub.com/charliermarsh) in [astral-sh/ruff#6631 - Avoid removing parentheses in `E712` fix by [@​charliermarsh](https://togithub.com/charliermarsh) in [astral-sh/ruff#6575 - Skip whitespace between comments at start of file e.g. for `I002` by [@​durumu](https://togithub.com/durumu) in [astral-sh/ruff#6523 - Add support for nested replacements inside format specifications e.g. for `PLE1300` by [@​zanieb](https://togithub.com/zanieb) in [astral-sh/ruff#6616 ##### Playground - Shared playground links now use short URLs by [@​charliermarsh](https://togithub.com/charliermarsh) in [astral-sh/ruff#6383 - Fix possible JSON parse error on playground load by [@​charliermarsh](https://togithub.com/charliermarsh) in [astral-sh/ruff#6519 - Fix unreachable panic in playground by [@​MichaReiser](https://togithub.com/MichaReiser) in [astral-sh/ruff#6623 ##### Performance - Improve tokenizer performance for ASCII only identifiers by [@​MichaReiser](https://togithub.com/MichaReiser) in [astral-sh/ruff#6609 #### New Contributors - [@​magic-akari](https://togithub.com/magic-akari) made their first contribution in [astral-sh/ruff#6472 - [@​durumu](https://togithub.com/durumu) made their first contribution in [astral-sh/ruff#6378 - [@​jamesbraza](https://togithub.com/jamesbraza) made their first contribution in [astral-sh/ruff#6520 - [@​takumaw](https://togithub.com/takumaw) made their first contribution in [astral-sh/ruff#6533 - [@​noklam](https://togithub.com/noklam) made their first contribution in [astral-sh/ruff#6573 - [@​Teraskull](https://togithub.com/Teraskull) made their first contribution in [astral-sh/ruff#6605 **Full Changelog**: astral-sh/ruff@v0.0.284...v0.0.285 </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Enabled. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR has been generated by [Mend Renovate](https://www.mend.io/free-developer-tools/renovate/). View repository job log [here](https://developer.mend.io/github/allenporter/pyrainbird). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNi40My4yIiwidXBkYXRlZEluVmVyIjoiMzYuNDMuMiIsInRhcmdldEJyYW5jaCI6Im1haW4ifQ==--> Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
renovate bot
added a commit
to allenporter/flux-local
that referenced
this pull request
Aug 20, 2023
[](https://renovatebot.com) This PR contains the following updates: | Package | Change | Age | Adoption | Passing | Confidence | |---|---|---|---|---|---| | [ruff](https://beta.ruff.rs/docs) ([source](https://togithub.com/astral-sh/ruff), [changelog](https://togithub.com/astral-sh/ruff/releases)) | `==0.0.284` -> `==0.0.285` | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | --- ### Release Notes <details> <summary>astral-sh/ruff (ruff)</summary> ### [`v0.0.285`](https://togithub.com/astral-sh/ruff/releases/tag/v0.0.285) [Compare Source](https://togithub.com/astral-sh/ruff/compare/v0.0.284...v0.0.285) #### What's Changed ##### New rules - \[`flake8-pytest-style`] Implement `pytest-unittest-raises-assertion` (`PT027`) by [@​harupy](https://togithub.com/harupy) in [astral-sh/ruff#6554 - \[`flake8-pytest-style`] Implement `pytest-duplicate-parametrize-test-cases` (`PT014`) by [@​harupy](https://togithub.com/harupy) in [astral-sh/ruff#6598 - \[`flake8-tidy-imports`] Implement `banned-module-level-imports` (`TID253`) by [@​durumu](https://togithub.com/durumu) in [astral-sh/ruff#6378 - \[`pylint`] Implement `bad-dunder-name` (`W3201`) (in the Ruff nursery) by [@​LaBatata101](https://togithub.com/LaBatata101) in [astral-sh/ruff#6486 - \[`pylint`] Implement `subprocess-run-check` (`W1510`) by [@​tjkuson](https://togithub.com/tjkuson) in [astral-sh/ruff#6487 - \[`ruff`] Implement `quadratic-list-summation` (`RUF017`) by [@​evanrittenhouse](https://togithub.com/evanrittenhouse) in [astral-sh/ruff#6489 ##### Rule changes - \[`flake8-bugbear`] Add autofix for `B006` by [@​qdegraaf](https://togithub.com/qdegraaf) in [astral-sh/ruff#6131 - \[`flake8-pyi`] Avoid applying `PYI055` to runtime-evaluated annotations by [@​charliermarsh](https://togithub.com/charliermarsh) in [astral-sh/ruff#6457 - \[`flake8-self`] Allow `os._exit` accesses in `SLF001` by [@​charliermarsh](https://togithub.com/charliermarsh) in [astral-sh/ruff#6490 - \[`perflint`] Ignore `PERF203` if `try` contains loop control flow statements by [@​evanrittenhouse](https://togithub.com/evanrittenhouse) in [astral-sh/ruff#6536 - \[`pylint`] Check for invalid format type specifiers in nested replacements for `PLE1300` by [@​zanieb](https://togithub.com/zanieb) in [astral-sh/ruff#6616 - \[`tryceratops`] Omit `NotImplementedError` from `TRY003` by [@​charliermarsh](https://togithub.com/charliermarsh) in [astral-sh/ruff#6568 ##### Settings - Respect `.ipynb` and `.pyi` sources when linting from stdin by [@​charliermarsh](https://togithub.com/charliermarsh) in [astral-sh/ruff#6628 - Support glob patterns for `raises_require_match_for` and `raises_require_match_for` by [@​harupy](https://togithub.com/harupy) in [astral-sh/ruff#6635 ##### Bug Fixes - Make `lambda-assignment` fix always-manual in class bodies by [@​charliermarsh](https://togithub.com/charliermarsh) in [astral-sh/ruff#6626 - Fix counting of message arguments when msg is provided as a keyword by [@​charliermarsh](https://togithub.com/charliermarsh) in [astral-sh/ruff#6456 - Add container types to `E721` by [@​charliermarsh](https://togithub.com/charliermarsh) in [astral-sh/ruff#6469 - Respect scoping rules when identifying builtins by [@​charliermarsh](https://togithub.com/charliermarsh) in [astral-sh/ruff#6468 - Respect tab width in line-length heuristic by [@​charliermarsh](https://togithub.com/charliermarsh) in [astral-sh/ruff#6491 - Respect dummy-variable-rgx for unused bound exceptions by [@​charliermarsh](https://togithub.com/charliermarsh) in [astral-sh/ruff#6492 - Fix detection of top-level imports with newlines in `E402` by [@​charliermarsh](https://togithub.com/charliermarsh) in [astral-sh/ruff#6526 - Allow if-expression with dual string arms in `invalid-envvar-value` by [@​charliermarsh](https://togithub.com/charliermarsh) in [astral-sh/ruff#6538 - Add deprecated unittest assertions to PT009 by [@​charliermarsh](https://togithub.com/charliermarsh) in [astral-sh/ruff#6572 - Avoid unused argument rules when functions call `locals()` by [@​charliermarsh](https://togithub.com/charliermarsh) in [astral-sh/ruff#6578 - Allow top-level `await` in Jupyter notebooks by [@​charliermarsh](https://togithub.com/charliermarsh) in [astral-sh/ruff#6607 - Don't detect `pandas#values` for stores, deletes, or class accesses by [@​charliermarsh](https://togithub.com/charliermarsh) in [astral-sh/ruff#6631 - Avoid removing parentheses in `E712` fix by [@​charliermarsh](https://togithub.com/charliermarsh) in [astral-sh/ruff#6575 - Skip whitespace between comments at start of file e.g. for `I002` by [@​durumu](https://togithub.com/durumu) in [astral-sh/ruff#6523 - Add support for nested replacements inside format specifications e.g. for `PLE1300` by [@​zanieb](https://togithub.com/zanieb) in [astral-sh/ruff#6616 ##### Playground - Shared playground links now use short URLs by [@​charliermarsh](https://togithub.com/charliermarsh) in [astral-sh/ruff#6383 - Fix possible JSON parse error on playground load by [@​charliermarsh](https://togithub.com/charliermarsh) in [astral-sh/ruff#6519 - Fix unreachable panic in playground by [@​MichaReiser](https://togithub.com/MichaReiser) in [astral-sh/ruff#6623 ##### Performance - Improve tokenizer performance for ASCII only identifiers by [@​MichaReiser](https://togithub.com/MichaReiser) in [astral-sh/ruff#6609 #### New Contributors - [@​magic-akari](https://togithub.com/magic-akari) made their first contribution in [astral-sh/ruff#6472 - [@​durumu](https://togithub.com/durumu) made their first contribution in [astral-sh/ruff#6378 - [@​jamesbraza](https://togithub.com/jamesbraza) made their first contribution in [astral-sh/ruff#6520 - [@​takumaw](https://togithub.com/takumaw) made their first contribution in [astral-sh/ruff#6533 - [@​noklam](https://togithub.com/noklam) made their first contribution in [astral-sh/ruff#6573 - [@​Teraskull](https://togithub.com/Teraskull) made their first contribution in [astral-sh/ruff#6605 **Full Changelog**: astral-sh/ruff@v0.0.284...v0.0.285 </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Enabled. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR has been generated by [Mend Renovate](https://www.mend.io/free-developer-tools/renovate/). View repository job log [here](https://developer.mend.io/github/allenporter/flux-local). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNi40My4yIiwidXBkYXRlZEluVmVyIjoiMzYuNDMuMiIsInRhcmdldEJyYW5jaCI6Im1haW4ifQ==--> Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
](https://renovatebot.com){kind=link}
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
When running Ruff from stdin, we were always falling back to the default source type, even if the user specified a path (as is the case when running from the LSP). This PR wires up the source type inference, which means we now get the expected result when checking
.pyiand.ipynbfiles.Closes #6627.
Test Plan
Verified that
cat crates/ruff/resources/test/fixtures/jupyter/valid.ipynb | cargo run -p ruff_cli -- --force-exclude --no-cache --no-fix --isolated --select ALL --stdin-filename foo.ipynb -yielded the expected results (and differs from the errors you get if you omit the filename).Verified that
cat foo.pyi | cargo run -p ruff_cli -- --force-exclude --no-cache --no-fix --format json --isolated --select TCH --stdin-filename path/to/foo.pyi -yielded no errors.