Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

perf(logical-lines): Various small perf improvements #4022

Merged
merged 2 commits into from
Apr 26, 2023

Conversation

MichaReiser
Copy link
Member

@MichaReiser MichaReiser commented Apr 19, 2023

This PR introduces a few perf improvements to the pycodestyle rules:

  • Introduce a new LogicalLinesContext to which rules add their diagnostics directly. This reduces the need to copy Diagnostics between different Vecs.
  • Avoid using Option<T> where we can instead use TextSize::default or Tok::EndOfFile. This reduces the required branching.
  • extraneous_whitespace: Move the DiagnosticKind::from closer to where we create the Diagnostic to avoid unnecessary allocations and message formatting in case the code is valid
  • LogicalLines: Use a Slice instead of a struct of vecs. Both implementations have roughly the same performance but using a Slice requires significantly less code.
  • LineIndex: Replace TextSize::try_from with a single assertion on the length of the string. We call this function so often that it shows up in traces. Using TextSize::new(x as u32) removes that slight overhead.

@github-actions
Copy link
Contributor

github-actions bot commented Apr 19, 2023

PR Check Results

Ecosystem

✅ ecosystem check detected no changes.

Benchmark

Linux

group                                      main                                   pr
-----                                      ----                                   --
linter/all-rules/large/dataset.py          1.01     14.9±0.15ms     2.7 MB/sec    1.00     14.7±0.05ms     2.8 MB/sec
linter/all-rules/numpy/ctypeslib.py        1.00      3.6±0.01ms     4.6 MB/sec    1.00      3.6±0.13ms     4.6 MB/sec
linter/all-rules/numpy/globals.py          1.01    381.1±5.66µs     7.7 MB/sec    1.00    378.3±1.71µs     7.8 MB/sec
linter/all-rules/pydantic/types.py         1.00      6.2±0.02ms     4.1 MB/sec    1.00      6.2±0.01ms     4.1 MB/sec
linter/default-rules/large/dataset.py      1.01      7.6±0.11ms     5.3 MB/sec    1.00      7.6±0.01ms     5.4 MB/sec
linter/default-rules/numpy/ctypeslib.py    1.00   1596.4±2.47µs    10.4 MB/sec    1.00   1594.5±5.73µs    10.4 MB/sec
linter/default-rules/numpy/globals.py      1.01    175.7±0.34µs    16.8 MB/sec    1.00    173.7±0.69µs    17.0 MB/sec
linter/default-rules/pydantic/types.py     1.01      3.4±0.01ms     7.5 MB/sec    1.00      3.4±0.01ms     7.5 MB/sec
parser/large/dataset.py                    1.00      5.9±0.01ms     6.9 MB/sec    1.00      5.9±0.02ms     6.9 MB/sec
parser/numpy/ctypeslib.py                  1.00   1147.6±1.69µs    14.5 MB/sec    1.00   1146.9±1.60µs    14.5 MB/sec
parser/numpy/globals.py                    1.00    116.6±0.28µs    25.3 MB/sec    1.00    116.4±0.69µs    25.3 MB/sec
parser/pydantic/types.py                   1.00      2.5±0.00ms    10.2 MB/sec    1.00      2.5±0.06ms    10.2 MB/sec

Windows

group                                      main                                   pr
-----                                      ----                                   --
linter/all-rules/large/dataset.py          1.00     20.5±0.48ms  2028.4 KB/sec    1.01     20.8±0.60ms  2002.6 KB/sec
linter/all-rules/numpy/ctypeslib.py        1.01      5.3±0.13ms     3.1 MB/sec    1.00      5.3±0.16ms     3.2 MB/sec
linter/all-rules/numpy/globals.py          1.00   626.3±15.49µs     4.7 MB/sec    1.00   625.7±16.55µs     4.7 MB/sec
linter/all-rules/pydantic/types.py         1.00      8.8±0.30ms     2.9 MB/sec    1.01      8.8±0.22ms     2.9 MB/sec
linter/default-rules/large/dataset.py      1.00     10.3±0.19ms     3.9 MB/sec    1.03     10.7±0.26ms     3.8 MB/sec
linter/default-rules/numpy/ctypeslib.py    1.01      2.3±0.13ms     7.4 MB/sec    1.00      2.2±0.06ms     7.5 MB/sec
linter/default-rules/numpy/globals.py      1.00   266.5±12.18µs    11.1 MB/sec    1.03   275.1±12.51µs    10.7 MB/sec
linter/default-rules/pydantic/types.py     1.00      4.7±0.09ms     5.4 MB/sec    1.02      4.8±0.12ms     5.3 MB/sec
parser/large/dataset.py                    1.00      8.5±0.47ms     4.8 MB/sec    1.01      8.6±0.14ms     4.8 MB/sec
parser/numpy/ctypeslib.py                  1.00  1599.9±39.65µs    10.4 MB/sec    1.04  1659.6±29.99µs    10.0 MB/sec
parser/numpy/globals.py                    1.00   165.2±11.41µs    17.9 MB/sec    1.01    166.4±6.66µs    17.7 MB/sec
parser/pydantic/types.py                   1.00      3.6±0.08ms     7.0 MB/sec    1.01      3.7±0.07ms     7.0 MB/sec

@MichaReiser MichaReiser force-pushed the byte-offset-parser branch 3 times, most recently from 90fc963 to 5995306 Compare April 26, 2023 17:22
Base automatically changed from byte-offset-parser to main April 26, 2023 18:11
@MichaReiser MichaReiser marked this pull request as ready for review April 26, 2023 19:09
@MichaReiser MichaReiser merged commit f3e6ddd into main Apr 26, 2023
15 checks passed
@MichaReiser MichaReiser deleted the perf-logical-lines branch April 26, 2023 19:10
renovate bot added a commit to ixm-one/pytest-cmake-presets that referenced this pull request May 2, 2023
[![Mend
Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com)

This PR contains the following updates:

| Package | Change | Age | Adoption | Passing | Confidence |
|---|---|---|---|---|---|
| [ruff](https://togithub.com/charliermarsh/ruff) | `^0.0.263` ->
`^0.0.264` |
[![age](https://badges.renovateapi.com/packages/pypi/ruff/0.0.264/age-slim)](https://docs.renovatebot.com/merge-confidence/)
|
[![adoption](https://badges.renovateapi.com/packages/pypi/ruff/0.0.264/adoption-slim)](https://docs.renovatebot.com/merge-confidence/)
|
[![passing](https://badges.renovateapi.com/packages/pypi/ruff/0.0.264/compatibility-slim/0.0.263)](https://docs.renovatebot.com/merge-confidence/)
|
[![confidence](https://badges.renovateapi.com/packages/pypi/ruff/0.0.264/confidence-slim/0.0.263)](https://docs.renovatebot.com/merge-confidence/)
|

---

### Release Notes

<details>
<summary>charliermarsh/ruff</summary>

###
[`v0.0.264`](https://togithub.com/charliermarsh/ruff/releases/tag/v0.0.264)

[Compare
Source](https://togithub.com/charliermarsh/ruff/compare/v0.0.263...v0.0.264)

<!-- Release notes generated using configuration in .github/release.yml
at 8cb76f85eba1c970a8c800348fd1e0c874621a57 -->

#### What's Changed

##### Rules

- Autofix `EM101`, `EM102`, `EM103` if possible by
[@&#8203;dhruvmanila](https://togithub.com/dhruvmanila) in
[astral-sh/ruff#4123
- Add bugbear immutable functions as allowed in dataclasses by
[@&#8203;mosauter](https://togithub.com/mosauter) in
[astral-sh/ruff#4122

##### Settings

- Add support for providing command-line arguments via `argfile` by
[@&#8203;charliermarsh](https://togithub.com/charliermarsh) in
[astral-sh/ruff#4087

##### Bug Fixes

- Make D410/D411 autofixes mutually exclusive by
[@&#8203;evanrittenhouse](https://togithub.com/evanrittenhouse) in
[astral-sh/ruff#4110
- Remove `pyright` comment prefix from PYI033 checks by
[@&#8203;evanrittenhouse](https://togithub.com/evanrittenhouse) in
[astral-sh/ruff#4152
- Fix F811 false positive with match by
[@&#8203;JonathanPlasse](https://togithub.com/JonathanPlasse) in
[astral-sh/ruff#4161
- Fix `E713` and `E714` false positives for multiple comparisons by
[@&#8203;JonathanPlasse](https://togithub.com/JonathanPlasse) in
[astral-sh/ruff#4083
- Fix B023 shadowed variables in nested functions by
[@&#8203;MichaReiser](https://togithub.com/MichaReiser) in
[astral-sh/ruff#4111
- Preserve star-handling special-casing for force-single-line by
[@&#8203;charliermarsh](https://togithub.com/charliermarsh) in
[astral-sh/ruff#4129
- Respect parent-scoping rules for `NamedExpr` assignments by
[@&#8203;charliermarsh](https://togithub.com/charliermarsh) in
[astral-sh/ruff#4145
- Fix UP032 auto-fix by
[@&#8203;JonathanPlasse](https://togithub.com/JonathanPlasse) in
[astral-sh/ruff#4165
- Allow boolean parameters for `pytest.param` by
[@&#8203;charliermarsh](https://togithub.com/charliermarsh) in
[astral-sh/ruff#4176

##### Internal

- Replace row/column based `Location` with byte-offsets. by
[@&#8203;MichaReiser](https://togithub.com/MichaReiser) in
[astral-sh/ruff#3931
- perf(logical-lines): Various small perf improvements by
[@&#8203;MichaReiser](https://togithub.com/MichaReiser) in
[astral-sh/ruff#4022
- Use `memchr` to speedup newline search on x86 by
[@&#8203;MichaReiser](https://togithub.com/MichaReiser) in
[astral-sh/ruff#3985
- Remove `ScopeStack` in favor of child-parent `ScopeId` pointers by
[@&#8203;charliermarsh](https://togithub.com/charliermarsh) in
[astral-sh/ruff#4138

**Full Changelog**:
astral-sh/ruff@v0.0.263...v0.0.264

</details>

---

### Configuration

📅 **Schedule**: Branch creation - At any time (no schedule defined),
Automerge - At any time (no schedule defined).

🚦 **Automerge**: Enabled.

♻ **Rebasing**: Whenever PR is behind base branch, or you tick the
rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.

---

- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box

---

This PR has been generated by [Mend
Renovate](https://www.mend.io/free-developer-tools/renovate/). View
repository job log
[here](https://app.renovatebot.com/dashboard#github/ixm-one/pytest-cmake-presets).

<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNS42Ni4zIiwidXBkYXRlZEluVmVyIjoiMzUuNjYuMyIsInRhcmdldEJyYW5jaCI6Im1haW4ifQ==-->

Signed-off-by: Renovate Bot <bot@renovateapp.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
renovate bot added a commit to allenporter/flux-local that referenced this pull request May 3, 2023
[![Mend
Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com)

This PR contains the following updates:

| Package | Change | Age | Adoption | Passing | Confidence |
|---|---|---|---|---|---|
| [ruff](https://togithub.com/charliermarsh/ruff) | `==0.0.263` ->
`==0.0.264` |
[![age](https://badges.renovateapi.com/packages/pypi/ruff/0.0.264/age-slim)](https://docs.renovatebot.com/merge-confidence/)
|
[![adoption](https://badges.renovateapi.com/packages/pypi/ruff/0.0.264/adoption-slim)](https://docs.renovatebot.com/merge-confidence/)
|
[![passing](https://badges.renovateapi.com/packages/pypi/ruff/0.0.264/compatibility-slim/0.0.263)](https://docs.renovatebot.com/merge-confidence/)
|
[![confidence](https://badges.renovateapi.com/packages/pypi/ruff/0.0.264/confidence-slim/0.0.263)](https://docs.renovatebot.com/merge-confidence/)
|

---

### Release Notes

<details>
<summary>charliermarsh/ruff</summary>

###
[`v0.0.264`](https://togithub.com/charliermarsh/ruff/releases/tag/v0.0.264)

[Compare
Source](https://togithub.com/charliermarsh/ruff/compare/v0.0.263...v0.0.264)

<!-- Release notes generated using configuration in .github/release.yml
at 8cb76f85eba1c970a8c800348fd1e0c874621a57 -->

#### What's Changed

##### Rules

- Autofix `EM101`, `EM102`, `EM103` if possible by
[@&#8203;dhruvmanila](https://togithub.com/dhruvmanila) in
[astral-sh/ruff#4123
- Add bugbear immutable functions as allowed in dataclasses by
[@&#8203;mosauter](https://togithub.com/mosauter) in
[astral-sh/ruff#4122

##### Settings

- Add support for providing command-line arguments via `argfile` by
[@&#8203;charliermarsh](https://togithub.com/charliermarsh) in
[astral-sh/ruff#4087

##### Bug Fixes

- Make D410/D411 autofixes mutually exclusive by
[@&#8203;evanrittenhouse](https://togithub.com/evanrittenhouse) in
[astral-sh/ruff#4110
- Remove `pyright` comment prefix from PYI033 checks by
[@&#8203;evanrittenhouse](https://togithub.com/evanrittenhouse) in
[astral-sh/ruff#4152
- Fix F811 false positive with match by
[@&#8203;JonathanPlasse](https://togithub.com/JonathanPlasse) in
[astral-sh/ruff#4161
- Fix `E713` and `E714` false positives for multiple comparisons by
[@&#8203;JonathanPlasse](https://togithub.com/JonathanPlasse) in
[astral-sh/ruff#4083
- Fix B023 shadowed variables in nested functions by
[@&#8203;MichaReiser](https://togithub.com/MichaReiser) in
[astral-sh/ruff#4111
- Preserve star-handling special-casing for force-single-line by
[@&#8203;charliermarsh](https://togithub.com/charliermarsh) in
[astral-sh/ruff#4129
- Respect parent-scoping rules for `NamedExpr` assignments by
[@&#8203;charliermarsh](https://togithub.com/charliermarsh) in
[astral-sh/ruff#4145
- Fix UP032 auto-fix by
[@&#8203;JonathanPlasse](https://togithub.com/JonathanPlasse) in
[astral-sh/ruff#4165
- Allow boolean parameters for `pytest.param` by
[@&#8203;charliermarsh](https://togithub.com/charliermarsh) in
[astral-sh/ruff#4176

##### Internal

- Replace row/column based `Location` with byte-offsets. by
[@&#8203;MichaReiser](https://togithub.com/MichaReiser) in
[astral-sh/ruff#3931
- perf(logical-lines): Various small perf improvements by
[@&#8203;MichaReiser](https://togithub.com/MichaReiser) in
[astral-sh/ruff#4022
- Use `memchr` to speedup newline search on x86 by
[@&#8203;MichaReiser](https://togithub.com/MichaReiser) in
[astral-sh/ruff#3985
- Remove `ScopeStack` in favor of child-parent `ScopeId` pointers by
[@&#8203;charliermarsh](https://togithub.com/charliermarsh) in
[astral-sh/ruff#4138

**Full Changelog**:
astral-sh/ruff@v0.0.263...v0.0.264

</details>

---

### Configuration

📅 **Schedule**: Branch creation - At any time (no schedule defined),
Automerge - At any time (no schedule defined).

🚦 **Automerge**: Enabled.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.

---

- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box

---

This PR has been generated by [Mend
Renovate](https://www.mend.io/free-developer-tools/renovate/). View
repository job log
[here](https://app.renovatebot.com/dashboard#github/allenporter/flux-local).

<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNS42OS4zIiwidXBkYXRlZEluVmVyIjoiMzUuNjkuMyIsInRhcmdldEJyYW5jaCI6Im1haW4ifQ==-->

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
renovate bot added a commit to allenporter/pyrainbird that referenced this pull request May 3, 2023
[![Mend
Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com)

This PR contains the following updates:

| Package | Change | Age | Adoption | Passing | Confidence |
|---|---|---|---|---|---|
| [ruff](https://togithub.com/charliermarsh/ruff) | `==0.0.263` ->
`==0.0.264` |
[![age](https://badges.renovateapi.com/packages/pypi/ruff/0.0.264/age-slim)](https://docs.renovatebot.com/merge-confidence/)
|
[![adoption](https://badges.renovateapi.com/packages/pypi/ruff/0.0.264/adoption-slim)](https://docs.renovatebot.com/merge-confidence/)
|
[![passing](https://badges.renovateapi.com/packages/pypi/ruff/0.0.264/compatibility-slim/0.0.263)](https://docs.renovatebot.com/merge-confidence/)
|
[![confidence](https://badges.renovateapi.com/packages/pypi/ruff/0.0.264/confidence-slim/0.0.263)](https://docs.renovatebot.com/merge-confidence/)
|

---

### Release Notes

<details>
<summary>charliermarsh/ruff</summary>

###
[`v0.0.264`](https://togithub.com/charliermarsh/ruff/releases/tag/v0.0.264)

[Compare
Source](https://togithub.com/charliermarsh/ruff/compare/v0.0.263...v0.0.264)

<!-- Release notes generated using configuration in .github/release.yml
at 8cb76f85eba1c970a8c800348fd1e0c874621a57 -->

#### What's Changed

##### Rules

- Autofix `EM101`, `EM102`, `EM103` if possible by
[@&#8203;dhruvmanila](https://togithub.com/dhruvmanila) in
[astral-sh/ruff#4123
- Add bugbear immutable functions as allowed in dataclasses by
[@&#8203;mosauter](https://togithub.com/mosauter) in
[astral-sh/ruff#4122

##### Settings

- Add support for providing command-line arguments via `argfile` by
[@&#8203;charliermarsh](https://togithub.com/charliermarsh) in
[astral-sh/ruff#4087

##### Bug Fixes

- Make D410/D411 autofixes mutually exclusive by
[@&#8203;evanrittenhouse](https://togithub.com/evanrittenhouse) in
[astral-sh/ruff#4110
- Remove `pyright` comment prefix from PYI033 checks by
[@&#8203;evanrittenhouse](https://togithub.com/evanrittenhouse) in
[astral-sh/ruff#4152
- Fix F811 false positive with match by
[@&#8203;JonathanPlasse](https://togithub.com/JonathanPlasse) in
[astral-sh/ruff#4161
- Fix `E713` and `E714` false positives for multiple comparisons by
[@&#8203;JonathanPlasse](https://togithub.com/JonathanPlasse) in
[astral-sh/ruff#4083
- Fix B023 shadowed variables in nested functions by
[@&#8203;MichaReiser](https://togithub.com/MichaReiser) in
[astral-sh/ruff#4111
- Preserve star-handling special-casing for force-single-line by
[@&#8203;charliermarsh](https://togithub.com/charliermarsh) in
[astral-sh/ruff#4129
- Respect parent-scoping rules for `NamedExpr` assignments by
[@&#8203;charliermarsh](https://togithub.com/charliermarsh) in
[astral-sh/ruff#4145
- Fix UP032 auto-fix by
[@&#8203;JonathanPlasse](https://togithub.com/JonathanPlasse) in
[astral-sh/ruff#4165
- Allow boolean parameters for `pytest.param` by
[@&#8203;charliermarsh](https://togithub.com/charliermarsh) in
[astral-sh/ruff#4176

##### Internal

- Replace row/column based `Location` with byte-offsets. by
[@&#8203;MichaReiser](https://togithub.com/MichaReiser) in
[astral-sh/ruff#3931
- perf(logical-lines): Various small perf improvements by
[@&#8203;MichaReiser](https://togithub.com/MichaReiser) in
[astral-sh/ruff#4022
- Use `memchr` to speedup newline search on x86 by
[@&#8203;MichaReiser](https://togithub.com/MichaReiser) in
[astral-sh/ruff#3985
- Remove `ScopeStack` in favor of child-parent `ScopeId` pointers by
[@&#8203;charliermarsh](https://togithub.com/charliermarsh) in
[astral-sh/ruff#4138

**Full Changelog**:
astral-sh/ruff@v0.0.263...v0.0.264

</details>

---

### Configuration

📅 **Schedule**: Branch creation - At any time (no schedule defined),
Automerge - At any time (no schedule defined).

🚦 **Automerge**: Enabled.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.

---

- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box

---

This PR has been generated by [Mend
Renovate](https://www.mend.io/free-developer-tools/renovate/). View
repository job log
[here](https://app.renovatebot.com/dashboard#github/allenporter/pyrainbird).

<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNS42OS4zIiwidXBkYXRlZEluVmVyIjoiMzUuNjkuMyIsInRhcmdldEJyYW5jaCI6Im1haW4ifQ==-->

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants