Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fmt: off..on suppression comments #6477

Merged
merged 3 commits into from
Aug 14, 2023
Merged

fmt: off..on suppression comments #6477

merged 3 commits into from
Aug 14, 2023

Conversation

MichaReiser
Copy link
Member

@MichaReiser MichaReiser commented Aug 10, 2023

Summary

Implements support for fmt: off and fmt: on suppression comments on statement level.

This seems trivial but quickly gets complicated because you can have:

# fmt: off
# fmt: on
test

in which case we should not disable formatting (see test cases for more absurd usages of fmt: off and fmt: on combinations).

This PR does not address:

  • warn if a suppression comment is used incorrectly
  • Fixing indentation of suppressed ranges

Test Plan

I added many test files and believe that I covered all branches.

Small improvements in the similarity index:

Before

project similarity index
build 0.75623
cpython 0.75472
django 0.99802
transformers 0.99470
typeshed 0.74233
warehouse 0.99601
zulip 0.99727

After

project similarity index
build 0.75623
cpython 0.75472
django 0.99804
transformers 0.99618
typeshed 0.74233
warehouse 0.99601
zulip 0.99727

@MichaReiser
Copy link
Member Author

MichaReiser commented Aug 10, 2023

@MichaReiser MichaReiser force-pushed the fmt-on-off branch 4 times, most recently from fcdea72 to fccab61 Compare August 10, 2023 12:29
- 'unformatted'
+ a, b = *hello
+ "unformatted"
+ #hey, that won't work
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It does ;)

def spaces(a=1, b=(), c=[], d={}, e=True, f=-1, g=1 if False else 2, h="", i=r""):
offset = attr.ib(default=attr.Factory(lambda: _r.uniform(1, 2)))
@@ -63,15 +77,15 @@
@@ -63,15 +63,15 @@

something = {
# fmt: off
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ruff doesn't support expression level suppression comments.

+)
@@ -4,17 +4,10 @@
3, 4,
])
# fmt: on
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suppression comments between decorators are not supported.

@MichaReiser MichaReiser added the formatter Related to the formatter label Aug 10, 2023
@MichaReiser MichaReiser marked this pull request as ready for review August 10, 2023 12:31
@MichaReiser MichaReiser mentioned this pull request Aug 10, 2023
6 tasks
// ```
write!(f, [empty_line()])?;
}
write!(f, [first.format()])?;
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had to move the first.format out to avoid having to handle suppression comments in each branch.

Comment on lines 107 to 108
write!(f, [empty_line(), second.format()])?;
last = second;
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had to move this into the loop to avoid having to handle the suppression comments on second

Comment on lines 102 to 116
let mut preceding = if comments
.leading_comments(first)
.iter()
.any(|comment| comment.is_suppression_off_comment(source))
{
write_suppressed_statements_starting_with_leading_comment(first, &mut iter, f)?
} else if comments
.trailing_comments(first)
.iter()
.any(|comment| comment.is_suppression_off_comment(source))
{
write_suppressed_statements_starting_with_trailing_comment(first, &mut iter, f)?
} else {
first.format().fmt(f)?;
first
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is new

Comment on lines 168 to 185
} else if after_docstring {
// Enforce an empty line after a class docstring, e.g., these are both stable
// formatting:
// ```python
// class Test:
// """Docstring"""
//
// ...
//
//
// class Test:
//
// """Docstring"""
//
// ...
// ```
empty_line().fmt(f)?;
after_docstring = false;
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is new

Comment on lines 224 to 277
if comments
.leading_comments(following)
.iter()
.any(|comment| comment.is_suppression_off_comment(source))
{
preceding = write_suppressed_statements_starting_with_leading_comment(
following, &mut iter, f,
)?;
} else if comments
.trailing_comments(following)
.iter()
.any(|comment| comment.is_suppression_off_comment(source))
{
preceding = write_suppressed_statements_starting_with_trailing_comment(
following, &mut iter, f,
)?;
} else {
following.format().fmt(f)?;
preceding = following;
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is new

@github-actions
Copy link
Contributor

github-actions bot commented Aug 10, 2023

PR Check Results

Benchmark

Linux

group                                      main                                   pr
-----                                      ----                                   --
formatter/large/dataset.py                 1.03      4.4±0.11ms     9.2 MB/sec    1.00      4.3±0.13ms     9.5 MB/sec
formatter/numpy/ctypeslib.py               1.00   821.4±22.03µs    20.3 MB/sec    1.03   843.9±21.86µs    19.7 MB/sec
formatter/numpy/globals.py                 1.00     84.0±2.83µs    35.1 MB/sec    1.04     87.7±1.74µs    33.6 MB/sec
formatter/pydantic/types.py                1.00  1695.1±38.16µs    15.0 MB/sec    1.01  1718.9±93.61µs    14.8 MB/sec
linter/all-rules/large/dataset.py          1.07     12.5±0.15ms     3.3 MB/sec    1.00     11.7±0.37ms     3.5 MB/sec
linter/all-rules/numpy/ctypeslib.py        1.03      3.3±0.05ms     5.0 MB/sec    1.00      3.2±0.06ms     5.2 MB/sec
linter/all-rules/numpy/globals.py          1.06   471.6±10.18µs     6.3 MB/sec    1.00   444.3±10.05µs     6.6 MB/sec
linter/all-rules/pydantic/types.py         1.06      6.4±0.08ms     4.0 MB/sec    1.00      6.1±0.13ms     4.2 MB/sec
linter/default-rules/large/dataset.py      1.09      6.7±0.13ms     6.1 MB/sec    1.00      6.1±0.08ms     6.7 MB/sec
linter/default-rules/numpy/ctypeslib.py    1.06  1436.6±14.27µs    11.6 MB/sec    1.00  1356.3±30.18µs    12.3 MB/sec
linter/default-rules/numpy/globals.py      1.03    167.4±2.01µs    17.6 MB/sec    1.00    162.4±4.26µs    18.2 MB/sec
linter/default-rules/pydantic/types.py     1.06      2.9±0.04ms     8.7 MB/sec    1.00      2.8±0.05ms     9.2 MB/sec

Windows

group                                      main                                   pr
-----                                      ----                                   --
formatter/large/dataset.py                 1.09      5.1±0.31ms     8.0 MB/sec    1.00      4.7±0.41ms     8.7 MB/sec
formatter/numpy/ctypeslib.py               1.05   927.6±35.41µs    18.0 MB/sec    1.00   882.6±50.07µs    18.9 MB/sec
formatter/numpy/globals.py                 1.00     89.5±4.45µs    33.0 MB/sec    1.00     89.4±4.89µs    33.0 MB/sec
formatter/pydantic/types.py                1.04  1861.9±71.24µs    13.7 MB/sec    1.00  1783.5±86.34µs    14.3 MB/sec
linter/all-rules/large/dataset.py          1.00     14.9±0.49ms     2.7 MB/sec    1.03     15.4±0.35ms     2.6 MB/sec
linter/all-rules/numpy/ctypeslib.py        1.00      4.0±0.17ms     4.1 MB/sec    1.08      4.4±0.14ms     3.8 MB/sec
linter/all-rules/numpy/globals.py          1.00   531.2±21.23µs     5.6 MB/sec    1.04   554.7±20.82µs     5.3 MB/sec
linter/all-rules/pydantic/types.py         1.00      7.7±0.32ms     3.3 MB/sec    1.05      8.1±0.28ms     3.1 MB/sec
linter/default-rules/large/dataset.py      1.02      8.1±0.40ms     5.0 MB/sec    1.00      8.0±0.39ms     5.1 MB/sec
linter/default-rules/numpy/ctypeslib.py    1.04  1780.4±51.26µs     9.4 MB/sec    1.00  1716.0±61.20µs     9.7 MB/sec
linter/default-rules/numpy/globals.py      1.06   231.7±10.42µs    12.7 MB/sec    1.00    219.6±8.22µs    13.4 MB/sec
linter/default-rules/pydantic/types.py     1.03      3.8±0.11ms     6.8 MB/sec    1.00      3.7±0.13ms     7.0 MB/sec

crates/ruff_python_formatter/src/verbatim.rs Outdated Show resolved Hide resolved
crates/ruff_python_formatter/src/verbatim.rs Show resolved Hide resolved
crates/ruff_python_formatter/src/verbatim.rs Show resolved Hide resolved
crates/ruff_python_formatter/src/verbatim.rs Outdated Show resolved Hide resolved
crates/ruff_python_formatter/src/verbatim.rs Show resolved Hide resolved
crates/ruff_python_formatter/src/verbatim.rs Show resolved Hide resolved
format_on_comment.start(),
)),
trailing_comments(std::slice::from_ref(format_on_comment)),
trailing_comments(formatted_comments),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could it ever be the case that we're missing out on "special" formatting of some of these comments? I don't have a more precise idea of when that could happen, but just acknowledging that these are typically handled by children, and we're now intercepting and formatting them here.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch. This could be a problem if a statement overrides fmt_leading_comments or fmt_trailing_comments. I checked, and no expression or statement overrides these methods on FormatNodeRule. I went ahead and inlined the functions to not give the impression that this behavior should be customized.

Either way, I think it would be okay to format the comments with the default formatting. It may not be perfect but we're dealing with suppressed ranges anyway.

crates/ruff_python_formatter/src/verbatim.rs Outdated Show resolved Hide resolved
}
}

struct CommentRangeIter<'a> {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you intentionally not #[derive(Debug)] here? (Asking for my own learning.)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mainly because I didn't need it, it's a private type, and printing iterators is always a bit awkward.

crates/ruff_python_formatter/src/verbatim.rs Outdated Show resolved Hide resolved
@MichaReiser
Copy link
Member Author

MichaReiser commented Aug 14, 2023

The main reason why this PR regresses performance is because it is now necessary to look up the leading and trailing comments for every statement, and we do the same when formatting the statement.

We could improve this by moving the leading/trailing comments formatting for statements into suite.rs to avoid the additional hash map lookup. We may want to do this in the future, but it isn't necessary doing right now.

@MichaReiser MichaReiser enabled auto-merge (squash) August 14, 2023 15:48
@MichaReiser MichaReiser merged commit 09c8b17 into main Aug 14, 2023
16 checks passed
@MichaReiser MichaReiser deleted the fmt-on-off branch August 14, 2023 15:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
formatter Related to the formatter
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants