Fix D204 when there's a semicolon after a docstring

[konstin]

Summary

Another statement on the same line as the docstring would previous make the D204 (newline after docstring) fix fail:

class StatementOnSameLineAsDocstring:
    "After this docstring there's another statement on the same line separated by a semicolon." ;priorities=1
    def sort_services(self):
        pass

The fix handles this case manually:

class StatementOnSameLineAsDocstring:
    "After this docstring there's another statement on the same line separated by a semicolon."

    priorities=1
    def sort_services(self):
        pass

Fixes #7088

Test Plan

Added a new D test case

[konstin]

Current dependencies on/for this PR:

• main
  • PR Ignore single quote docstrings with newline escape #7173
    • PR Fix D204 when there's a semicolon after a docstring #7174 👈

This comment was auto-generated by Graphite.

[konstin]

@MichaReiser since this is not indentation but whitespace between statement and a following ; i don't need to use the python whitespace functions, right?

[charliermarsh]

It’s best to use the Python whitespace functions unless you’re parsing text within a string or within a comment (is my understanding).

[MichaReiser]

From the python spec

Except at the beginning of a logical line or in string literals, the whitespace characters space, tab and formfeed can be used interchangeably to separate tokens. Whitespace is needed between two tokens only if their concatenation could otherwise be interpreted as a different token (e.g., ab is one token, but a b is two tokens).

source

To me this means that any other whitespace should not be considered whitespace and are a syntax error (because whitespace isn't a valid identifier start).

I use the python whitespace methods consistently to:

• Avoid having to reason about whether it is guaranteed that the input program is correct and thus, it is safe to accept all whitesapce
• is_python_whitespace (or the trim variants) could in theory be faster because they can operate on bytes (no unicode)

[codspeed-hq]

CodSpeed Performance Report

Merging #7174 will degrade performances by 4.52%

_{Comparing 09-05-Fix_D204_when_there_s_a_semicolon_after_a_docstring (9d114a8) with 09-05-Ignore_single_quote_docstrings_with_newline_escape (2d20e37)}

Summary

❌ 1 regressions
✅ 24 untouched benchmarks

⚠️ Please fix the performance issues or acknowledge them on CodSpeed.

Benchmarks breakdown

	Benchmark	09-05-Ignore_single_quote_docstrings_with_newline_escape	09-05-Fix_D204_when_there_s_a_semicolon_after_a_docstring	Change
❌	linter/all-rules[unicode/pypinyin.py]	15.4 ms	16.2 ms	-4.52%

[charliermarsh]

What if it’s followed by a comment?

[konstin]

Good catch, added a comment test

[MichaReiser]

From the python spec

Except at the beginning of a logical line or in string literals, the whitespace characters space, tab and formfeed can be used interchangeably to separate tokens. Whitespace is needed between two tokens only if their concatenation could otherwise be interpreted as a different token (e.g., ab is one token, but a b is two tokens).

source

To me this means that any other whitespace should not be considered whitespace and are a syntax error (because whitespace isn't a valid identifier start).

I use the python whitespace methods consistently to:

• Avoid having to reason about whether it is guaranteed that the input program is correct and thus, it is safe to accept all whitesapce
• is_python_whitespace (or the trim variants) could in theory be faster because they can operate on bytes (no unicode)

[github-actions]

PR Check Results

Ecosystem

✅ ecosystem check detected no changes.