Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Disable the stability check with --line-ranges for now. #4034

Merged
merged 6 commits into from Nov 21, 2023

Conversation

yilei
Copy link
Contributor

@yilei yilei commented Nov 9, 2023

Description

When performing a second formatting pass with --line-ranges, we can't simply use the original lines specified by the user.

For example, unformatted:

def restrict_to_this_line(arg1,
  arg2,
  arg3):
    print  ( "This should not be formatted." )
    print  ( "This should not be formatted." )

If we let it format lines 1-3, after the first pass it becomes:

def restrict_to_this_line(arg1, arg2, arg3):
    print  ( "This should not be formatted." )
    print  ( "This should not be formatted." )

If we use the original 1-3 lines in a second pass, it would format all the lines now.

To solve this, we have an adjusted_lines function to calculate the new line ranges for the second pass. It uses the diffing algorithm from difflib.SequenceMatcher. Unfortunately it could produce undesired results for certain edge cases like a list of unformatted lines with the same content.

I think we could use a custom algorithm that takes advantage of the diff is produced by a formatter (it rarely adds/removes lines of code, except blank lines, optional parenthesis). But it's complicated so I'm proposing to simply disable the stability check for now to avoid the crash mentioned in #4033.

This could still lead to formatting more lines than requested, but it should be very rare in practice. Even when that happens, it usually doesn't hurt too much if a few extra lines are reformatted. Also, if we could get rid of the two-pass formatting, this will no longer be an issue. (We still need a better adjusted_lines to re-enable the stability check, but it should be a smaller issue.)

Checklist - did you ...

  • Add an entry in CHANGES.md if necessary?
  • Add / update tests if necessary?
  • Add new / update outdated documentation?

print ( "format me" )
print("format me")
print("format me")
print("format me")
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why are these lines formatted? Shouldn't it only format the second and third prints?

Copy link
Contributor Author

@yilei yilei Nov 18, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is caused exactly by the diff algorithm used in adjusted_lines. In the second formatting pass, the adjusted lines becomes 12-13 but it should have stayed as 10-11.

This PR doesn't fix this underlying issue (I'll file a separate issue if this PR is merged). Until it's really fixed, I'm proposing to skip the stability check (affected by the same issue) to avoid the crash.

The edge case is very rare in real code as it requires many same unformatted lines.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see, thanks. Maybe in that case we should put a warning in the docs for now about this bug.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree that this doesn't feel very likely in real code, though @tartley apparently did run into this.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated the doc.

@tartley can correct me, but #4033 appears to be a "toy" example for just trying out the new --line-ranges feature. #4052 can also serves the purpose of collecting real world example in case it happens.

What do you think?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed that I only encountered the problem using toy example code. By bad luck the toy example was the first thing I tried --line-ranges out on, while trying to integrate the use of it into my editor (e.g. so Vim would call Black on the selected range of lines). Although only a toy example it did genuinely confuse the heck out of me, and convinced me at the time that --line-ranges wasn't working for me at all.

yilei and others added 2 commits November 18, 2023 13:14
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
@JelleZijlstra JelleZijlstra merged commit a806298 into psf:main Nov 21, 2023
45 of 46 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants