Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix incorrect linenos on fstring tokens with escaped newlines #4423

Merged
merged 4 commits into from
Aug 1, 2024

Conversation

JelleZijlstra
Copy link
Collaborator

I don't think this can affect Black itself much (maybe for formatting
ranges), but I ran into this with https://github.com/JelleZijlstra/lib2toast

cc @tusharsadhwani

JelleZijlstra and others added 4 commits August 1, 2024 10:18
Copy link

github-actions bot commented Aug 1, 2024

diff-shades reports zero changes comparing this PR (d193bd5) to main (7fa1faf).


What is this? | Workflow run | diff-shades documentation

if __name__ == "__main__":
code = sys.stdin.read()
tokens = get_tokens(code)
text = f"assert_tokenizes({code!r}, {tokens!r})"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

consider using the builtin tokenize module for generating tests? that way you don't have to verify the locations before adding the test.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure how compatible exactly are the tokens between Python and blib2to3 though.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They don't match exactly in all cases (e.g., builtin has an extra encoding token at the beginning). We can try to work towards exact compatibility, but not sure that's necessary.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, for now this is good

@JelleZijlstra JelleZijlstra merged commit 4b4ae43 into psf:main Aug 1, 2024
46 checks passed
@JelleZijlstra JelleZijlstra deleted the fstrlineno branch August 1, 2024 18:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants