Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add f-string parsing for Python 3.12 (PEP 701) #1027

Merged
merged 1 commit into from Oct 1, 2023

Conversation

encukou
Copy link
Contributor

@encukou encukou commented Sep 21, 2023

Since Python 3.12, f-strings are tokenized and parsed like the rest of Python's grammar, using the new tokens FSTRING_START, FSTRING_MIDDLE and FSTRING_END, as described in pep 701 #new-tokens.

Make the babel message extractor concatenate these three if they're adjacent to each other. If they're not, that means there are dynamic substitutions, so the f-string is ignored.

@encukou
Copy link
Contributor Author

encukou commented Sep 21, 2023

Not sure if y'all want to add 3.12 to the CI yet?
For that to pass setuptools will need to be added to tox.ini as it's needed in the data setup.

@encukou encukou changed the title Add F f-string parsing for Python 3.12 (PEP 701) Add f-string parsing for Python 3.12 (PEP 701) Sep 21, 2023
@mgorny
Copy link
Contributor

mgorny commented Sep 21, 2023

Thanks. I can confirm that with this patch, the tests pass both on 3.11 and 3.12.

@akx akx self-requested a review September 21, 2023 18:30
@codecov
Copy link

codecov bot commented Oct 1, 2023

Codecov Report

Merging #1027 (8260a4d) into master (6fc07a2) will decrease coverage by 0.17%.
Report is 1 commits behind head on master.
The diff coverage is 50.00%.

@@            Coverage Diff             @@
##           master    #1027      +/-   ##
==========================================
- Coverage   90.94%   90.78%   -0.17%     
==========================================
  Files          25       25              
  Lines        4375     4393      +18     
==========================================
+ Hits         3979     3988       +9     
- Misses        396      405       +9     
Files Coverage Δ
babel/messages/extract.py 92.89% <50.00%> (-2.42%) ⬇️

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

@akx akx force-pushed the py3.12-fstring-extraction branch from be26d03 to 1f06b36 Compare October 1, 2023 10:48
Since Python 3.12, f-strings are tokenized and parsed like the rest
of Python's grammar, using the new tokens FSTRING_START, FSTRING_MIDDLE
and FSTRING_END.

Make the babel message extractor concatenate these three if they're
adjacent to each other. If they're not, that means there are dynamic
substitutions, so the f-string is ignored.
@akx akx force-pushed the py3.12-fstring-extraction branch from 1f06b36 to 8260a4d Compare October 1, 2023 10:48
Copy link
Member

@akx akx left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! I tested this locally on 3.12 and seems to work great. I took the liberty of fixing the lint fail too.

@akx akx enabled auto-merge (squash) October 1, 2023 10:49
@akx
Copy link
Member

akx commented Oct 1, 2023

Not sure if y'all want to add 3.12 to the CI yet? For that to pass setuptools will need to be added to tox.ini as it's needed in the data setup.

Working on that in #1028 :)

@akx akx merged commit 72bccff into python-babel:master Oct 1, 2023
20 of 22 checks passed
@encukou encukou deleted the py3.12-fstring-extraction branch October 3, 2023 07:37
@BrentHuang
Copy link

how to use? In babel 2.14.0, the code can't be extracted to .pot file:

foo= 'Hardy'
print(_(f"Hello {foo}"))

@BrentHuang
Copy link

BrentHuang commented Apr 26, 2024

how to use? In babel 2.14.0, the code can't be extracted to .pot file:

foo= 'Hardy'
print(_(f"Hello {foo}"))
foo= 'Hardy'
print(f'{_("Hello")} {foo}')

in the code above, extract ok, but not translated by gettext.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants