Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Comparison to PEP 501 #7

Closed
gvanrossum opened this issue May 9, 2022 · 7 comments
Closed

Comparison to PEP 501 #7

gvanrossum opened this issue May 9, 2022 · 7 comments

Comments

@gvanrossum
Copy link
Collaborator

gvanrossum commented May 9, 2022 •
edited

PEP 501 introduces i-strings, which are syntactically the same as f-strings (using i instead of f as prefix) but return an object representing the entire i-string. The interpolations and their fields can be extracted from this.

  • It is opinionated about byte strings: it doesn't support them. (But the Discussion section suggests they could be added if needed.)
  • It claims "string literal concatenation operates as normal, with the entire combined literal forming the interpolation template." This seems to imply that i"x" "y" "z" is equivalent to i"xyz". But this is not how f-strings work: f"{foo}" "{bar}" does not interpolate bar. (UPDATE: Maybe this part of PEP 498 wasn't settled when PEP 501 was written.)
  • You cannot get the original \ escapes back. You could not tell whether the source says i"h\x61m" or i"ham" -- the raw string is always "ham".
  • The interpolated expressions are evaluated immediately (when the i-string is encountered), not turned into lambdas. See section below.
  • It uses format(t) to render the final string (though you can also explicitly call t.render()).
  • Use cases like html, sql etc. are supported as plain functions that take an InterpolationTemplate instance, e.g. html(i"<body>{stuff}</body>"). This is probably less disturbing for users who are familiar with the status quo. (UPDATE: And avoids corner cases like fr, rf, Fr, Rf etc. being unavailable as tags.)
  • It translates conversions (!r, !s, !a) into explicit calls to repr(), str(), ascii(). (It doesn't need to recover the conversion letter from the template object.)
  • I cannot figure out its stance about interpolations occurring in format specifiers. Possibly this did not occur to Nick, or they weren't in PEP 498 yet (that PEP wasn't approved yet when PEP 501 was last posted).
  • It admits it doesn't solve the i18n problem, even though that was the original motivating use case. Then again, neither do tag strings: shared templates pose a problem here, but the bigger issue may be that the i18n community is likely stuck with the interpolation syntax of string.Template.

About the logging module: Not lambdafying interpolation expressions may avoid some issues with turning plain variables into cells (though I'm not sure such issues exist), but means that there's a minimal cost when i-strings are used as logging arguments -- they are always evaluated (though not formatted or concatenated with the fixed parts). Then again, this is also the case for the current best practice -- passing those expressions as extra arguments to the logging function. However, PEP 501 notes that the logging module supports using various builtin attributes of log records (e.g. %(name)s substitutes the log record name) and suggests that additional integration would be needed to request these (e.g. {'record.name'} would interpolate the log record name). PEP 501 doesn't specify exactly how this should be done, but suggests that the logging module would have to look at the raw text of the interpolated expression to distinguish this usage from interpolating a string variable whose contents happens to be 'record.name'. At that point I'd much rather just have lambdafied interpolations. (UPDATE:) Sadly, using {name} to refer to the record name would be a user trap, since this would override any local variable whose name happened to be a log record attribute (there are over 20 of those).
@jimbaker jimbaker mentioned this issue May 18, 2022
Closed
11 tasks
@jimbaker
Copy link
Owner

jimbaker commented Jun 5, 2022

Implemented support for i-strings and the corresponding InterpolationTemplate in examples:
https://github.com/jimbaker/tagstr/blob/main/examples/interpolation_template.py

I believe this is a complete implementation as specified by that PEP, other than showing the optional logging integration, which would require writing a minimal expression parser that would work with any dynamically scoped variables (and thus showing the nice aspect of lambdafied expressions).

@gvanrossum
Copy link
Collaborator Author

So the point for logging would be to detect when an interpolation references one of the predefined attributes, right? Delayed interpolation (also mentioned in that section) is not available in the status wou and people seem to be fine with that.

@jimbaker
Copy link
Owner

jimbaker commented Jun 5, 2022

Regular interpolation support is available in the implemented example, through the use of InterpolationTemplate's support for two callback parameters in the render method, which has this signature, per PEP 501:
def render(self, *, render_template=''.join, render_field=format):

So for a logging specific renderer, we need to write these two functions roughly as follows:

  • render_field would probably wrap the field to distinguish it from a source string.
  • render_template is a function to construct the desired object , so it would resemble how we write tag functions - iterating over args (tempate_parts in PEP 501's source for the render method), and put them together in a useful fashion, presumably as a LogRecord.

@gvanrossum
Copy link
Collaborator Author

So, to take a specific example, suppose we had a logging template log(i"Too spammy: {pathname}") then in order to recognize 'pathname' as a reference to a LogRecord attribute the logging module would have to look at the parsed_template attribute of the InterpolationTemplate and discover that the raw interpolation text is "pathname" -- or, as you say, it would have to parse that interpolation text into an expression and note mentions of pathname in the leaves of the AST. That's reasonable, and even if we had a dedicated log"Too spammy: {pathname}" tag string we would still have to do the same thing. (Sorry, I just had to research how all this actually works for myself and write it up here, even if it looks like you already know all that.)

I'm not sure why you'd want to wrap the field -- in the current implementation fields aren't treated special AFAIK -- you'd write log("Too spammy: %(pathname)s").

(Oh, I'm writing log(...) but it would have to be logging.debug(...), logging.warn(...) etc. So with the dedicated tag string it'd become logging.warn(log"Too spammy: {pathname}"). Maybe the tag shouldn't exactly be log.)

@gvanrossum
Copy link
Collaborator Author

There's now a Discourse thread about reopening PEP 501. I'm not excited about that, but some folks on Discourse appear to be.

@pauleveritt
Copy link
Collaborator

I believe we can close this. The author of the revived work said he'd adopt this tagstr work.

@jimbaker
Copy link
Owner

jimbaker commented Jul 3, 2023

I believe we can close this. The author of the revived work said he'd adopt this tagstr work.

Good to heaar about this collaboration! Closing accordingly.

@jimbaker jimbaker closed this as completed Jul 3, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@jimbaker @pauleveritt @gvanrossum