PEP 498: docstrings as f-strings

[1st1]

BPO	28739
Nosy	@gvanrossum, @rhettinger, @terryjreedy, @ericvsmith, @ned-deily, @vadmium, @serhiy-storchaka, @1st1, @Mariatta, @JockeTF
PRs	bpo-28739: Document that f-strings cannot be used as docstring #592 [3.6] bpo-28739: Document that f-strings cannot be used as docstring (GH-592) #600 [Do Not Merge] Convert Misc/NEWS so that it is managed by towncrier #552
Files	fstring-no-docstring.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2017-03-10.18:12:56.810>
created_at = <Date 2016-11-18.20:43:51.315>
labels = ['type-bug', '3.7', 'docs']
title = 'PEP 498: docstrings as f-strings'
updated_at = <Date 2017-03-31.16:36:13.068>
user = 'https://github.com/1st1'

bugs.python.org fields:

activity = <Date 2017-03-31.16:36:13.068>
actor = 'dstufft'
assignee = 'docs@python'
closed = True
closed_date = <Date 2017-03-10.18:12:56.810>
closer = 'Mariatta'
components = ['Documentation']
creation = <Date 2016-11-18.20:43:51.315>
creator = 'yselivanov'
dependencies = []
files = ['45658']
hgrepos = []
issue_num = 28739
keywords = []
message_count = 28.0
messages = ['281166', '281167', '281168', '281172', '281174', '281175', '281182', '281186', '281188', '281740', '281755', '281756', '281759', '281786', '281799', '281800', '281801', '281808', '281810', '282910', '282924', '282934', '287628', '289338', '289397', '289398', '290241', '290242']
nosy_count = 12.0
nosy_names = ['gvanrossum', 'rhettinger', 'terry.reedy', 'eric.smith', 'ned.deily', 'docs@python', 'python-dev', 'martin.panter', 'serhiy.storchaka', 'yselivanov', 'Mariatta', 'JockeTF']
pr_nums = ['592', '600', '552']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue28739'
versions = ['Python 3.6', 'Python 3.7']

[1st1]

Can f-strings be used as docstrings?

Right now:

    class Foo:
       f'spam'
    Foo.__doc__ is None

I couldn't find that f-strings cannot be used as docstrings in neither PEP-498 not in the 3.6 documentation, so I suppose this is a bug.

[ericvsmith]

As you've seen, the answer is "no"!

We'd need to add logic to evaluate them at function definition time. That would be a slight noticeable change, if the expressions had side effects.

[vadmium]

There was a bit of discussion at the top of <https://bugs.python.org/issue25179#msg254298\>.

IMO it would be simpler do disallow all f-strings as docstrings. Otherwise, what would the result of this be:

name = "module level"
class C:
    name = "class level"
    def m(self, name="default param"):
        f"{name}"

[1st1]

IMO it would be simpler do disallow all f-strings as docstrings.

How exactly you want to disallow them? Raise SyntaxError? If you don't raise anything, then the behaviour is just confusing -- the interpreter parses them, but __doc__ is None.

I think this needs to be fixed. Eric, can we still fix this in 3.6? If not, then we need to update the PEP and the docs.

[vadmium]

Actually, testing your code fragment, it seems you do get a doc string when the f-string has no substitutions in curly brackets, otherwise you don’t get any doc string. Maybe this is due to how different forms of string are compiled.

>>> class Foo:
...    f'spam'  # Compiled as plain 'spam'
... 
>>> Foo.__doc__
'spam'
>>> class Foo:
...     'spam' f'{"MMM"}'  # Compiled as f'spam{"MMM"}'
... 
>>> Foo.__doc__ is None
True

[vadmium]

Having an unassigned f-string as the first statement is still valid syntax, so could at most be a warning, not a SyntaxError.

[ericvsmith]

It's Ned's call, but I wouldn't recommend changing this in 3.6, at least not 3.6.0.

As Martin points out, the reason f'foo' is a "normal" string has to do with how strings and f-strings are assembled and concatenated.

Similarly:
'foo' f'bar' 'baz'
is a normal string, 'foobarbaz'.

I can't think of another place that requires a "normal" string, but if they exist, they'd be affected by this, too.

[gvanrossum]

Might I point out the precedent of b-strings? Those also don't contribute
to __doc__.

[ned-deily]

Since this was previously discussed and rejected (in bpo-25179), I don't think we should revisit this now for 3.6, other than potentially a documentation tweak.

[terryjreedy]

Much of this discussion seems to duplicate and effectively re-open of bpo-25179, wherein it was decided/accepted that true, non-degenerate, non-trivial, non-constant, f-strings that actually do formatting are not constants and do not and should not become docstrings. I agree. I think this issue should either be closed as 'not a bug' or redefined as a doc issue.

It was noted by Martin P. in bpo-25179 that "a constant f-string without any interpolations does become a doc string." That is because such is really a string literal and not really an f-string, in the same sense that 'circle of radius 0' is really a point and not a circle.

The current glossary entry is

"docstring
A string literal which appears as the first expression in a class, function or module. While ignored when the suite is executed, it is recognized by the compiler and put into the __doc__ attribute of the enclosing class, function or module. Since it is available via introspection, it is the canonical place for documentation of the object."

I suggest adding "Bytestring literals and non-trivial f-strings do not become docstrings." as the second sentence.

[rhettinger]

[TJR]

I think this issue should either be closed as 'not a bug'
or redefined as a doc issue.

I concur.

FWIW, here are some comparisons.
Not allowed:

'Not counted as' + ' a docstring'
'Not a docstring ' * 10
b'Also not a docstring'

Allowed:

'This is a ' 'docstring'
r'This is a docstring'

[gvanrossum]

I don't really care that much, but I personally think that it would be more consistent (and a simpler rule) if *no* f-string (not even ones without substitutions) were to be allowed as docstrings.

In all other examples that Raymond shows it's the syntactic form that matters -- no on b-strings, yes on r-strings, yes on concatenation (using space), no on +, etc.

The language reference clearly defines f-strings as all strings with an f-prefix, and says that they *may* contain replacement fields. So it's clear that an f-string without replacements is still an f-string, and it is still distinguished from other strings. Hence I think it should not be allowed as a docstring.

(Also, what purpose could using the f-prefix for a docstring possibly have? All the other allowable combinations do have a use.)

[serhiy-storchaka]

Adding more confusion, the expression ('This is a docstring') is accepted as a docstring despite the fact that it is not a string literal.

The cases f'string' and ('string') looks similar to me. Both are simple expressions that become indistinguishable from string literals due to some optimization at parser level. It would be nice to disallow them as docstrings, but it may be not easy to do.

[gvanrossum]

OK, clearly the code that sets __doc__ is too closely tied to the generated
code (or to the reduced AST used to generate code). I still think code that
uses any of these is on thin ice and should expect to be broken in the
future.

[terryjreedy]

I withdraw my previously suggested addition to the Docstring glossary entry (msg281740). It implies that trivial f-strings are acceptable and I agree that other implementations and future Cpython should be free to strictly follow the literal meaning of the first sentence: docstring = initial string literal expression.

I suggest instead that 'string literal' in the first sentence link to https://docs.python.org/3/reference/lexical_analysis.html#string-and-bytes-literals. This would make it clearer that 'string literal' is being used the the Python technical sense rather than in any more informal English sense.

We could possibly add a version of what Guido said above, such as: "(Acceptance of anything other than a string literal as a docstring is an implementation accident and should not be relied upon.)"

[serhiy-storchaka]

Proposed patch makes f-strings not be accepted as docstrings. It also disallow f-strings in ast.literal_eval().

[gvanrossum]

I think such a patch is fine -- for 3.6.1.

Also note that linking to the definition of "string literal" is
insufficient, assuming we will want to continue supporting literal
concatenation (
https://docs.python.org/3/reference/lexical_analysis.html#string-literal-concatenation).
(And I want to, because I can see a use case.)

[terryjreedy]

I considered concatenated string literals to be included in 'string literal'. If that is not obvious, then it should be made so. Replace 'string literal' with 'string literal or concatenated strings literals' and link each part to their respective (and successive) sections.

[gvanrossum]

I was just noticing that the formal grammar in the reference manual first
defines a single string literal and then separately describes concatenation.

[serhiy-storchaka]

Eric, could you please make a review of the patch?

[ericvsmith]

It looks good to me, save for one tiny issue. I left a review comment.

[python-dev]

New changeset 30341d5c1423 by Serhiy Storchaka in branch '3.6':
Issue bpo-28739: f-string expressions no longer accepted as docstrings and
https://hg.python.org/cpython/rev/30341d5c1423

New changeset 8e0f147dfa3d by Serhiy Storchaka in branch 'default':
Issue bpo-28739: f-string expressions no longer accepted as docstrings and
https://hg.python.org/cpython/rev/8e0f147dfa3d

[JockeTF]

I got slightly confused here while playing around.

Python 3.6.0 (default, Jan 31 2017, 11:39:39) 
[GCC 4.9.2] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> class Huacaya:
...   f"""Huacaya!"""
... 
>>> class Suri:
...   f"""{'Suri!'}"""
... 
>>> Huacaya.__doc__ is None
False
>>> Suri.__doc__ is None
True

At first I thought f-strings *did* work as docstrings since it worked just fine for the first class. But, the docstring suddenly vanished when putting an actual expression into it.

[Mariatta]

Hi, I updated the documentation mentioning that f-strings cannot be used as docstring. Please review it.
Thanks.

[Mariatta]

Thanks for reviewing, Serhiy and Eric. Documentation has been updated and backported to 3.6.

OK to close this issue?

[ericvsmith]

Yes, I think it can be closed. Thanks!

[Mariatta]

New changeset ff6f371 by Mariatta in branch '3.6':
bpo-28739: Document that f-strings cannot be used as docstring (GH-592) (GH-600)
ff6f371

[Mariatta]

New changeset d4e8928 by Mariatta in branch 'master':
bpo-28739: Document that f-strings cannot be used as docstring (GH-592)
d4e8928