Tutorial documentation contains undefined reference to #!

[DimitrisJim]

BPO	29381
Nosy	@rhettinger, @marco-buttu, @DimitrisJim, @Mariatta
Files	interpreter_tut.patch interpreter_tut2.7.patch issue29381.patch issue29381v2.patch issue29381v3.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/Mariatta'
closed_at = <Date 2017-02-02.05:00:23.157>
created_at = <Date 2017-01-27.17:20:02.661>
labels = ['3.7', 'docs']
title = 'Tutorial documentation contains undefined reference to #!'
updated_at = <Date 2017-02-02.06:00:28.531>
user = 'https://github.com/DimitrisJim'

bugs.python.org fields:

activity = <Date 2017-02-02.06:00:28.531>
actor = 'python-dev'
assignee = 'Mariatta'
closed = True
closed_date = <Date 2017-02-02.05:00:23.157>
closer = 'python-dev'
components = ['Documentation']
creation = <Date 2017-01-27.17:20:02.661>
creator = 'Jim Fasarakis-Hilliard'
dependencies = []
files = ['46438', '46439', '46452', '46453', '46458']
hgrepos = []
issue_num = 29381
keywords = ['patch']
message_count = 22.0
messages = ['286378', '286388', '286410', '286414', '286415', '286424', '286465', '286468', '286470', '286481', '286488', '286502', '286508', '286509', '286736', '286737', '286738', '286742', '286743', '286744', '286745', '286746']
nosy_count = 6.0
nosy_names = ['rhettinger', 'docs@python', 'python-dev', 'marco.buttu', 'Jim Fasarakis-Hilliard', 'Mariatta']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue29381'
versions = ['Python 2.7', 'Python 3.6', 'Python 3.7']

[DimitrisJim]

After moving a certain chunk of the 'interpreter.rst' contents to 'appendix.rst' in bpo-16827 the reference to #! in the section '2.2.3. Source Code Encoding' is currently confusing for new readers.

Attached patches reword the sentence to remove that reference for 3.4+ and 2.7.

[rhettinger]

Mariatta, could you check this patch and tell us your recommendation about whether it should be applied, modified, or rejected?

[DimitrisJim]

Hi, Raymond. Is Mariatta responsible for reviewing Documentation submissions? If so, should I nosy Mariatta in any future documentation issues?

[marco-buttu]

You wrote: "It is also possible to specify a different encoding for source files. In order to do this, you can use a special comment line that defines the source file encoding::". I think that is not true, because the line have to be the first line, or right below a comment (just one, as in the case of the shebang). For instance, in this case Python apply the declared coding:

  $ cat foo.py
  # The first line is a comment
  # -*- coding: ascii -*-
  print('è')  # Encoded in UTF-8
  $ python foo.py
     ...
  SyntaxError: encoding problem: ascii

In this case it does not:

  $ cat foo.py
  # The first line is a comment
  # and also the sencond line
  # -*- coding: ascii -*-
  print('è')  # Encoded in UTF-8
  $ python foo.py 
  è

But I think you are right that the current doc is confusing. Maybe yon can write something like this:

"It is also possible to specify a different encoding for source files. In order to do this, put one special comment line to define the source file encoding:

# -- coding: encoding --

This coding comment has to be the first line of the file, or the second line in case the first one is the #! line."

[DimitrisJim]

Indeed, this does create an issue.

The last sentence in the documentation actually specifies where the encoding comment can be but doesn't strictly specify it can be on the second line if and only if preceded by #!.

I'm thinking the last sentence should contain the additional change, that is, change it from :

The special encoding comment must be in the first or second line within the file.

to:

The special encoding comment must be in the first line, or, if the first line is used to create an executable Python script, the second line within the file.

linking to (https://docs.python.org/3/tutorial/appendix.html#executable-python-scripts) accordingly.

Let's see what Mariatta says about these proposals. Thanks for noticing.

[rhettinger]

Is Mariatta responsible for reviewing Documentation submissions?

No, she is a new core developer and this is a good issue for her to do the initial review and post any thoughts on the subject. We've all got to start somewhere.

If so, should I nosy Mariatta in any future documentation issues?

You did everything just right. Just marking this as a doc patch is sufficient.

[Mariatta]

There are certain rules about the encoding declaration line.
It has to be the first line of the source code, or in the case that the file starts with a unix "shebang" line, then the encoding declaration has to be on the second line. In fact, the first or the second lines of the source code has to follow a certain regular expression. This is all described in PEP-263, but we don't have to repeat all of those here. IMO we can just refer the reader to the PEP.

I've prepared a patch that addresses this. Please review, and let me know if you have additional feedback.

Thanks :)

[rhettinger]

Suggestions:

• Please add a cross-link to from '''UNIX “shebang” line"''' to 16.1.2. Executable Python Scripts.

• Instead of a link to a PEP (which is outside the main documentation), let's just add an example:

  # /usr/bin/env python3
  # -- coding: cp-1252 --

[Mariatta]

Thanks for the feedback, Raymond. I adjusted my patch.

[marco-buttu]

The patch LGTM. I think there is just one type (see review).

By the way, looking at the PEP-0263, the sentence "To define a source code encoding, a magic comment must be placed into the source files either as first or second line in the file" actually is not true. Also because I did not see any point in the PEP that clarifies that the fist line has to be a comment. Maybe it could be deduced by the examples (all using the shebang), but maybe not, and the user could think this is a valid declaration:

print('très jolie')
# -*- coding: ascii -*-

If you think is worth to clarify, I open an issue with a patch for the PEP-0263.

[DimitrisJim]

Added a comment too. Other than that and the comment by Marco it looks fine to me too.

[Mariatta]

Thanks all for the feedback. I updated the patch.

Marco, IMO it's not necessary to update the PEP.
This section of the documentation serves as a tutorial for it, and the update in this patch clarifies the situation.

[rhettinger]

It looks like everyone is happy and the patch is ready for Mariatta to apply.

The procedure is:
$ hg pull -u
$ hg diff # Verify it has only your intended patch
$ hg update 3.6
$ cd Doc
$ make html
$ open build/html/index.html # Now review generated HTML
$ hg commit -m "Issue bpo-29381: Clarify ordering of UNIX shebang line as source encoding line"
$ hg update default
$ hg merge 3.6
$ hg diff # Verify it has only your intended patch
$ hg commit -m "merge"
$ hg push

Then return to this issue, verify the bot has posted the commit.

Write a comment thanking the OP and reviewers. Then mark the status as "closed" and resolution as "fixed".

[rhettinger]

I would only post this to 2.7, 3.6 and 3.7. Also, no Misc/NEWS entry is needed.

[python-dev]

New changeset 3d712292f2fa by Mariatta Wijaya in branch '3.6':
Issue bpo-29381: Clarify ordering of UNIX shebang line as source encoding line
https://hg.python.org/cpython/rev/3d712292f2fa

[python-dev]

New changeset 483d9133fd7e by Mariatta Wijaya in branch '3.6':
Issue bpo-29381: Clarify ordering of UNIX shebang line as source encoding line
https://hg.python.org/cpython/rev/483d9133fd7e

New changeset 762a93935afd by Mariatta Wijaya in branch 'default':
Issue bpo-29381: merge with 3.6
https://hg.python.org/cpython/rev/762a93935afd

[python-dev]

New changeset df356d3c916e by Mariatta Wijaya in branch '2.7':
Issue bpo-29381: Clarify ordering of UNIX shebang line as source encoding line
https://hg.python.org/cpython/rev/df356d3c916e

[Mariatta]

Thanks, Jim, Marco, and Raymond :)

[python-dev]

New changeset abfa17511f7ce8f1a6394f28f82ffb9a916fcf03 by Mariatta Wijaya in branch '3.6':
Issue bpo-29381: Clarify ordering of UNIX shebang line as source encoding line
abfa175

[python-dev]

New changeset 848eeb1debc94a82660bf5af4e3d554b02b81c2e by Mariatta Wijaya in branch '3.6':
Issue bpo-29381: Clarify ordering of UNIX shebang line as source encoding line
848eeb1

[python-dev]

New changeset 3f5b24dd4b38bf885a80f4bb5f605cd6b0c49ead by Mariatta Wijaya in branch '2.7':
Issue bpo-29381: Clarify ordering of UNIX shebang line as source encoding line
3f5b24d

[python-dev]

New changeset abfa17511f7ce8f1a6394f28f82ffb9a916fcf03 by Mariatta Wijaya in branch 'master':
Issue bpo-29381: Clarify ordering of UNIX shebang line as source encoding line
abfa175

New changeset 848eeb1debc94a82660bf5af4e3d554b02b81c2e by Mariatta Wijaya in branch 'master':
Issue bpo-29381: Clarify ordering of UNIX shebang line as source encoding line
848eeb1

New changeset 35519d021ece942efc4a710ea3e94bcc5c06b130 by Mariatta Wijaya in branch 'master':
Issue bpo-29381: merge with 3.6
35519d0