First line can be executed twice

[serhiy-storchaka]

BPO	18960
Nosy	@malemburg, @loewis, @terryjreedy, @benjaminp, @ezio-melotti, @serhiy-storchaka
Files	double_first_line.py: Sample script source_encoding_second_line.patch source_encoding_second_line_2.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/serhiy-storchaka'
closed_at = <Date 2014-01-10.19:44:15.761>
created_at = <Date 2013-09-07.13:11:33.734>
labels = ['interpreter-core', 'type-bug']
title = 'First line can be executed twice'
updated_at = <Date 2014-09-05.07:26:25.101>
user = 'https://github.com/serhiy-storchaka'

bugs.python.org fields:

activity = <Date 2014-09-05.07:26:25.101>
actor = 'python-dev'
assignee = 'serhiy.storchaka'
closed = True
closed_date = <Date 2014-01-10.19:44:15.761>
closer = 'serhiy.storchaka'
components = ['Interpreter Core']
creation = <Date 2013-09-07.13:11:33.734>
creator = 'serhiy.storchaka'
dependencies = []
files = ['31644', '33304', '33307']
hgrepos = []
issue_num = 18960
keywords = ['patch']
message_count = 13.0
messages = ['197156', '197157', '197614', '197617', '197938', '207250', '207274', '207576', '207662', '207747', '207757', '207765', '226405']
nosy_count = 8.0
nosy_names = ['lemburg', 'loewis', 'terry.reedy', 'benjamin.peterson', 'ezio.melotti', 'Arfrever', 'python-dev', 'serhiy.storchaka']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue18960'
versions = ['Python 3.3', 'Python 3.4']

[serhiy-storchaka]

When run in Python 3 followed script:

print('first')
#coding=iso8859-1
print('second')

it prints the 'first' string twice.

Besides this weird behavior I think PEP-0263 should specify that if a magic comment is second line in the file then first line should be a comment too.

[ezio-melotti]

I think PEP-0263 should specify that if a magic comment is
second line in the file then first line should be a comment too.

+1

[terryjreedy]

I thought that that was the specification*: the encoding comment should be the first line unless it has to be the second line to allow for an executable comment. Executing a non-comment first line twice is definitely contrary to the intent of the PEP.

• I have not read it recently ;-).

When running that snippet from 3.3.2 Idle editor, 'first' is only printed once. I consider this to be correct and I would not want to change Idle to match the buggy behavior.

[malemburg]

On 13.09.2013 19:24, Terry J. Reedy wrote:

Terry J. Reedy added the comment:

I thought that that was the specification*: the encoding comment should be the first line unless it has to be the second line to allow for an executable comment. Executing a non-comment first line twice is definitely contrary to the intent of the PEP.

• I have not read it recently ;-).

When running that snippet from 3.3.2 Idle editor, 'first' is only printed once. I consider this to be correct and I would not want to change Idle to match the buggy behavior.

Agreed. The PEP never intended to allow executable first lines in
a Python code file.

[serhiy-storchaka]

It also makes non-working a common idiom for running Python files on Windows. When add first line "@python -x", it will be interpreted by Python and will cause SyntaxError.

[serhiy-storchaka]

Here is a patch which fixes this issue (and related issues).

• Encoding declaration now detected in second line only if first line is spaces-only (this is needed for support -x option) or comment-only (needed for she-bang).

• Fixed support for -x option.

• PyTokenizer_FromString() no longer check encoding declaration in second line if it was found in first line.

• Fixed encoding detection in the tokenize module, in IDLE, in 2to3 and in the findnocoding script.

[serhiy-storchaka]

Oh, I had not compiled the code after last change, so first patch is wrong. Here is fixed patch.

[serhiy-storchaka]

Benjamin, current version (without switch) looks more clean to me. Are you insist?

[benjaminp]

Yeah, I'm okay now.

[python-dev]

New changeset 1bdcaf6c0eb5 by Serhiy Storchaka in branch '3.3':
Issue bpo-18960: Fix bugs with Python source code encoding in the second line.
http://hg.python.org/cpython/rev/1bdcaf6c0eb5

New changeset 04c05e408cbd by Serhiy Storchaka in branch 'default':
Issue bpo-18960: Fix bugs with Python source code encoding in the second line.
http://hg.python.org/cpython/rev/04c05e408cbd

[serhiy-storchaka]

Now traceback test is failed.

http://buildbot.python.org/all/builders/AMD64%20Ubuntu%20LTS%203.3/builds/1358/steps/test/logs/stdio
======================================================================
FAIL: test_encoded_file (test.test_traceback.SyntaxTracebackCases)
----------------------------------------------------------------------

Traceback (most recent call last):
  File "/opt/python/3.3.langa-ubuntu/build/Lib/test/test_traceback.py", line 146, in test_encoded_file
    text, charset, 4)
  File "/opt/python/3.3.langa-ubuntu/build/Lib/test/test_traceback.py", line 129, in do_test
    stdout[1], lineno))
AssertionError: 'line 4' not found in '  File "@test_6376_tmp", line 3, in <module>' : Invalid line number: '  File "@test_6376_tmp", line 3, in <module>' instead of 4

---

[python-dev]

New changeset 875a514671dd by Serhiy Storchaka in branch '3.3':
Do not reset the line number because we already set file position to correct
http://hg.python.org/cpython/rev/875a514671dd

New changeset 2af308f79727 by Serhiy Storchaka in branch 'default':
Do not reset the line number because we already set file position to correct
http://hg.python.org/cpython/rev/2af308f79727