Invalid coding error hidden on Windows

[ubershmekel]

BPO	36800
Nosy	@pfmoore, @tjguk, @zware, @eryksun, @zooba
Superseder	bpo-20844: SyntaxError: encoding problem: iso-8859-1 on Windows
Files	log_monitor.py: Reproducing exception

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2019-05-05.18:32:02.466>
created_at = <Date 2019-05-05.06:32:07.041>
labels = ['type-bug', '3.7', 'OS-windows']
title = 'Invalid coding error hidden on Windows'
updated_at = <Date 2019-05-06.07:34:24.149>
user = 'https://bugs.python.org/ubershmekel'

bugs.python.org fields:

activity = <Date 2019-05-06.07:34:24.149>
actor = 'ubershmekel'
assignee = 'none'
closed = True
closed_date = <Date 2019-05-05.18:32:02.466>
closer = 'eryksun'
components = ['Windows']
creation = <Date 2019-05-05.06:32:07.041>
creator = 'ubershmekel'
dependencies = []
files = ['48300']
hgrepos = []
issue_num = 36800
keywords = []
message_count = 5.0
messages = ['341420', '341421', '341459', '341468', '341488']
nosy_count = 6.0
nosy_names = ['paul.moore', 'tim.golden', 'ubershmekel', 'zach.ware', 'eryksun', 'steve.dower']
pr_nums = []
priority = 'normal'
resolution = 'duplicate'
stage = 'resolved'
status = 'closed'
superseder = '20844'
type = 'behavior'
url = 'https://bugs.python.org/issue36800'
versions = ['Python 3.7']

[ubershmekel]

These lines fail on Windows in a surprising way:

    # -*- coding: utf8 -*-
    import threading
    print("threading %s" % threading)

Normally they would throw this:

SyntaxError: encoding problem: utf8```

But attached is a file that throws this instead:

Traceback (most recent call last):
File "C:\Python36\Lib\site-packages\missinglink_kernel\callback\log_monitor.py", line 2, in
import threading
NameError: name 'threading' is not defined

It seems that the amount of lines in the file will cause the exception hiding bug to manifest.

This issue did reproduce on my Windows machine here:

* Python 3.6.4 (v3.6.4:d48eceb, Dec 19 2017, 06:54:40) [MSC v.1900 64 bit (AMD64)] on win32
* Python 3.7.1 (default, Dec 10 2018, 22:54:23) [MSC v.1915 64 bit (AMD64)] :: Anaconda, Inc. on win32
* Python 3.7.3 (v3.7.3:ef4ec6ed12, Mar 25 2019, 21:26:53) [MSC v.1916 32 bit (Intel)] on win32

It did NOT reproduce on my Mac at all:

It does not repro on my mac at all:
* Python 3.7.1 (default, Dec 14 2018, 13:28:58)  [Clang 4.0.1 (tags/RELEASE_401/final)] :: Anaconda, Inc. on darwin
* Python 3.6.5 |Anaconda, Inc.| (default, Apr 26 2018, 08:42:37) [GCC 4.2.1 Compatible Clang 4.0.1 (tags/RELEASE_401/final)] on darwin

[ubershmekel]

Note I am aware the actual problem is "utf8" vs "utf-8". But for some reason on Windows the exception does not reflect that.

[tjguk]

Python 3.6.6 (v3.6.6:4cf1f54eb7, Jun 27 2018, 03:37:03) [MSC v.1900 64 bit (AMD64)] on win32

I can't reproduce this on Windows. And it does seem an unlikely combination of effect and cause.

Do I take it that if you take out all the "logging..." lines, the thing works? ie you do correctly import threading and it shows a reasonable value when you print(threading)?

[eryksun]

Note I am aware the actual problem is "utf8" vs "utf-8".

That shouldn't be an issue since "utf8" is an alias.

The fact that your test file has LF line endings is an issue. It should be fixed in 3.7.4. See bpo-20844.

[ubershmekel]

Replacing utf8 with utf-8 is the best workaround in my situation. Using the utf8 alias is required to reproduce the issue based on my testing and the cited issue (https://bugs.python.org/issue20844).

Thank you for the reference.