IDLE: edit/save files created by Windows Explorer

[terryjreedy]

BPO	41373
Nosy	@rhettinger, @terryjreedy, @serhiy-storchaka, @eamanu, @miss-islington
PRs	bpo-41373: IDLE: Fix saving files loaded with no newlines or mixed newlines #21597 [3.9] bpo-41373: IDLE: Fix saving files loaded with no newlines or mixed newlines (GH-21597) #21610 [3.8] bpo-41373: IDLE: Fix saving files loaded with no newlines or mixed newlines (GH-21597) #21611

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2020-07-26.07:22:38.249>
created_at = <Date 2020-07-23.03:47:16.562>
labels = ['type-bug', '3.8', '3.9', '3.10']
title = 'IDLE: edit/save files created by Windows Explorer'
updated_at = <Date 2020-07-26.07:22:38.249>
user = 'https://github.com/terryjreedy'

bugs.python.org fields:

activity = <Date 2020-07-26.07:22:38.249>
actor = 'serhiy.storchaka'
assignee = 'none'
closed = True
closed_date = <Date 2020-07-26.07:22:38.249>
closer = 'serhiy.storchaka'
components = []
creation = <Date 2020-07-23.03:47:16.562>
creator = 'terry.reedy'
dependencies = []
files = []
hgrepos = []
issue_num = 41373
keywords = ['patch']
message_count = 8.0
messages = ['374119', '374121', '374127', '374164', '374165', '374227', '374228', '374229']
nosy_count = 5.0
nosy_names = ['rhettinger', 'terry.reedy', 'serhiy.storchaka', 'eamanu', 'miss-islington']
pr_nums = ['21597', '21610', '21611']
priority = 'high'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue41373'
versions = ['Python 3.8', 'Python 3.9', 'Python 3.10']

[terryjreedy]

bpo-41300 fixed one bug in the patch for bpo-41158. A user reported another on on idle-dev list. Create a file in Windows Explorer. Leave as .txt or rename to .py. Right click and Edit with IDLE 3.8 (.4/.5 or 3.9.0b4 or 5). Edit works, Save (or Run) does not. Open the same file from Command Prompt, change it, and trying to save produces

Exception in Tkinter callback
Traceback (most recent call last):
  File "C:\Programs\Python38\lib\tkinter\__init__.py", line 1883, in __call__
    return self.func(*args)
  File "C:\Programs\Python38\lib\idlelib\multicall.py", line 176, in handler
    r = l[i](event)
  File "C:\Programs\Python38\lib\idlelib\iomenu.py", line 200, in save
    if self.writefile(self.filename):
  File "C:\Programs\Python38\lib\idlelib\iomenu.py", line 232, in writefile
    text = self.fixnewlines()
  File "C:\Programs\Python38\lib\idlelib\iomenu.py", line 252, in fixnewlines
    text = text.replace("\n", self.eol_convention)
TypeError: replace() argument 2 must be str, not None

The replacement line is guarded with
if self.eol_convention != "\n":
Either the condition should be include 'and self.eol_convention is not None' or the setting of the attribute should be changed. I will look into how it was set before the bpo-41158 patch.

[serhiy-storchaka]

I missed that the file's attribute newlines can be None if no line separators were read. It can also be a tuple if mixed newlines are used. I think that eol_convention should be set to the default value os.linesep in these cases.

I am currently not able to test this since IDLE does not work on my computer after upgrading to Ubuntu 20.04.

[terryjreedy]

So loading an externally created blank file (on any system) results in None. File=>New must set eol_convention, I presume to the same default. Before I got the traceback, I though it might be an issue with the Windows utf-8 BOM being added.

3.9.0rc1 is due August 10. It should have fix and new tests.

[rhettinger]

Bumping-up the priority. Today, I and a (virtual) room full of engineers simultaneously hit this problem and we all lost work. See bpo-41378

[terryjreedy]

Raymond, I tested patch today and expect to apply it (without unittests yet) tomorrow, so will be in 3.9.0rc1 abd 3.8.6. In the meanwhile, either create new files within IDLE (File=>NEW) or add "isinstance(self.eol_convention, str) and " after 'if' on line 251 of idlelib/iomenu.py, just before the line in the traceback.

The added check will be put elsewhere by the patch, but it should work on 251.

[terryjreedy]

New changeset 0dd463c by Serhiy Storchaka in branch 'master':
bpo-41373: IDLE: Fix saving files loaded with no newlines or mixed newlines (GH-21597)
0dd463c

[miss-islington]

New changeset 1c562ce by Miss Islington (bot) in branch '3.8':
bpo-41373: IDLE: Fix saving files loaded with no newlines or mixed newlines (GH-21597)
1c562ce

[miss-islington]

New changeset 15fdbb7 by Miss Islington (bot) in branch '3.9':
bpo-41373: IDLE: Fix saving files loaded with no newlines or mixed newlines (GH-21597)
15fdbb7