IDLE: catch user cfg file error, better error message, continue

[Tomk]

BPO	21973
Nosy	@terryjreedy, @roseman
Files	issue21973.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/terryjreedy'
closed_at = None
created_at = <Date 2014-07-13.12:01:45.314>
labels = ['expert-IDLE', 'type-bug', '3.9']
title = 'IDLE: catch user cfg file error, better error message, continue'
updated_at = <Date 2019-09-20.21:04:52.104>
user = 'https://bugs.python.org/Tomk'

bugs.python.org fields:

activity = <Date 2019-09-20.21:04:52.104>
actor = 'terry.reedy'
assignee = 'terry.reedy'
closed = False
closed_date = None
closer = None
components = ['IDLE']
creation = <Date 2014-07-13.12:01:45.314>
creator = 'Tomk'
dependencies = []
files = ['36257']
hgrepos = []
issue_num = 21973
keywords = ['patch']
message_count = 12.0
messages = ['222916', '222919', '222929', '222933', '222941', '222945', '222962', '224753', '224782', '224795', '224812', '280614']
nosy_count = 6.0
nosy_names = ['terry.reedy', 'SilentGhost', 'markroseman', 'jesstess', 'ingrid', 'Tomk']
pr_nums = []
priority = 'normal'
resolution = None
stage = 'test needed'
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue21973'
versions = ['Python 3.9']

[Tomk]

I am a retired programmer. I still like to write small programs for my own interest. Python is ideal for this. I prefer to use Idle rather than the command line. For some time now when I click on the Idle shortcut on the home screen, the command line screen flashes and then nothing. I am not able to do anything on Idle. I had been using Python33. So I upgraded to Python34. It was the same thing. I looked at the batch file, idle.bat. This is what it contains.

@echo off
rem Start IDLE using the appropriate Python interpreter
set CURRDIR=%~dp0
start "IDLE" "%CURRDIR%..\..\pythonw.exe" "%CURRDIR%idle.pyw" %1 %2 %3 %4 %5 %6 %7 %8 %9

I don't understand what %~dp0 means. I can't think what changes I have made to the operating system, it is Windows 7, that could cause this problem.

As I say Python is great for what I do. But I am stuck with the command line if I cannot fix this.

Tomk

[SilentGhost]

This is the normal content of idle.bat
Could you run c:\python34\python.exe -m idlelib.idle from the command-line and post the output here?

[Tomk]

I ran what you said. Here's what I got.

C:\Python34>c:\python34\python.exe -m idlelib.idle
Traceback (most recent call last):
  File "c:\python34\lib\runpy.py", line 170, in _run_module_as_main
    "__main__", mod_spec)
  File "c:\python34\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "c:\python34\lib\idlelib\idle.py", line 10, in <module>
    import idlelib.PyShell
  File "c:\python34\lib\idlelib\PyShell.py", line 29, in <module>
    from idlelib.EditorWindow import EditorWindow, fixwordbreaks
  File "c:\python34\lib\idlelib\EditorWindow.py", line 22, in <module>
    from idlelib.configHandler import idleConf
  File "c:\python34\lib\idlelib\configHandler.py", line 703, in <module>
    idleConf=IdleConf()
  File "c:\python34\lib\idlelib\configHandler.py", line 171, in __init__
    self.LoadCfgFiles()
  File "c:\python34\lib\idlelib\configHandler.py", line 694, in LoadCfgFiles
    self.userCfg[key].Load() #same keys
  File "c:\python34\lib\idlelib\configHandler.py", line 68, in Load
    self.read(self.file)
  File "c:\python34\lib\configparser.py", line 672, in read
    self._read(fp, filename)
  File "c:\python34\lib\configparser.py", line 1058, in _read
    raise MissingSectionHeaderError(fpname, lineno, line)
configparser.MissingSectionHeaderError: File contains no section headers.
file: 'C:\\Users\\Tomk\\.idlerc\\config-extensions.cfg', line: 1
'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x0
0\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'

---

From: SilentGhost <report@bugs.python.org>
To: t.kember1@btinternet.com
Sent: Sunday, 13 July 2014, 13:15
Subject: [bpo-21973] Can't use Idle

SilentGhost added the comment:

This is the normal content of idle.bat
Could you run c:\python34\python.exe -m idlelib.idle from the command-line and post the output here?

----------
components: +IDLE
nosy: +SilentGhost, kbk, roger.serwy, terry.reedy

---

Python tracker <report@bugs.python.org>
<http://bugs.python.org/issue21973\>

---

[SilentGhost]

If you would delete this file: C:\Users\Tomk\.idlerc\config-extensions.cfg (back it up some place else just in case). Does it start? and if not, would you mind re-running the same command again?

[Tomk]

Thanks, I backed that file on to a memory stick and deleted it. I made a shortcut to the idle batch file on the desktop. When I click on it, Idle comes up fine.
What has happened here? What is config-extensions.cfg for and why has it stopped Idle coming up? And  why can Idle work without it? In any case, I am now able to work on my little programs.
Thank you again.
Tomk

---

From: SilentGhost <report@bugs.python.org>
To: t.kember1@btinternet.com made a
Sent: Sunday, 13 July 2014, 15:30
Subject: [bpo-21973] Can't use Idle

SilentGhost added the comment:

If you would delete this file: C:\Users\Tomk\.idlerc\config-extensions.cfg (back it up some place else just in case). Does it start? and if not, would you mind re-running the same command again?

----------

---

Python tracker <report@bugs.python.org>
<http://bugs.python.org/issue21973\>

---

[SilentGhost]

IDLE supports extensions: https://docs.python.org/3/library/idle.html#extensions
And that was a user-configuration file that got corrupted somehow. As it's an optional feature, IDLE can work without it.

You can use your favourite text editor to compose programs and then use console to run them. It might be more convenient even for smaller programs.

[terryjreedy]

Explanation: there are a few config-xy files in lib/idlelib that contain default configuration values. Custom values for a particular user, which override the defaults, are kept in a .idlerc directory placed in a users home directory (here C:/users/Tomk). Config-main and config-keys are edited by Idle in response to entries in the preferences/options dialog. Any can be edited by hand.

Since Idle can work without these files, I consider it a bug that it quits when there is a bad one. Instead of raising an uncaught exception, it should display a warning message and continue*. It could also offer to delete or rename the file so the message does not reappear the next time Idle is started.

• (Note to myself) This could be done either by replacing the raise statement or by catching the exception further up the call chain, where other errors could be caught too, and all turned into warnings.

We are actively working on improving the error handling for config-keys, but I don't know if there is an issue for this particular error. So I am re-opening this issue until this problem is fixed or I know that this is a duplicate report.

Thomas, thank you for the report, and in particular for the full traceback. when you reply by email, please delete the message you are responding to except possibly for a line or two that you need to quote. When viewed on the site, your response is placed just beneath the previous one, so bulk quoting is redundant.

[ingrid]

Here is a patch that will print a warning if there is an invalid config when starting idle. Example:

Warning (from warnings module):
File "/Users/ingrid/.idlerc/config-extensions.cfg", line 1
enable=1
MissingSectionHeaderError: File contains no section headers.
Ignoring configurations within config-extensions.cfg.
Delete this file or fix formatting errors to remove this warning.

[terryjreedy]

Some background context before I comment on the patch: Printing warnings to a console is a flawed idea. It does not work on Windows when Idle is run with pythonw and there is no console to write to. This is the normal case on Windows and, I suspect, the majority of all Idle uses across all systems. The attempt to write (to sys.__stderr__ == None) raises OSError, which shuts Idle down unless caught. (Thomas experiences exactly this when Python attempted to write the traceback.) Either way, there is no message to be seen. Even when writing to console is successful, the message will often be hidden by Idle windows. Maybe it will be seen later, but possibly not.

Idle is a gui program and should not assume a text console. We should use tk messages boxes instead. Or perhaps put warnings in an Output Window (either instead, or in addition). Or put an error.log in .idlerc/ (this would be 'in addition'. Or also write to console when there is one, but not as the only message delivery system.

For problems in config files, I think we should, if possible, fix, rename, or delete the file. In any case, make it so the same warning will not appear next time Idle it is started. If we do anything, the warning message should say what was done.

There are 6 places in configHandler with code like the following:
if <config problem>:
# do default, print warning
warning=...
try:
sys.stderr.write(warning)
except OSError:
pass

(In one of the 6, Idle exits instead of catching OSError; there is another issue to fix this.) These blocks mix together situation-specific message creation with a common and flawed delivery system. The latter should be factored out and fixed in a separate function at the top of the file (which I can write a first version of).

def config_warn(message): ... 

The problem with the patch is that it does not follow existing configHandler code. First, it stores errors and shuffles them off to be dealt with in PyShell. They should instead be immediately dealt with where detected. Second, it uses idle_showwarning to format and emit the warning.  Formatting with idle_formatwarning is ok, but idle_showwarning is designed specifically for the shell, ending warnings with the shell prompt '>>> '. It also has the flaw of the code above.  By default, it sends messages to an often non_existent sys.__stderr__. The patch should use the new config_warn function.

I will look at the new tests later.

When GSOC is done, I want to look as Tal's extension config dialog. That should reduce errors from users hand-editing config-extensions. Saimadhav has also done some work on key validation that needs to be applied to configHandler and configDialog. That should reduce key errors a bit.

[ingrid]

Thanks for your feedback. I can try again when you have written the config warn function or I can try writing the config warn function. The tests aren't relevant anymore if the warning is raised in a tk message box. I think deleting the config files could potentially be frustrating for users, and I'm not sure about fixing them either. I think renaming them would be safer and easier, what would you rename them to?

[terryjreedy]

config-xyz.bad?

I have not look at the tests, but they should be relevant, just in need of changing to using mock message boxes from idle_test/mock_tk.py.

[terryjreedy]

I closed bpo-28658 in favor of this. For that issue, the exception was a UnicodeDecodeError instead of a configparser.Error. Both should be caught.

Ingrid, sorry I never got back to this. A few months ago, Serhiy wrote a warning function for the config(Handler) function that was included in a larger patch for the module. I wrote a test_config that test both the warning function and the other new code. This was for 3.6 only.

I am going to patch the lowest level IDLE function, IDLEConfigParser.Load, which currently calls ConfigParser.read, and catch exceptions and call the warning function right there. For 2.7 and 3.5, I will make a minimal change, and test by hand (by editing errors into one of my user config files). For 3.6/7, I will try replacing .read (which reads a list of files) with .read_file (which reads one file).