Idlelib: changing file names

[AlSweigart]

BPO	24225
Nosy	@terryjreedy, @kbkaiser, @ncoghlan, @ned-deily, @serwy, @roseman, @serhiy-storchaka
Files	idle_renaming.patch idletest_renaming.patch idle_updating_imports.patch @newnames.txt i3rename.bat

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/terryjreedy'
closed_at = <Date 2016-06-01.00:24:38.914>
created_at = <Date 2015-05-18.07:36:27.540>
labels = ['expert-IDLE', 'type-feature']
title = 'Idlelib: changing file names'
updated_at = <Date 2016-06-04.05:20:28.435>
user = 'https://bugs.python.org/AlSweigart'

bugs.python.org fields:

activity = <Date 2016-06-04.05:20:28.435>
actor = 'ned.deily'
assignee = 'terry.reedy'
closed = True
closed_date = <Date 2016-06-01.00:24:38.914>
closer = 'terry.reedy'
components = ['IDLE']
creation = <Date 2015-05-18.07:36:27.540>
creator = 'Al.Sweigart'
dependencies = []
files = ['39427', '39428', '39429', '42678', '42951']
hgrepos = []
issue_num = 24225
keywords = ['patch']
message_count = 34.0
messages = ['243464', '243570', '243571', '243572', '243573', '243612', '243613', '243622', '243623', '243635', '250451', '264620', '264622', '264623', '264625', '265255', '265545', '265585', '266116', '266213', '266215', '266220', '266392', '266395', '266409', '266412', '266571', '266572', '266573', '266792', '266794', '267211', '267229', '267232']
nosy_count = 9.0
nosy_names = ['terry.reedy', 'kbk', 'ncoghlan', 'ned.deily', 'roger.serwy', 'markroseman', 'python-dev', 'Al.Sweigart', 'serhiy.storchaka']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue24225'
versions = ['Python 3.6']

[AlSweigart]

From the idlelib/idle_test/README.txt file:

"The idle_test subdirectory should contain a test_xyz.py for each, where 'xyz' is lowercased even if xyz.py is not."

However, this means the test file names are inconsistent with the scripts they test. Of the 20 test_*.py files, all 20 are inconsistent. Some of the test filenames don't even match the file, such as test_config_name.py which tests configSectionNameDialog.py.

Since the original filenames aren't lowercase like the rest of the standard library, it would be better to rename them rather than renaming the test files: AutoComplete.py to autocomplete.py, ColorDelegator.py to colordelegator.py, etc. This is a find-and-replace change. Any shorter, better names for modules can also be made at this time.

[AlSweigart]

The idle_renaming.patch file is for the file renamings under idlelib

The idletest_renaming.patch file is for the file renamings under idlelib/idle_test

The idle_updating_imports.patch file is for the content updates (mostly for import statements) due to the renamings.

The unit tests all pass with these changes.

[AlSweigart]

(I can reformat these patch files if that helps.)

[AlSweigart]

For reference, here's all the files and their new filenames:

aboutDialog.py about_dialog.py
AutoComplete.py autocomplete.py
AutoCompleteWindow.py autocomplete_window.py
AutoExpand.py autoexpand.py
Bindings.py bindings.py
CallTips.py calltips.py
CallTipWindow.py calltip_window.py
ClassBrowser.py class_browser.py
CodeContext.py code_context.py
ColorDelegator.py color_delegator.py
configDialog.py config_dialog.py
configHandler.py config_handler.py
configHelpSourceEdit.py config_help_source_edit.py
configSectionNameDialog.py config_section_name_dialog.py
Debugger.py debugger.py
Delegator.py delegator.py
dynOptionMenuWidget.py dyn_option_menu_widget.py
EditorWindow.py editor_window.py
FileList.py file_list.py
FormatParagraph.py format_paragraph.py
GrepDialog.py grep_dialog.py
HyperParser.py hyper_parser.py
IdleHistory.py idle_history.py
idlever.py idle_ver.py
IOBinding.py io_binding.py
keybindingDialog.py keybinding_dialog.py
macosxSupport.py mac_osx_support.py
MultiCall.py multi_call.py
MultiStatusBar.py multi_status_bar.py
ObjectBrowser.py object_browser.py
OutputWindow.py output_window.py
ParenMatch.py paren_match.py
PathBrowser.py path_browser.py
Percolator.py percolator.py
PyParse.py pyparse.py
PyShell.py pyshell.py
RemoteDebugger.py remote_debugger.py
RemoteObjectBrowser.py remote_object_browser.py
ReplaceDialog.py replace_dialog.py
RstripExtension.py rstrip_extension.py
ScriptBinding.py script_binding.py
ScrolledList.py scrolled_list.py
SearchDialog.py search_dialog.py
SearchDialogBase.py search_dialog_base.py
SearchEngine.py search_engine.py
StackViewer.py stack_viewer.py
tabbedpages.py tabbed_pages.py
textView.py text_view.py
ToolTip.py tool_tip.py
TreeWidget.py tree_widget.py
UndoDelegator.py undo_delegator.py
WidgetRedirector.py widget_redirector.py
WindowList.py window_list.py
ZoomHeight.py zoom_height.py
test_config_name.py test_config_section_name_dialog.py
test_configdialog.py test_config_dialog.py
test_formatparagraph.py test_format_paragraph.py
test_grep.py test_grep_dialog.py
test_hyperparser.py test_hyper_parser.py
test_idlehistory.py test_idle_history.py
test_parenmatch.py test_paren_match.py
test_pathbrowser.py test_path_browser.py
test_rstrip.py test_rstrip_extension.py
test_searchdialogbase.py test_search_dialog_base.py
test_textview.py test_text_view.py
test_searchengine.py test_search_engine.py
test_widgetredir.py test_widget_redirector.py

[serhiy-storchaka]

Usually we avoid such mass renamings. This is errorprone and can break third-party plugins or even applications that use parts of IDLE. It also will made much harder to transfer patches between versions. I'm -1 for this proposition.

[AlSweigart]

Serhiy: I completely understand. My first instinct is to look at mass renamings and dismiss it as cosmetic changes for the sake of cosmetic changes. However, there are several problems that this fix corrects:

1. It makes it obvious which identifiers are modules and which are classes:

• AutoComplete.AutoComplete is a module and class with the same name, making them ambiguous.
• Same for AutoCompleteWindow.AutoCompleteWindow.
• Same for AutoExpand.AutoExpand.
• Bindings.py is a module but there is also a Bindings member in the EditorWindow class.
• Same for CallTips.CallTips having ambiguous module/class name.
• Same for ClassBrowser.ClassBrowser.
• Same for CodeContext.CodeContext.
• Same for ColorDelegator.ColorDelegator
• But not the same for configDialog.ConfigDialog.
• Same for Debugger.Debugger
• Same for Delegator.Delegator
• Same for EditorWindow (this is a big one, where the ambiguious use is in several files)
• Same for FileList.FileList
• Same for FormatParagraph.FormatParagraph
• Same for HyperParser.HyperParser
• Same for IOBinding.IOBinding
• Same for MultiCall.MultiCallCreator's inner class, MultiCall.
• Same for MultiStatusBar.MultiStatusBar
• Same for OutputWindow, ParenMatch, PathBrowser, Percolator, PyShell, ReplaceDialog, RstripExtension, ScriptBinding, ScrolledList, SearchDialog, SearchDialogBase, SearchEngine, ToolTip, UndoDelegator, WidgetRedirector, WindowList, ZoomHeight
• There is a StackViewer.py module and a StackViewer class, except the class is in Debugger.py

This isn't a trivial, "just remember it works like this in one or two places" kind of technical debt.

2. These ambiguous names can be mitigated with "from x import *" statements; always use "from idlelib.AutoExpand import AutoExpand" and you can always assume "AutoExpand" refers to the class, not the module.

But the codebase doesn't consistently do that. There is "from idlelib.AutoExpand import AutoExpand" but also "from idlelib import Debugger" (so in that file, Debugger refers to the module while Debugger.Debugger refers to the class).

2. It puts idle_test names in sync with the files they test. Currently there is a test_config_name.py but there is no config_name.py or configName.py file. (It tests configSectionNameDialog.py)

3. Any extensions that break due to these changes can be fixed in one line: import new_name as OldName

I agree that large changes are error prone. In this case, that's partly because names are confusing and ambiguous, which is precisely what this change tries to mitigate.

I really beg folks to not dismiss this change out of hand. These readability issues are death-by-a-thousand-paper-cuts. I think we should address this technical debt, not actively preserve it. IDLE dev is at a standstill, with the bulk of it resting on the efforts of a handful of people. Getting it into a better state can invite more contributors (I can personally name some people who would want to become involved with IDLE but are intimidated by labyrinthine codebases.)

I volunteer to test any extension people can name for compatibility issues with this change. (Where should I start? IDLEX? The https://wiki.python.org/moin/IDLEextensions also lists lwickjr. Are there any others?)

All the unit tests pass after these changes, which is something at least.

[terryjreedy]

The idlelib xyz.py filenames (about 60) are a collectively a mess. There are about 6 different styles. I would like to *eventually* change all to short, lowercase, PEP-8 names, but the reasons have nothing to do with test names (which intentionally look to the future).

1. The variety of styles and length of some makes some hard to learn and remember. This discourages new people from working on Idle code. After years, I still find some a problem. For this purpose, the patch does not go far enough. Merely changing 'X' to '_x' does not make a long, multi word or word pieces name into a short, one-word name. For examples: config_section_name_dialog.py and dyn_option_menu_widget.py are not where I want to end up.

2. Having a module and a class with the same name is confusing. Consider 'EditorWindow'. In text, having to disambiguate by appending 'module' or 'class' is a nuisance. In code, even that is not possible. One must jump up to the top of a file to see if the import is 'import EditorWindow' or 'from EditorWindow import EditorWindow'.

3. A deeper reason I want name changes to be possible is that I would also like to be able to do some package-level refactoring. 60 files is already a lot and we may need more. I think AutoComplete.py and AutoCompleteWindow.py should be combined into completions.py. I would combine CallTips.py and CallTipWindow.py (I had to look to be sure it was not CallTipsWindow.py) into calltips.py.

In the other hand, PyShell.py has startup code (which I think might better be in a separate startup.py), some code for debugger and editor windows, and finally shell window code. I think the confusing Window class hierarchy should be redone, which would also mean some file reshuffling and renaming.

4. PEP-434 suggested that a few generally useful, non-Idle specific files might be moved to the tkinter package. This would also be a name change.

One problem with name changes is continued support for 2.7, which was originally scheduled to end about now. If it were not for that, it would more easily be possible to introduce new names in some 3.x.0, possible after deprecation warnings.

Another barrier is the unknown usage by extensions of parts of idlelib
beyond the narrow interface defined in extend.txt: an EditorWindow instance with Test, IOBinding, and FileList instances.

Summary: I agree with the goal. But I have to reject the current patch because a) the change will have to be a process, not a single patch, and b) I do not agree with many of the new names.

[terryjreedy]

Nick, as PEP-434 approver, I would like your comments on this issue.

The pep says that other than the startup modules, "the modules are undocumented and effectively private implementations." It also says that the documented extension API should be respected but is vague about the extent to which the existence of extensions should otherwise be reasons to block changes to idlelib.

While this issue is about module names, changes to names and behavior within modules could also break extensions. We have in fact made such changes without worring about this.

Fedora 22 replaced yum with dnf as its package managater. Accordig to
http://dnf.baseurl.org/2015/05/11/yum-is-dead-long-live-dnf/
one of the reasons is problems with extensions. "Yum plugins are using any method from Yum code base thus any change there would cause the sudden crash of the Yum utility." (What yum and idlelib have is common is mostly being undocumented.) In a comment, the author summed up my feelings with respect to idlelib: "It really hard to maintain a project where you can’t change anything."

An example is bpo-24199. I want to remove a zombie module, idlever. Can I? At least for 2.7 and 3.4, I think I will add a deprecation warning.

If we rename and refactor modules for 3.6, I would propose to add stub files to 2.7 and 3.5 with imports and a deprecation warning. A combined completions module, for example, would get two stub files. (An alternative for 2.7 could be to fix the worst bugs, declare Idle 2.7 frozen, and stop backporting.)

During the PEP-434 discussion, some core developers said that they would like to drop Idle completely. This would break extensions much more thoroughly than changes proposed here.

Al: some research into extension practices would be helpful. But keep in mind that idlex is much more than a set of extension modules. I believe it adds its own ui.

Al: the current discrepancies between module and test names are intentional. The 'controversy' is entirely about idlelib, hence the title change. When idlelib modules get 'nice' names, test names can be changed to match.

[terryjreedy]

Al, I meant to add that msg243612 better expresses the proper rationale for disruptive changes: making it easier for people (newcomers and this no-longer-newcomer) to learn idlelib, fix bugs, and make improvements

[ncoghlan]

One path you may want to consider is progressively factoring out a public "idlelib.extensions" API, and treat everything else in idlelib as fair game for renaming. Folks tend to be more tolerant of disruption if the new state of affairs is also clearly better for them, not just the folks making the change.

In terms of supporting legacy module names without retaining all that complexity in the main source code, I'd suggest looking into the way the "six.moves" module works.

[terryjreedy]

A new factor has entered the picture: the opportunity to move dialogs from using tk widgets to ttk widgets. However, we cannot yet drop support for systems that do not have ttk widgets. We could accommodate using both sets of dialog widgets by adding lots of conditional code within each file as needed, but that would only solve the ttk or not issue.

I decided instead to have separate files. This way, the ttk files can have PEP-8 names and style and refactorings. New files will be declared private in docstrings. When a new file is written, corresponding old files, if imported (by an extension or non-idle program) when ttk *is* being used , will raise a Deprecation Warning. The same idea will be used for the text window files for shell, edit, and output, which badly need refactoring both as they are and to allow tabbed windows with multiple files.

Overall, the plan is to gradually add new files to all current versions (except, probably, 3.4) and delete old files in the future (possibly in 3.6, but not decided now). The transition overlap will be messier than a simple rename, but will avoid the problems of patch merges and immediately breaking 3rd-party code. And the result will be much more and much better than a simple rename.