Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make IDLE recognize .pyi stub files as source for open, save, and edit #89610

Open
AlexWaygood opened this issue Oct 12, 2021 · 28 comments
Open

Make IDLE recognize .pyi stub files as source for open, save, and edit #89610

AlexWaygood opened this issue Oct 12, 2021 · 28 comments
Assignees
Labels
3.11 expert-IDLE type-feature

Comments

@AlexWaygood
Copy link
Member

@AlexWaygood AlexWaygood commented Oct 12, 2021

BPO 45447
Nosy @terryjreedy, @ronaldoussoren, @zooba, @miss-islington, @E-Paine, @AlexWaygood
PRs
  • #28950
  • #31303
  • #31304
  • #31305
  • #31306
  • #31307
  • #31308
  • #31309
  • #31310
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = 'https://github.com/terryjreedy'
    closed_at = None
    created_at = <Date 2021-10-12.15:24:36.540>
    labels = ['expert-IDLE', 'type-feature', '3.11']
    title = 'Make IDLE recognize .pyi stub files as source for open, save, and edit'
    updated_at = <Date 2022-02-14.03:35:10.254>
    user = 'https://github.com/AlexWaygood'

    bugs.python.org fields:

    activity = <Date 2022-02-14.03:35:10.254>
    actor = 'terry.reedy'
    assignee = 'terry.reedy'
    closed = False
    closed_date = None
    closer = None
    components = ['IDLE']
    creation = <Date 2021-10-12.15:24:36.540>
    creator = 'AlexWaygood'
    dependencies = []
    files = []
    hgrepos = []
    issue_num = 45447
    keywords = ['patch']
    message_count = 28.0
    messages = ['403738', '403741', '403755', '403772', '403884', '403889', '403915', '403998', '404021', '404023', '404030', '404100', '404102', '404110', '410637', '410669', '412561', '412617', '413147', '413149', '413150', '413156', '413157', '413158', '413168', '413169', '413170', '413211']
    nosy_count = 6.0
    nosy_names = ['terry.reedy', 'ronaldoussoren', 'steve.dower', 'miss-islington', 'epaine', 'AlexWaygood']
    pr_nums = ['28950', '31303', '31304', '31305', '31306', '31307', '31308', '31309', '31310']
    priority = 'normal'
    resolution = None
    stage = 'patch review'
    status = 'open'
    superseder = None
    type = 'enhancement'
    url = 'https://bugs.python.org/issue45447'
    versions = ['Python 3.11']

    @AlexWaygood
    Copy link
    Member Author

    @AlexWaygood AlexWaygood commented Oct 12, 2021

    IDLE currently does not do any syntax highlighting if you use it to open a .pyi stub file, even though everything in a .pyi file is a valid Python expression. It would be great if syntax highlighting for .pyi files could be supported.

    @AlexWaygood AlexWaygood added expert-IDLE type-feature labels Oct 12, 2021
    @AlexWaygood AlexWaygood added expert-IDLE type-feature labels Oct 12, 2021
    @AlexWaygood AlexWaygood changed the title Support syntax highlighting for .pyi stub files IDLE: Support syntax highlighting for .pyi stub files Oct 12, 2021
    @AlexWaygood AlexWaygood changed the title Support syntax highlighting for .pyi stub files IDLE: Support syntax highlighting for .pyi stub files Oct 12, 2021
    @terryjreedy
    Copy link
    Member

    @terryjreedy terryjreedy commented Oct 12, 2021

    It should be fairly easy to recognize x.pyi as a python file. Just have to find the place. Something like
    def f(i:int) -> int: ...
    would have 'def' and 'int' highlighted.

    @AlexWaygood
    Copy link
    Member Author

    @AlexWaygood AlexWaygood commented Oct 12, 2021

    It looks like support for .py files is hardcoded into IDLE's source code in the following places:

    • browser.ModuleBrowserTreeItem.OnDoubleClick
    • browser.ModuleBrowserTreeItem.IsExpandable
    • browser.ModuleBrowserTreeItem.listchildren
    • editor.ispythonsource << This is the one that controls syntax highlighting
    • iomenu.IOBinding.filetypes

    If I were to file a PR adding support for .pyi files, would you prefer that I hardcoded ".pyi" in, or would it be better to add a SUPPORTED_FILES constant somewhere?

    @terryjreedy
    Copy link
    Member

    @terryjreedy terryjreedy commented Oct 13, 2021

    Thank you for doing the research.

    I have been thinking about adding a file containing idlelib 'leaf' objects, those with no idlelib dependencies, which are needed in more than one file. would be to reduce the complexity of the idlelib dependency graph, which has enough cyclic dependencies to make startup tricky. Time to do it. idlelib/common.py?, leaves.py?, or ??? Draft:
    ---
    """Idlelib objects with no external idlelib dependencies and which are needed in more than one idlelib module. They are included here because a) they don't particularly belong elsewhere or b) because inclusion here simplifies the idlelib dependency graph.

    TODO: Python versions (editor and help_about), tk version and patchlevel (pyshell, help_about, maxos?, editor?), std streams (pyshell, run), warning stuff (ditto).
    """

    # python_extensions is used in editor, browser, and iomenu.
    # .pyw is for Windows; .pyi is for stub files.
    python_extensions = ('.py', '.pyw', '.pyi')  
    extension_string = "*" + " *".join(python_extensions)

    editor.EditorWindow.ispythonsource could be moved into the new file if a 'firstline=None' argument were added.

    OnDoubleClick should use IsExpandable. IsExpandable should use the logic in ispythonsource so Linux no-extension python source can be browsed as well as edited.

    After manual testing, add a new test_common.py file in idlelib.idle_test. After imports, it could start with one test:

    class ExtensionTest(TestCase):
        def test_stub(self):
            self.assertIn('.pyi', common.python_extensions)
            self.assertIn('.pyi', common.extension_string)

    For IDLE, I can and will backport.

    @terryjreedy
    Copy link
    Member

    @terryjreedy terryjreedy commented Oct 14, 2021

    I am going to call the file util.py and will work on a PR.

    @AlexWaygood
    Copy link
    Member Author

    @AlexWaygood AlexWaygood commented Oct 14, 2021

    Oh — I've already nearly finished a PR. Do you mind if I submit mine? Happy to make any changes you like, obviously.

    @AlexWaygood
    Copy link
    Member Author

    @AlexWaygood AlexWaygood commented Oct 14, 2021

    Have posted my PR. I've tested it manually and run the IDLE test suite on it with the -u all setting; it seems fine. The new tests I've written are a bit thin on the ground right now, though. I may need some guidance/help if more extensive tests are needed -- I'm a little new to unittest.

    One other thing I'm not sure on: the docstring at the top of browser.py includes the line "- finish removing limitation to x.py files (ModuleBrowserTreeItem)" in the TODO items. Can this be removed now?

    @E-Paine
    Copy link
    Mannequin

    @E-Paine E-Paine mannequin commented Oct 15, 2021

    Would it make sense, since this issue pretty much makes IDLE officially support .pyi files, to add the "Edit with IDLE" menu when right-clicking on a .pyi file in Windows explorer?

    @terryjreedy
    Copy link
    Member

    @terryjreedy terryjreedy commented Oct 15, 2021

    Yes. After this issue is done, we should open a new issue and nosy Steve Dower, who is in charge of that for Windows, and Ned Deily, in case anything should be done on mac.

    @ronaldoussoren
    Copy link
    Contributor

    @ronaldoussoren ronaldoussoren commented Oct 15, 2021

    The most straightforward change for macOS is to add “pyi” to the list of extensions in “Mac/IDLE/IDLE.app/Contents/Info.plist”. That way IDLE will be seen as a valid editor for .pyi files.

    @terryjreedy
    Copy link
    Member

    @terryjreedy terryjreedy commented Oct 15, 2021

    Steve, how do we make it so that "Edit with IDLE" appears when right-clicking on a .pyi file in Windows explorer? As easy as on mac (see Ronald's answer above)?

    @zooba
    Copy link
    Member

    @zooba zooba commented Oct 16, 2021

    Bit more complicated. It'll need changes to Tools/msi/launcher/launcher_reg.wxs, Tools/msi/tcltk/tcltk_reg.wxs, and PC/layout/support/appxmanifest.py, but I don't think it's that obvious where (we don't want these files to open with py.exe by default, so they'll need a whole new definition).

    @AlexWaygood
    Copy link
    Member Author

    @AlexWaygood AlexWaygood commented Oct 16, 2021

    Shall I add the change to the mac file to my PR, or would it be better to leave it for a separate PR? (I'm not confident enough to make the changes to the Windows files.)

    @terryjreedy
    Copy link
    Member

    @terryjreedy terryjreedy commented Oct 16, 2021

    We can add the mac change once it is manually tested on some system. I may try on my macbook. Windows will be another PR or even issue.

    @AlexWaygood
    Copy link
    Member Author

    @AlexWaygood AlexWaygood commented Jan 15, 2022

    @terry, have you had a chance to glance at my PR at all? I'd really appreciate your review! :)

    @terryjreedy
    Copy link
    Member

    @terryjreedy terryjreedy commented Jan 16, 2022

    A minimal version of util.py just for this issue is fine. What is blocking the issue deciding exactly what the minimum should be. Are both the extension list and function needed?

    Alex says somewhere in PR comments that ispythonsource passes directories. That seems wrong. Perhaps moving and revising it and making no-extension files browsable and removing the x.py(?) limitation should be a separate issue. IDLE users on *nix don't mind using extensions. I have not seen a complaint, though perhaps people who do mind silently use something else. I need to check again how the editor function is used and decide if we can leave it alone for now. So don't remove anything just now.

    I need to experiment with Ron's suggestion on my Mac.

    The Windows context menu is definitely a separate issue.

    Please add idle_test/example.pyi with the line in msg403741 for manual testing now and possibly future automatic testing.

    @terryjreedy
    Copy link
    Member

    @terryjreedy terryjreedy commented Feb 5, 2022

    For the open dialog, we only need the expanded extension list. People with no-extension python code file must select 'all files'. Once the file is open, we need the issource function to decide whether to turn on the colorizer. I intend to review the patch itself tomorrow.

    @terryjreedy
    Copy link
    Member

    @terryjreedy terryjreedy commented Feb 6, 2022

    I broaden the issue title scope and will correspondingly restrict the PR scope. 'Recognition' applies to open and save dialogs, syntax marking, and module browsing. (The latter needs fixing for .pyw also.)

    Improving the handling of no extension source with initial #! line will be a different issue.

    @terryjreedy terryjreedy changed the title IDLE: Support syntax highlighting for .pyi stub files Make IDLE recognize .pyi stub files (and .pyw) as python source Feb 6, 2022
    @terryjreedy terryjreedy changed the title IDLE: Support syntax highlighting for .pyi stub files Make IDLE recognize .pyi stub files (and .pyw) as python source Feb 6, 2022
    @terryjreedy
    Copy link
    Member

    @terryjreedy terryjreedy commented Feb 13, 2022

    New changeset 50cf499 by Alex Waygood in branch 'main':
    bpo-45447: Add syntax highlighting for .pyi files in IDLE (GH-28950)
    50cf499

    @terryjreedy
    Copy link
    Member

    @terryjreedy terryjreedy commented Feb 13, 2022

    New changeset 9fabcfb by Terry Jan Reedy in branch '3.10':
    bpo-45447: Add syntax highlighting for .pyi files in IDLE (GH-28950)
    9fabcfb

    @miss-islington
    Copy link
    Contributor

    @miss-islington miss-islington commented Feb 13, 2022

    New changeset 1d4d44c by Miss Islington (bot) in branch '3.9':
    bpo-45447: Add syntax highlighting for .pyi files in IDLE (GH-28950)
    1d4d44c

    @terryjreedy
    Copy link
    Member

    @terryjreedy terryjreedy commented Feb 13, 2022

    New changeset 2d98433 by Terry Jan Reedy in branch 'main':
    bpo-45447: Add entry to What's new 3.10 (GH-31304)
    2d98433

    @terryjreedy
    Copy link
    Member

    @terryjreedy terryjreedy commented Feb 13, 2022

    New changeset cef91ca by Terry Jan Reedy in branch 'main':
    bpo-45447: Add entry to What's new 3.9 (GH-31305)
    cef91ca

    @terryjreedy
    Copy link
    Member

    @terryjreedy terryjreedy commented Feb 13, 2022

    New changeset 6331c08 by Terry Jan Reedy in branch 'main':
    bpo-45447: Fix entry in What's New 3.11 (GH-31307)
    6331c08

    @miss-islington
    Copy link
    Contributor

    @miss-islington miss-islington commented Feb 13, 2022

    New changeset 5698e0b by Miss Islington (bot) in branch '3.10':
    bpo-45447: Add entry to What's new 3.9 (GH-31305)
    5698e0b

    @miss-islington
    Copy link
    Contributor

    @miss-islington miss-islington commented Feb 13, 2022

    New changeset ebe73e6 by Miss Islington (bot) in branch '3.9':
    bpo-45447: Add entry to What's new 3.9 (GH-31305)
    ebe73e6

    @miss-islington
    Copy link
    Contributor

    @miss-islington miss-islington commented Feb 13, 2022

    New changeset 44666c3 by Miss Islington (bot) in branch '3.10':
    bpo-45447: Add entry to What's new 3.10 (GH-31304)
    44666c3

    @terryjreedy
    Copy link
    Member

    @terryjreedy terryjreedy commented Feb 14, 2022

    bpo-46746 is the followup for browsers (currently restricted to .py files) and anything else.

    @terryjreedy terryjreedy changed the title Make IDLE recognize .pyi stub files (and .pyw) as python source Make IDLE recognize .pyi stub files as source for open, save, and edit Feb 14, 2022
    @terryjreedy terryjreedy changed the title Make IDLE recognize .pyi stub files (and .pyw) as python source Make IDLE recognize .pyi stub files as source for open, save, and edit Feb 14, 2022
    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    3.11 expert-IDLE type-feature
    Projects
    None yet
    Development

    No branches or pull requests

    5 participants