IDLE: set and unset __file__ for startup files

[terryjreedy]

BPO	32984
Nosy	@terryjreedy, @csabella, @miss-islington
PRs	bpo-32984: IDLE - set __file__ for startup files #5981 [3.7] bpo-32984: IDLE - set __file__ for startup files (GH-5981) #5986 [3.6] bpo-32984: IDLE - set __file__ for startup files (GH-5981) #5987 [3.6] bpo-32984: IDLE - set __file__ for startup files (GH-5981) #5993

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/terryjreedy'
closed_at = <Date 2018-03-05.19:46:08.581>
created_at = <Date 2018-03-02.17:13:39.384>
labels = ['3.8', 'expert-IDLE', 'type-bug', '3.7']
title = 'IDLE: set and unset __file__ for startup files'
updated_at = <Date 2018-03-05.19:46:08.580>
user = 'https://github.com/terryjreedy'

bugs.python.org fields:

activity = <Date 2018-03-05.19:46:08.580>
actor = 'terry.reedy'
assignee = 'terry.reedy'
closed = True
closed_date = <Date 2018-03-05.19:46:08.581>
closer = 'terry.reedy'
components = ['IDLE']
creation = <Date 2018-03-02.17:13:39.384>
creator = 'terry.reedy'
dependencies = []
files = []
hgrepos = []
issue_num = 32984
keywords = ['patch']
message_count = 8.0
messages = ['313140', '313151', '313152', '313238', '313248', '313254', '313267', '313280']
nosy_count = 3.0
nosy_names = ['terry.reedy', 'cheryl.sabella', 'miss-islington']
pr_nums = ['5981', '5986', '5987', '5993']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue32984'
versions = ['Python 3.6', 'Python 3.7', 'Python 3.8']

[terryjreedy]

'python somefile.py' sets main.__file__ to 'somefile.py'. 'python' leaves __file__ unset.  If PYTHONSTARTUP is set to somefile.py, 'python' executes somefile.py in main with __file__ set to 'somefile.py', then unsets __file__ before the >>> prompt, as if somefile has not been executed.  Any explicit setting of __file__ in somefile is undone.

tem2.py:
print(name, __file__)
__file__ = 'abc.py'

> F:\dev\3x> set PYTHONSTARTUP=f:/python/a/tem2.py
> F:\dev\3x> python
...
__main__ f:/python/a/tem2.py
>>> __file__
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name '__file__' is not defined

With IDLE, when 'python -m idlelib.idle' is run with '-s' or '-r f:/python/a/tem2.py', NameError is raised for the print in tem2.py. This was reported this SO question.
https://stackoverflow.com/questions/49054093/cannot-use-file-when-opening-module-in-idle

In both cases, the file is run with execfile(filename).

    def execfile(self, filename, source=None):
        "Execute an existing file"
        if source is None:
            with tokenize.open(filename) as fp:
                source = fp.read()

My guess is that wrapping the source with f"__file__ = {filename}\n" and "del __file__\n" should work.

[terryjreedy]

When the IDLE GUI process is started, its main.__file__ is set to the idlelib .py file used to start the process.  If IDLE is started with -n, so that user code is run in the same process (the original mode), the -r or -s startup file and code entered at >>> sees this value*, rather than the user startup file name or a NameError.

• I am not sure why it does not see the value of __file__ in the pyshell module, which I would expect to be '.../idlelib/pyshell.py'.

The change proposed above would fix __file__ for -n also, but would disable any subsequently executed idle code that uses __file__ that gets unset. So I think the wrapping should be conditioned on use_subprocess (True unless -n).

[terryjreedy]

Revised, more exact, URL:
https://stackoverflow.com/questions/49054093/cannot-use-file-when-running-startup-file-in-idles-normal-mode

[terryjreedy]

With the PR, IDLE behaves the same as Python on Windows.
   .../3x> python -m idlelib -r f:/python/a/tem2.py
(see original post for content of tem2.py)
prints __file__ instead of raising NameError, while
  >>> __file__ continues to raise NameError

Cheryl, I really expect the PR to work on linux, but can you manually test it anyway? If you think the blurb needs rst markup and feel like adding some, go ahead.

[csabella]

The change works on Linux with the patch (and gave a NameError without the patch). I left a note on the PR for a possible SyntaxError.

I didn't have authority to push the blurb markup change to your branch, so I copied it into a comment. If you click edit on it, you should be able to cut and paste it as is into the blurb file.

[terryjreedy]

New changeset 22c82be by Terry Jan Reedy in branch 'master':
bpo-32984: IDLE - set __file__ for startup files (GH-5981)
22c82be

[miss-islington]

New changeset fd340bf by Miss Islington (bot) in branch '3.7':
bpo-32984: IDLE - set __file__ for startup files (GH-5981)
fd340bf

[miss-islington]

New changeset 6935a51 by Miss Islington (bot) in branch '3.6':
bpo-32984: IDLE - set __file__ for startup files (GH-5981)
6935a51