Add directory for startup files #950

Merged
merged 5 commits into from Oct 31, 2011

Projects

None yet

4 participants

@minrk
IPython member

profile/startup dir is checked for files, and they are run prior to explicit exec_code and exec_files.

a short README is added to the dir explaining how they work, and a short paragraph is added to the docs, in both the interactive tutorial and config overview.

closes #914

@fperez
IPython member

Looks great, thanks! I'd like to have a test in the suite for this though: a simple temp-generated file with zzz = 123456 and checking afterwards that the initialized instance has zzz with that value should suffice. But at least it will ensure the codepaths for loading startup files do get exercised in the test suite.

Once we get a test, good to go. Great work, this will be really liked by users.

@minrk
IPython member

I don't quite know how to write that test, but I'll figure it out. I think I need to copy a lot of the env code from test_path.

@fperez
IPython member

Great, thanks. Let me know if you get stuck and I can pitch in.

@minrk
IPython member

test added (and fixed a small bug in ipexec_validate, which was needed)

@fperez
IPython member

Mmh, the check for the startup files/directory must be optional. Right now, IPython won't even start if those files aren't there:

p
Error in sys.excepthook:
Traceback (most recent call last):
  File "/home/fperez/usr/lib/python2.7/site-packages/IPython/core/crashhandler.py", line 144, in __call__
    traceback = TBhandler.text(etype,evalue,etb,context=31)
  File "/home/fperez/usr/lib/python2.7/site-packages/IPython/core/ultratb.py", line 402, in text
    tb_offset, context)
  File "/home/fperez/usr/lib/python2.7/site-packages/IPython/core/ultratb.py", line 964, in structured_traceback
    ipinst = ipapi.get()
  File "/home/fperez/usr/lib/python2.7/site-packages/IPython/core/ipapi.py", line 28, in get
    return InteractiveShell.instance()
  File "/home/fperez/usr/lib/python2.7/site-packages/IPython/config/configurable.py", line 318, in instance
    inst = cls(*args, **kwargs)
  File "/home/fperez/usr/lib/python2.7/site-packages/IPython/core/interactiveshell.py", line 384, in __init__
    self.init_profile_dir(profile_dir)
  File "/home/fperez/usr/lib/python2.7/site-packages/IPython/core/interactiveshell.py", line 503, in init_profile_dir
    ProfileDir.create_profile_dir_by_name(self.ipython_dir, 'default')
  File "/home/fperez/usr/lib/python2.7/site-packages/IPython/core/profiledir.py", line 181, in create_profile_dir_by_name
    return cls(location=profile_dir, config=config)
  File "/home/fperez/usr/lib/python2.7/site-packages/IPython/config/configurable.py", line 87, in __init__
    super(Configurable, self).__init__(**kwargs)
  File "/home/fperez/usr/lib/python2.7/site-packages/IPython/utils/traitlets.py", line 421, in __init__
    setattr(self, key, value)
  File "/home/fperez/usr/lib/python2.7/site-packages/IPython/utils/traitlets.py", line 307, in __set__
    obj._notify_trait(self.name, old_value, new_value)
  File "/home/fperez/usr/lib/python2.7/site-packages/IPython/utils/traitlets.py", line 458, in _notify_trait
    c(name, old_value, new_value)
  File "/home/fperez/usr/lib/python2.7/site-packages/IPython/core/profiledir.py", line 86, in _location_changed
    self.startup_dir = os.path.join(new, self.startup_dir_name)
  File "/home/fperez/usr/lib/python2.7/site-packages/IPython/utils/traitlets.py", line 307, in __set__
    obj._notify_trait(self.name, old_value, new_value)
  File "/home/fperez/usr/lib/python2.7/site-packages/IPython/utils/traitlets.py", line 458, in _notify_trait
    c(name, old_value, new_value)
  File "/home/fperez/usr/lib/python2.7/site-packages/IPython/core/profiledir.py", line 98, in _startup_dir_changed
    self.check_startup_dir()
  File "/home/fperez/usr/lib/python2.7/site-packages/IPython/core/profiledir.py", line 106, in check_startup_dir
    shutil.copy(src, readme)
  File "/usr/lib/python2.7/shutil.py", line 116, in copy
    copyfile(src, dst)
  File "/usr/lib/python2.7/shutil.py", line 81, in copyfile
    with open(src, 'rb') as fsrc:
IOError: [Errno 2] No such file or directory: u'/home/fperez/usr/lib/python2.7/site-packages/IPython/config/profile/README_STARTUP'

Having those files and directories should be completely optional... Furthermore, it looks like you forgot to commit the README file itself, I think.

@minrk
IPython member
minrk added some commits Oct 21, 2011
@minrk minrk add startup_dir to profiles
Any .py or .ipy file in this directory will be run at startup.
Files are run in in lexicographical order, as described in #914
7c21f7c
@minrk minrk add README to startup dir 8a68c1b
@minrk minrk add startup files to docs 3d36e7e
@minrk minrk fix missing options arg to ipexec in ipexec_validate a4d4260
@minrk minrk add test_profile with startup tests
more profile tests should be added to this file
5a904a2
@minrk
IPython member

Missing readme added to git.

You absolutely don't have to create the dir yourself, IPython will do it for you (just like log, security, pid), and stage the README into it.

@fperez
IPython member

Looks great now, thanks! Merging...

@fperez fperez merged commit b131150 into ipython:master Oct 31, 2011
@minrk
IPython member

awesome, thanks!

@pierre-haessig

Hi,
One question about startup files in terms of namespace. Looking at http://python.6.n6.nabble.com/Run-code-at-startup-td1652498.html it feels like startup files would work in a similar fasion as exec_files and exec_lines.

However, I can use exec_lines to import a module like

c.InteractiveShellApp.exec_lines = ['from future import division']

and then the module is available in the shell
but if I drop the line from __future__ import division in a simple Python startup file, it is not. I guess it is not executed in the shell namespace ?

Was this behavioral difference intentional ?

@takluyver
IPython member
@pierre-haessig

Good point ! Indeed I've just checked that for instance import math works well. Thanks for the feedback.

I don't know the internal machinery behind future imports though. I just noticed that there is a complain if the future import statement is not the first one in the file. But I guess it's the same for any Python script. Is it ?

@takluyver
IPython member
@pierre-haessig

I've checked that indeed using the .ipy extension makes future imports work. Thanks for the tip.

I feel like this issue is too small to deserve a bug report but I think it should be fixed at some point. In the meantime, my suggestion is to add a footnote in the README in the startup directory that would explain: 1) the problem 2) the solution you mentioned (using .ipy instead of .py).

I propose something like this :

About importing modules : using import statements in startup .py files will make the imported modules available in the running ipython session. This is one of the useful uses of startup scripts.
However, there is an exception for imports from the future module (like division, print_function, ...) which won't be available due to the way the startup scripts are currently executed. This limitation may be lifted in a next release but in the meantime, the simplest solution is to use the .ipy extension which changes the execution path of the script and fixes the problem.

What do you think ?

@takluyver
IPython member

It may require a note in the documentation, depending on what we decide to do. On reflection, though, using .ipy is a rather ugly hack, and we might decide that it shouldn't work from .ipy files either. A better workaround is to call get_ipython().run_cell('from __future__ import division') in your startup script.

The tricky thing is working out what semantics people expect. If your startup.py includes from __future__ import division, and you do from startup import * at a plain Python prompt, you don't get new style division. These __future__ imports were always intended to work at the module level, without affecting other code in the process. You may want new style division only within that startup file, or you may want it to affect interactive use.

So, where can IPython run interactive code that we need to consider:

  • At startup: exec_lines, exec_files, startup files. There are .py and .ipy files, and they should definitely behave the same way with future statements.
  • %run (and %run -i), %rerun, %edit, macros, %loadpy
  • Extensions
  • Code loaded by the autoreload extension

My gut feeling is that of those, exec_lines, %loadpy, macros and maybe %run -i should 'leak' future statements to the interactive environment. Everything else should keep future statements to itself (except by using run_cell()). I'll take this to the mailing list.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment