check_security_dir() and check_pid_dir() fail on network filesystem

[kaazoo]

For testing purposes and quick setup, I use SSHFS as a simple network filesystem. On Linux and Mac OSX there is FUSE SSHFS available. On Windows there's Dokan SSHFS.
I use SSHFS because Samba and NFS share were leading to incorrect uid/gid mapping. At least for Mac OSX and Ubuntu is was possble to avoid these problems by using SSHFS.

When starting ipengine on Windows, IPython executes check_security_dir() and check_pid_dir() which both try to set the permissions using chmod to 0700. This fails in my setup. After commenting out the offending lines in core/profiledir.py ipengine works.

It would be nice to be able to configure the permission setup.

[minrk]

Thanks, it should be easy to fallback on a warning if the permissions can't be secure. Does the mkdir also fail, or just chmod? I assume it is an IOError?

[kaazoo]

The output was (error message is in German):

ipengine --debug
[IPEngineApp] Config changed:
[IPEngineApp] {'Application': {'log_level': 10}}
ERROR: An unexpected error occurred while tokenizing input
The following traceback may be corrupted or invalid
The error message is: ('EOF in multi-line statement', (62, 0))

Error in sys.excepthook:
Traceback (most recent call last):
  File "C:\Python27\lib\site-packages\ipython-0.11-py2.7.egg\IPython\core\crashhandler.py", line 139, in __call__
    traceback = TBhandler.text(etype,evalue,etb,context=31)
  File "C:\Python27\lib\site-packages\ipython-0.11-py2.7.egg\IPython\core\ultratb.py", line 397, in text
    tb_offset, context)
  File "C:\Python27\lib\site-packages\ipython-0.11-py2.7.egg\IPython\core\ultratb.py", line 958, in structured_traceback
    ipinst = ipapi.get()
  File "C:\Python27\lib\site-packages\ipython-0.11-py2.7.egg\IPython\core\ipapi.py", line 29, in get
    return InteractiveShell.instance()
  File "C:\Python27\lib\site-packages\ipython-0.11-py2.7.egg\IPython\config\configurable.py", line 295, in instance
    inst = cls(*args, **kwargs)
  File "C:\Python27\lib\site-packages\ipython-0.11-py2.7.egg\IPython\core\interactiveshell.py", line 368, in __init__
    self.init_profile_dir(profile_dir)
  File "C:\Python27\lib\site-packages\ipython-0.11-py2.7.egg\IPython\core\interactiveshell.py", line 480, in init_profile_dir
    ProfileDir.create_profile_dir_by_name(self.ipython_dir, 'default')
  File "C:\Python27\lib\site-packages\ipython-0.11-py2.7.egg\IPython\core\profiledir.py", line 160, in create_profile_dir_by_name
    return cls(location=profile_dir, config=config)
  File "C:\Python27\lib\site-packages\ipython-0.11-py2.7.egg\IPython\config\configurable.py", line 88, in __init__
    super(Configurable, self).__init__(**kwargs)
  File "C:\Python27\lib\site-packages\ipython-0.11-py2.7.egg\IPython\utils\traitlets.py", line 420, in __init__
    setattr(self, key, value)
  File "C:\Python27\lib\site-packages\ipython-0.11-py2.7.egg\IPython\utils\traitlets.py", line 306, in __set__
    obj._notify_trait(self.name, old_value, new_value)
  File "C:\Python27\lib\site-packages\ipython-0.11-py2.7.egg\IPython\utils\traitlets.py", line 457, in _notify_trait
    c(name, old_value, new_value)
  File "C:\Python27\lib\site-packages\ipython-0.11-py2.7.egg\IPython\core\profiledir.py", line 82, in _location_changed
    self.security_dir = os.path.join(new, self.security_dir_name)
  File "C:\Python27\lib\site-packages\ipython-0.11-py2.7.egg\IPython\utils\traitlets.py", line 306, in __set__
    obj._notify_trait(self.name, old_value, new_value)
  File "C:\Python27\lib\site-packages\ipython-0.11-py2.7.egg\IPython\utils\traitlets.py", line 457, in _notify_trait
    c(name, old_value, new_value)
  File "C:\Python27\lib\site-packages\ipython-0.11-py2.7.egg\IPython\core\profiledir.py", line 95, in _security_dir_changed
    self.check_security_dir()
  File "C:\Python27\lib\site-packages\ipython-0.11-py2.7.egg\IPython\core\profiledir.py", line 101, in check_security_dir
    os.chmod(self.security_dir, 0700)
WindowsError: [Error 2] Das System kann die angegebene Datei nicht finden: u'n:\\ipython\\profile_default\\security'

Original exception was:
Traceback (most recent call last):
  File "C:\Python27\Scripts\ipengine-script.py", line 9, in <module>
    load_entry_point('ipython==0.11', 'console_scripts', 'ipengine')()
  File "C:\Python27\lib\site-packages\ipython-0.11-py2.7.egg\IPython\parallel\apps\ipengineapp.py", line 301, in launch_new_instance
    app.initialize()
  File "C:\Python27\lib\site-packages\ipython-0.11-py2.7.egg\IPython\parallel\apps\ipengineapp.py", line 285, in initialize
    super(IPEngineApp, self).initialize(argv)
  File "C:\Python27\lib\site-packages\ipython-0.11-py2.7.egg\IPython\parallel\apps\baseapp.py", line 133, in initialize
    super(BaseParallelApplication, self).initialize(argv)
  File "C:\Python27\lib\site-packages\ipython-0.11-py2.7.egg\IPython\core\application.py", line 304, in initialize
    self.init_profile_dir()
  File "C:\Python27\lib\site-packages\ipython-0.11-py2.7.egg\IPython\core\application.py", line 218, in init_profile_dir
    p = ProfileDir.find_profile_dir_by_name(self.ipython_dir, self.profile, self.config)
  File "C:\Python27\lib\site-packages\ipython-0.11-py2.7.egg\IPython\core\profiledir.py", line 186, in find_profile_dir_by_name
    return cls(location=profile_dir, config=config)
  File "C:\Python27\lib\site-packages\ipython-0.11-py2.7.egg\IPython\config\configurable.py", line 88, in __init__
    super(Configurable, self).__init__(**kwargs)
  File "C:\Python27\lib\site-packages\ipython-0.11-py2.7.egg\IPython\utils\traitlets.py", line 420, in __init__
    setattr(self, key, value)
  File "C:\Python27\lib\site-packages\ipython-0.11-py2.7.egg\IPython\utils\traitlets.py", line 306, in __set__
    obj._notify_trait(self.name, old_value, new_value)
  File "C:\Python27\lib\site-packages\ipython-0.11-py2.7.egg\IPython\utils\traitlets.py", line 457, in _notify_trait
    c(name, old_value, new_value)
  File "C:\Python27\lib\site-packages\ipython-0.11-py2.7.egg\IPython\core\profiledir.py", line 82, in _location_changed
    self.security_dir = os.path.join(new, self.security_dir_name)
  File "C:\Python27\lib\site-packages\ipython-0.11-py2.7.egg\IPython\utils\traitlets.py", line 306, in __set__
    obj._notify_trait(self.name, old_value, new_value)
  File "C:\Python27\lib\site-packages\ipython-0.11-py2.7.egg\IPython\utils\traitlets.py", line 457, in _notify_trait
    c(name, old_value, new_value)
  File "C:\Python27\lib\site-packages\ipython-0.11-py2.7.egg\IPython\core\profiledir.py", line 95, in _security_dir_changed
    self.check_security_dir()
  File "C:\Python27\lib\site-packages\ipython-0.11-py2.7.egg\IPython\core\profiledir.py", line 101, in check_security_dir
    os.chmod(self.security_dir, 0700)
WindowsError: [Error 2] Das System kann die angegebene Datei nicht finden: u'n:\\ipython\\profile_default\\security'

It says the file (directory) could not be found, allthough it's existing:

dir n:\ipython\profile_default\security
 Datenträger in Laufwerk N: ist DOKAN
 Volumeseriennummer: 1983-1116

 Verzeichnis von n:\ipython\profile_default\security

07.09.2011  13:50               127 ipcontroller-client.json
07.09.2011  13:50               127 ipcontroller-engine.json
               2 Datei(en),            254 Bytes
               0 Verzeichnis(se), 10.737.418.240 Bytes frei

[kaazoo]

In your patch, I had to change IOError to WindowsError, but it complains about the missing log attribute:

...
...
  File "C:\Python27\lib\site-packages\ipython-0.11-py2.7.egg\IPython\core\profiledir.py", line 95, in _security_dir_changed
    self.check_security_dir()
  File "C:\Python27\lib\site-packages\ipython-0.11-py2.7.egg\IPython\core\profiledir.py", line 104, in check_security_dir
    self.log.warn("Could not set security dir permissions to private.")
AttributeError: 'ProfileDir' object has no attribute 'log'

[minrk]

Silly first attempt - it should be OSError (WindowsError subclasses this), not IOError, and the log attribute has been fixed.

It's very strange that the error claims that it can't be found, since the chmod is only attempted if os.path.isdir(loc) returns True. Still, it should now just log and ignore the error, rather than crashing.

[kaazoo]

With your changes, I get the following output:

ipengine
[IPEngineApp] Could not set security dir permissions to private.
Traceback (most recent call last):
  File "C:\Python27\lib\site-packages\ipython-0.11-py2.7.egg\IPython\core\profiledir.py", line 102, in check_security_dir
    os.chmod(self.security_dir, 0700)
WindowsError: [Error 2] Das System kann die angegebene Datei nicht finden: u'n:\\ipython\\profile_default\\security'
[IPEngineApp] Could not set pid dir permissions to private.
Traceback (most recent call last):
  File "C:\Python27\lib\site-packages\ipython-0.11-py2.7.egg\IPython\core\profiledir.py", line 114, in check_pid_dir
    os.chmod(self.pid_dir, 0700)
WindowsError: [Error 2] Das System kann die angegebene Datei nicht finden: u'n:\\ipython\\profile_default\\pid'
[IPEngineApp] Could not set security dir permissions to private.
Traceback (most recent call last):
  File "C:\Python27\lib\site-packages\ipython-0.11-py2.7.egg\IPython\core\profiledir.py", line 102, in check_security_dir
    os.chmod(self.security_dir, 0700)
WindowsError: [Error 2] Das System kann die angegebene Datei nicht finden: u'n:\\ipython\\profile_default\\security'
[IPEngineApp] Could not set pid dir permissions to private.
Traceback (most recent call last):
  File "C:\Python27\lib\site-packages\ipython-0.11-py2.7.egg\IPython\core\profiledir.py", line 114, in check_pid_dir
    os.chmod(self.pid_dir, 0700)
WindowsError: [Error 2] Das System kann die angegebene Datei nicht finden: u'n:\\ipython\\profile_default\\pid'
[IPEngineApp] Using existing profile dir: u'n:\\ipython\\profile_default'
[IPEngineApp] Loading url_file u'n:\\ipython\\profile_default\\security\\ipcontroller-engine.json'
[IPEngineApp] Registering with controller at tcp://192.168.1.12:65317

The exeception is complaining about u'n:\ipython\profile_default\security' but that's the Unicode Python value as far as I understand. Shouldn't the normal string representation of the value be used for chmod() ?