make_watershed_bem - IOError: [Errno 13] Permission denied: 'brain'

[WilliamCollishaw]

System:
Python 2.7.12
Ubuntu 16.04.1 LTS
MNE version 0.14.git

I have been running into a few problems with this function. Initially I had the same problem as #3274 where freesurfer was not being found/setup correctly. I thought this was fixed in 0.14 but I ran into the same issue. I manually modified the utils.py to setup freesurfer before it ran the command. That fixed the problem but I get this new problem, Permission denied, 'brain'
The following shows the error. I have editted a few things such as subject directory for privacy reasons.

Saving /SUBJECT_DIR/bem/watershed/ws
done
Triangle file: created by USER on Tue Oct 18 15:30:33 2016 nvert = 10242 ntri = 20480
Triangle neighbors and vertex normals...
Traceback (most recent call last):
File "pythonScripts/pipeline.py", line 208, in <module>
main()
File "pythonScripts/pipeline.py", line 169, in main
bem.make_watershed_bem(URSI, subjects_dir=MRISubjectDirectory, overwrite=True)
File "<string>", line 2, in make_watershed_bem
File "/usr/local/lib/python2.7/dist-packages/mne/utils.py", line 708, in verbose
return function(*args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/mne/bem.py", line 1081, in make_watershed_bem
write_surface(s, surf['rr'], surf['tris'], volume_info=volume_info)
File "/usr/local/lib/python2.7/dist-packages/mne/surface.py", line 764, in write_surface
with open(fname, 'wb') as fid:
IOError: [Errno 13] Permission denied: 'brain'

I hope this gives enough information!

Extra information, I was getting these issues previously using an older system:
CentOS release 6.7
Python 2.6.6
MNE version 0.12

Upgraded system to get newer version of MNE but the problems still exist

[larsoner]

If you do import pdb; pdb.pm() then p fname (then q to quit debugging) you can see what filename it was trying to write. Can you write a file to that directory yourself using e.g. touch /path/to/file?

[WilliamCollishaw]

I am not at the computer right now to check this. But when I had the problem on the old system it was trying to create a file brain. No file type or directory with it. Will confirm this tomorrow when I am back in!

[WilliamCollishaw]

Trying to run the make_watershed_bem inside the debugger returned me to a previous problem
Running subprocess: mri_watershed -useSRAS -surf
would return:
/bin/sh: 1: source: not found
('', '')

[WilliamCollishaw]

When i was previously getting this error on the old set up i found where the fail was occurring and ran the MNE command alone:

./mne_convert_surface --surf /SUBJECT_DIR/bem/watershed/SUBJECTURSI_brain_surface --mghmri /SUBJECT_DIR/mri/T1.mgz --surfout brain --replacegeom

I got the following output:

mne_convert_surface version 1.13 compiled at Jan 6 2011 02:25:19

surf input file :/SUBJECT_DIR/bem/watershed/SUBJECTURSI_brain_surface
surf output file : brain

/SUBJECT_DIR/bem/watershed/SUBJECT_URSI_brain_surface : No such file or directory

I know this works differently in 0.14 but both times I was getting permission denied, brain

This output makes it look like there is no directory with the filename brain.

I hope this helps!

[larsoner]

/bin/sh: 1: source: not found

Hmm, this might be a shell difference. But before we go there...

Try using postmortem debugging (let it hit the error, then do import pdb; pdb.pm()) to getting the path to the file, and try to manually write something yourself at that location using the shell. If the fname is something short like brain, you can use p os.getcwd() to get the path where it's actually trying to write it, or better yet just do p os.path.abspath(fname).

[agramfort]

apparently you don't have write access to the bem folder. to me it's not an MNE problem

[WilliamCollishaw]

Okay, I will run the script as an admin. Will see if that helps. Eric. I am trying to get permissions to debug in the surface.py file which is why it is taking a while

[WilliamCollishaw]

In the SUBJECT_DIR/bem/watershed directory I have 4 files.
SUBJECTURSI_brain_surface
SUBJECTURSI_inner_skull_surface
SUBJECTURSI_outer_skull_surface
SUBJECTURSI_outer_skin_surface

in the bem.py file the lines being run before write surface is:
surfs = ['brain', 'inner_skull', 'outer_skull', 'outer_skin']
for s in surfs:

surf_ws_out = op.join(ws_dir, '%s_%s_surface' % (subject, s))
surf, volume_info = _read_surface_geom(surf_ws_out, read_metadata=True)
volume_info.update(new_info)**** # replace volume info, 'head' stays
write_surface(s, surf['rr'], surf['tris'], volume_info=volume_info)

the values of surfs are not edited before write_surface and write_surface is trying to open them. As far as I am aware there is no file brain with no file type after it.

[larsoner]

I am trying to get permissions to debug in the surface.py file which is why it is taking a while

You shouldn't need write permissions to do post-mortem debugging. Run the code in a python interpreter in interactive mode, and when you hit an error do pdb.pm(). Like:

>>> import pdb
>>> import mne
>>> mne.bem.make_watershed_bem('foo')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<string>", line 2, in make_watershed_bem
  File "/home/larsoner/custombuilds/mne-python/mne/utils.py", line 682, in verbose
    return function(*args, **kwargs)
  File "/home/larsoner/custombuilds/mne-python/mne/bem.py", line 1022, in make_watershed_bem
    requires_mne=True)[:2]
  File "/home/larsoner/custombuilds/mne-python/mne/bem.py", line 1504, in _prepare_env
    raise RuntimeError('I cannot find freesurfer. The FREESURFER_HOME '
RuntimeError: I cannot find freesurfer. The FREESURFER_HOME environment variable is not set.
>>> pdb.pm()
> /home/larsoner/custombuilds/mne-python/mne/bem.py(1504)_prepare_env()
-> raise RuntimeError('I cannot find freesurfer. The FREESURFER_HOME '
(Pdb) p subjects_dir
None

[WilliamCollishaw]

Progress! os.getcwd() and os.path.abspath(fname) both returned the location of my last os.chdir location in my script +/brain which does not exist. I am passing the subject directory in the make_watershed_bem call. But it seems to be working in a folder that i previous redirected to.

[larsoner]

So you're saying that at line 1081, in make_watershed_bem, your os.getcwd is something invalid / nonexistent? It should be trying to write to ./brain (where ./ refers to the directory from which you issued the command) but you're saying that instead it's trying to write to some other (probably read-only) directory? That would be a bug at our end, probably related to our handling of paths. There are some os.chdir calls in there. Probably we need to refactor those, or more easily add an os.abspath somewehre.

Does that summarize it?

[WilliamCollishaw]

If there are some os.chdir calls in make_watershed_bem then yes there must be a bug. Lets say i am using the directory test/directory/here/. I change to test/directory/here/ a few lines before calling make_watershed_bem.

Then make_wateshed_bem is failing with IOError: [Errno 13] Permission denied: 'brain'
where i would get:
os.getcwd()
test/directory/here/
os.path.abspath(fname)
test/directory/here/brain

my os.chdir() is a valid path for an operation i previously worked on. But your mne functions never seem to bring me out of that directory.

[WilliamCollishaw]

I just changed my directory to the specific subject directory before calling it and it worked! it seems that it does not correctly pick up the specified subject directory.