problem running with plugin PBS on cluster

[venpopov]

I'm following this tutorial to learn nipype: http://miykael.github.io/nipype-beginner-s-guide/firstSteps.html

The preprocessing script works fine on my personal computer:
https://github.com/miykael/nipype-beginner-s-guide/blob/master/scripts/tutorial_3_first_steps.py

However I get the following error on a CentOS 6.2 remote cluster:

/data2/rederlab/software/anaconda2/lib/python2.7/site-packages/IPython/kernel/__init__.py:13: ShimWarning: The `IPython.kernel` package has been deprecated. You should import from ipykernel or jupyter_client instead.
  "You should import from ipykernel or jupyter_client instead.", ShimWarning)

Traceback (most recent call last):
  File "/home/vpopov/nipype_tutorial/nipype_tutorial.py", line 125, in <module>
    plugin_args=dict(qsub_args='-l nodes=4:ppn=5'))
  File "/data2/rederlab/software/anaconda2/lib/python2.7/site-packages/nipype/pipeline/engine/workflows.py", line 595, in run
    runner.run(execgraph, updatehash=updatehash, config=self.config)
  File "/data2/rederlab/software/anaconda2/lib/python2.7/site-packages/nipype/pipeline/plugins/base.py", line 265, in run
    graph=graph)
  File "/data2/rederlab/software/anaconda2/lib/python2.7/site-packages/nipype/pipeline/plugins/base.py", line 402, in _send_procs_to_workers
    updatehash=updatehash)
  File "/data2/rederlab/software/anaconda2/lib/python2.7/site-packages/nipype/pipeline/plugins/base.py", line 564, in _submit_job
    return self._submit_batchtask(batchscriptfile, node)
  File "/data2/rederlab/software/anaconda2/lib/python2.7/site-packages/nipype/pipeline/plugins/pbs.py", line 100, in _submit_batchtask
    str(e))))

RuntimeError: Could not submit pbs task for node selectfiles.a04
decode() argument 1 must be string, not None
Interface CommandLine failed to run. 

I've changed the workflow run command to execute with the server's PBS scheduling system:

preprocess.run(plugin='PBS', 
               plugin_args=dict(qsub_args='-l nodes=4:ppn=5'))

Also, I've removed the write_graph command and the import of pygraphviz, because I couldn't install graphviz and pygraphviz on the server (I have no root access, maybe it's possible to circumvent this somehow?)

Any help is appreciated.

[satra]

on a cluster you can install an entire nipype+python environment using miniconda (http://conda.pydata.org/miniconda.html) without root access.

thanks for reporting this, hopefully this is fixed in master and you can test it using the local install.

[venpopov]

This looks like a stripped down version of Anaconda, which I already installed, and it doesn't include the pygraphviz or graphviz packages, which I can't install without root access. Can you please elaborate how that will solve my problem?

Let me reiterate that I've install almost everything just fine, with the exception of those two packages for which it says that they are highly recommended, but not required. And that my problem is the following error:

RuntimeError: Could not submit pbs task for node selectfiles.a04
decode() argument 1 must be string, not None
Interface CommandLine failed to run. 

[satra]

@venpopov

re: graphviz: https://anaconda.org/search?q=graphviz

re: pbs error, is this with current master?

[venpopov]

This was after following this installation tutorial: http://miykael.github.io/nipype-beginner-s-guide/installation.html, namely the following commands are related to the nipype installation:

pip install nipype
pip install nibabel
pip install rdflib
pip install nipy
pip install dipy
pip install --upgrade nipype
pip install --upgrade nibabel nipype rdflib nipy dipy

[satra]

that would not install the current master. could you please try:

pip install --upgrade https://github.com/nipy/nipype/archive/master.zip

[venpopov]

Actually yes, it was the current master with this command:
pip uninstall nipype; pip install
https://github.com/nipy/nipype/archive/master.zip

as suggested here due to another bug:
#1395 (comment)

Ven Popov
Ph.D. Student
Department of Psychology
Center for the Neural Basis of Cognition
Carnegie Mellon University
web: venpopov.com

On Mon, Mar 7, 2016 at 10:20 AM, Satrajit Ghosh notifications@github.com
wrote:

@venpopov https://github.com/venpopov

re: graphviz: https://anaconda.org/search?q=graphviz

re: pbs error, is this with current master?

—
Reply to this email directly or view it on GitHub
#1396 (comment).

[satra]

could you please provide the output from this?

from nipype import get_info
get_info()

[venpopov]

{'nibabel_version': '2.0.2', 'networkx_version': '1.11', 'numpy_version': '1.10.4', 'sys_platform': 'linux2', 'sys_version': '2.7.11 |Anaconda 2.5.0 (64-bit)| (default, Dec 6 2015, 18:08:32) \n[GCC 4.4.7 20120313 (Red Hat 4.4.7-1)]', 'commit_source': 'archive substitution', 'commit_hash': '%h', 'pkg_path': '/data2/rederlab/software/anaconda2/lib/python2.7/site-packages/nipype', 'sys_executable': '/data2/rederlab/software/anaconda2/bin/python', 'traits_version': '4.5.0', 'scipy_version': '0.17.0'}

[satra]

oops the one thing that i was looking for was commit_hash and i'm not sure why that didn't show up.

in that case, would you mind doing the pip install --upgrade https://github.com/nipy/nipype/archive/master.zip

and ensure that the error still happens, and post it? i did look at the code and the place where i see the error, the one thing that could possibly help would be to use text_type.

in the pbs.py file, one can try:

diff --git a/nipype/pipeline/plugins/pbs.py b/nipype/pipeline/plugins/pbs.py
index 5265468..0c29867 100644
--- a/nipype/pipeline/plugins/pbs.py
+++ b/nipype/pipeline/plugins/pbs.py
@@ -7,7 +7,7 @@ import subprocess

 from .base import (SGELikeBatchManagerBase, logger, iflogger, logging)

-from nipype.interfaces.base import CommandLine
+from ...interfaces.base import CommandLine, text_type


 class PBSPlugin(SGELikeBatchManagerBase):
@@ -97,7 +97,7 @@ class PBSPlugin(SGELikeBatchManagerBase):
                     iflogger.setLevel(oldlevel)
                     raise RuntimeError('\n'.join((('Could not submit pbs task'
                                                    ' for node %s') % node._id,
-                                                  str(e))))
+                                                  text_type(e))))
             else:
                 break
         iflogger.setLevel(oldlevel)

[venpopov]

I tried to upgrade nipype with this command and I get exactly the same
error as above.

Can you explain what I should do with the code you gave me? Am I supposed
to run it in the terminal, supplying the path to pbs.py?

Ven Popov
Ph.D. Student
Department of Psychology
Center for the Neural Basis of Cognition
Carnegie Mellon University
web: venpopov.com

On Mon, Mar 7, 2016 at 4:11 PM, Satrajit Ghosh notifications@github.com
wrote:

oops the one thing that i was looking for was commit_hash and i'm not sure
why that didn't show up.

in that case, would you mind doing the pip install --upgrade
https://github.com/nipy/nipype/archive/master.zip

and ensure that the error still happens, and post it? i did look at the
code and the place where i see the error, the one thing that could possibly
help would be to use text_type.

in the pbs.py file, one can try:

diff --git a/nipype/pipeline/plugins/pbs.py b/nipype/pipeline/plugins/pbs.py
index 5265468..0c29867 100644
--- a/nipype/pipeline/plugins/pbs.py
+++ b/nipype/pipeline/plugins/pbs.py
@@ -7,7 +7,7 @@ import subprocess

from .base import (SGELikeBatchManagerBase, logger, iflogger, logging)

-from nipype.interfaces.base import CommandLine
+from ...interfaces.base import CommandLine, text_type

class PBSPlugin(SGELikeBatchManagerBase):
@@ -97,7 +97,7 @@ class PBSPlugin(SGELikeBatchManagerBase):
iflogger.setLevel(oldlevel)
raise RuntimeError('\n'.join((('Could not submit pbs task'
' for node %s') % node._id,

•                                              str(e))))
  

•                                              text_type(e))))
       else:
           break
   iflogger.setLevel(oldlevel)
  

—
Reply to this email directly or view it on GitHub
#1396 (comment).

[venpopov]

I realized what I had to do with the code you provided. But even after modifying the pbs.py files like that, I still get the same error, though with the slight differentce that is says "text_type(e)" now instead:

  File "/data2/rederlab/software/anaconda2/lib/python2.7/site-packages/nipype/pipeline/plugins/pbs.py", line 100, in _submit_batchtask
    text_type(e))))
RuntimeError: Could not submit pbs task for node selectfiles.a16
decode() argument 1 must be string, not None
Interface CommandLine failed to run. 

[satra]

on your system could you please do:

import locale
print(locale.getdefaultlocale())

[venpopov]

('en_US', 'UTF-8')

Ven Popov
Ph.D. Student
Department of Psychology
Center for the Neural Basis of Cognition
Carnegie Mellon University
web: venpopov.com

On Mon, Mar 7, 2016 at 5:09 PM, Satrajit Ghosh notifications@github.com
wrote:

on your system could you please do:

import locale
print(locale.getdefaultlocale())

—
Reply to this email directly or view it on GitHub
#1396 (comment).

[satra]

ok here is the likely culprit - let me know if the change below helps. if so i'll update the codebase.

diff --git a/nipype/interfaces/base.py b/nipype/interfaces/base.py
index 742ccd3..e1b854c 100644
--- a/nipype/interfaces/base.py
+++ b/nipype/interfaces/base.py
@@ -1164,7 +1164,7 @@ class Stream(object):
     def _read(self, drain):
         "Read from the file descriptor"
         fd = self.fileno()
-        buf = os.read(fd, 4096).decode()
+        buf = os.read(fd, 4096).decode(locale.getdefaultlocale()[1])
         if not buf and not self._buf:
             return None
         if '\n' not in buf:

[venpopov]

I just tried that, I get the same error as before.

Ven Popov
Ph.D. Student
Department of Psychology
Center for the Neural Basis of Cognition
Carnegie Mellon University
web: venpopov.com

On Mon, Mar 7, 2016 at 5:33 PM, Satrajit Ghosh notifications@github.com
wrote:

ok here is the likely culprit - let me know if the change below helps. if
so i'll update the codebase.

diff --git a/nipype/interfaces/base.py b/nipype/interfaces/base.py
index 742ccd3..e1b854c 100644
--- a/nipype/interfaces/base.py
+++ b/nipype/interfaces/base.py
@@ -1164,7 +1164,7 @@ class Stream(object):
def _read(self, drain):
"Read from the file descriptor"
fd = self.fileno()

•    buf = os.read(fd, 4096).decode()
  

•    buf = os.read(fd, 4096).decode(locale.getdefaultlocale()[1])
   if not buf and not self._buf:
       return None
   if '\n' not in buf:
  

—
Reply to this email directly or view it on GitHub
#1396 (comment).