Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pbsdsh problem #121

Closed
brycelelbach opened this issue Jul 10, 2012 · 4 comments
Closed

pbsdsh problem #121

brycelelbach opened this issue Jul 10, 2012 · 4 comments

Comments

@brycelelbach
Copy link
Member

[reported by manderson] [Trac time Thu Aug 25 16:28:27 2011] hpx r5432

  pbsdsh /fslhome/mwa2/parallex/install/bin/adaptive1d -p ./idadaptive1d

PBS_NODEFILE

m6-1-1
m6-1-1
m6-1-1
m6-1-1
m6-1-1
m6-1-1
m6-1-1
m6-1-1
m6-1-2
m6-1-2
m6-1-2
m6-1-2
m6-1-2
m6-1-2
m6-1-2
m6-1-2

Output:

hpx::init: std::exception caught: hpx::init: std::exception caught: hpx::init: std::exception caught: listen: Address already in u
se: HPX(network_error)
bind: Address already in use: HPX(network_error)
bind: Address already in use: HPX(network_error)
hpx::init: std::exception caught: bind: Address already in use: HPX(network_error)
hpx::init: std::exception caught: listen: Address already in use: HPX(network_error)
hpx::init: std::exception caught: bind: Address already in use: HPX(network_error)
hpx::init: std::exception caught: bind: Address already in use: HPX(network_error)
hpx::init: std::exception caught: hpx::init: std::exception caught: bind: Address already in use: HPX(network_error)
hpx::init: std::exception caught: listen: Address already in use: HPX(network_error)
hpx::init: std::exception caught: listen: Address already in use: HPX(network_error)
hpx::init: std::exception caught: bind: Address already in use: HPX(network_error)
hpx::init: std::exception caught: bind: Address already in use: HPX(network_error)
bind: Address already in use: HPX(network_error)
Received Segmentation fault, 6 frames:
0x2ba3d5651dc9 : boost::stack_trace::trace(void**, int) + 0x9 in /fslhome/mwa2/parallex/install/lib/libhpx.so.0
0x2ba3d51e5b5f : boost::backtrace::backtrace(unsigned long) + 0x5f in /fslhome/mwa2/parallex/install/lib/libhpx.so.0
0x2ba3d51e5c24 : boost::trace() + 0x24 in /fslhome/mwa2/parallex/install/lib/libhpx.so.0
0x2ba3d51bc241 : hpx::detail::backtrace() + 0x11 in /fslhome/mwa2/parallex/install/lib/libhpx.so.0
0x2ba3d521660f : hpx::termination_handler(int) + 0x1f in /fslhome/mwa2/parallex/install/lib/libhpx.so.0
0x370d00e7c0 : ??? + 0xd00e7c0 in /lib64/libpthread.so.0
pbsdsh: task 0 exit status 262
pbsdsh: task 1 exit status 255
pbsdsh: task 2 exit status 255
pbsdsh: task 4 exit status 255
pbsdsh: task 5 exit status 255
pbsdsh: task 6 exit status 255
pbsdsh: task 7 exit status 255
pbsdsh: task 8 exit status 255
pbsdsh: task 9 exit status 255
pbsdsh: task 10 exit status 255
pbsdsh: task 11 exit status 255
pbsdsh: task 12 exit status 255
pbsdsh: task 13 exit status 255
pbsdsh: task 14 exit status 255
pbsdsh: task 15 exit status 255

@brycelelbach
Copy link
Member Author

[comment by manderson] [Trac time Thu Aug 25 17:13:04 2011] Some more information:

  cat $PBS_NODENUM
  hostname

returns:

cat: 0: No such file or directory
m6-1-1.local

@brycelelbach
Copy link
Member Author

[comment by manderson] [Trac time Thu Aug 25 20:06:02 2011] ```
Thu Aug 25 14:04:01 MDT 2011
MKLROOT=/opt/intel/mkl/10.2.5.035
MANPATH=/usr/mpi/fsl_openmpi_intel-1.4.2/man:/opt/intel/mkl/10.2.5.035/man/en_US:/opt/intel/Compiler/11.1/072/man/en_US:/opt/intel/Compiler/11.1/072/man/en_US:/usr/share/man:/opt/ganglia/man:/usr/local/man:/usr/local/share/man:/opt/rocks/man:/opt/pgi/linux86-64/7.1-6/man:/opt/pgi/linux86-64/10.5/man
HOSTNAME=m6-1-1.local
PBS_VERSION=TORQUE-2.5.5
IPPROOT=/opt/intel/Compiler/11.1/072/ipp/em64t
INTEL_LICENSE_FILE=/opt/intel/Compiler/11.1/072/licenses:/opt/intel/licenses:/fslhome/mwa2/intel/licenses:/opt/intel/Compiler/11.1/072/licenses:/opt/intel/licenses:/fslhome/mwa2/intel/licenses
SHELL=/usr/local/bin/bash
HISTSIZE=1000
PBS_JOBNAME=hpx_test
LIBRARY_PATH=/opt/intel/mkl/10.2.5.035/lib/em64t:/opt/intel/Compiler/11.1/072/lib/intel64:/opt/intel/Compiler/11.1/072/ipp/em64t/lib:/opt/intel/Compiler/11.1/072/tbb/intel64/cc4.1.0_libc2.4_kernel2.6.16.21/lib:/opt/intel/Compiler/11.1/072/lib/intel64:/opt/intel/Compiler/11.1/072/ipp/em64t/lib:/opt/intel/Compiler/11.1/072/tbb/intel64/cc4.1.0_libc2.4_kernel2.6.16.21/lib
SCHRODINGER=/fslapps/schrodinger/
PBS_ENVIRONMENT=PBS_BATCH
FPATH=/opt/intel/mkl/10.2.5.035/include
SDF_ROOT=/fslhome/mwa2/rnpl_gcc44
OLDPWD=/bluehome/mwa2
PBS_O_WORKDIR=/fslhome/mwa2/compute/1dtest
ANT_HOME=/opt/rocks
USER=mwa2
PBS_TASKNUM=1
LS_COLORS=
LD_LIBRARY_PATH=/fslhome/mwa2/rnpl_gcc44/lib:/usr/mpi/fsl_openmpi_intel-1.4.2/lib:/opt/intel/mkl/10.2.5.035/lib/em64t:/opt/intel/Compiler/11.1/072/lib/intel64:/opt/intel/Compiler/11.1/072/ipp/em64t/sharedlib:/opt/intel/Compiler/11.1/072/tbb/intel64/cc4.1.0_libc2.4_kernel2.6.16.21/lib:/opt/intel/Compiler/11.1/072/lib/intel64:/opt/intel/Compiler/11.1/072/ipp/em64t/sharedlib:/opt/intel/Compiler/11.1/072/tbb/intel64/cc4.1.0_libc2.4_kernel2.6.16.21/lib
PBS_O_HOME=/fslhome/mwa2
LIB=/opt/intel/Compiler/11.1/072/ipp/em64t/lib:/opt/intel/Compiler/11.1/072/ipp/em64t/lib:
ROCKS_ROOT=/opt/rocks
CPATH=/opt/intel/mkl/10.2.5.035/include:/opt/intel/Compiler/11.1/072/ipp/em64t/include:/opt/intel/Compiler/11.1/072/tbb/include:/opt/intel/Compiler/11.1/072/ipp/em64t/include:/opt/intel/Compiler/11.1/072/tbb/include
PBS_MOMPORT=15003
PBS_GPUFILE=/usr/spool/PBS/aux//4317327.fslsched.fsl.byu.edugpu
PBS_O_QUEUE=batch
NLSPATH=/opt/intel/mkl/10.2.5.035/lib/em64t/locale/%l_%t/%N:/opt/intel/Compiler/11.1/072/lib/intel64/locale/%l_%t/%N:/opt/intel/Compiler/11.1/072/ipp/em64t/lib/locale/%l_%t/%N:/opt/intel/Compiler/11.1/072/idb/intel64/locale/%l_%t/%N:/opt/intel/Compiler/11.1/072/lib/intel64/locale/%l_%t/%N:/opt/intel/Compiler/11.1/072/ipp/em64t/lib/locale/%l_%t/%N:/opt/intel/Compiler/11.1/072/idb/intel64/locale/%l_%t/%N
PGI=/opt/pgi
PATH=/fslhome/mwa2/cmake/bin:/opt/pgi/linux86-64/10.5/bin:/usr/mpi/fsl_openmpi_intel-1.4.2/bin:/usr/kerberos/bin:/usr/java/latest/bin:/opt/intel/Compiler/11.1/072/bin/intel64:/opt/intel/Compiler/11.1/072/bin/intel64:/bin:/usr/bin:/opt/ganglia/bin:/opt/ganglia/sbin:/opt/454/bin:/opt/rocks/bin:/opt/rocks/sbin:/opt/dell/srvadmin/bin
PBS_O_LOGNAME=mwa2
MAIL=/var/spool/mail/mwa2
PBS_O_LANG=en_US.iso885915
PBS_JOBCOOKIE=2F73083E778D6AB7B93E9262ECFC499E
MKL_NUM_THREADS=1
PWD=/fslhome/mwa2/compute/1dtest
INPUTRC=/etc/inputrc
JAVA_HOME=/usr/java/latest
LANG=en_US.iso885915
PBS_NODENUM=0
PBS_NUM_NODES=2
PBS_O_SHELL=/usr/local/bin/bash
PBS_SERVER=fslsched.fsl.byu.edu
PBS_JOBID=4317327.fslsched.fsl.byu.edu
LM_LICENSE_FILE=/opt/pgi/license.dat
ENVIRONMENT=BATCH
SSH_ASKPASS=/usr/libexec/openssh/gnome-ssh-askpass
HOME=/fslhome/mwa2
SHLVL=2
TCMALLOC_ROOT=/fslhome/mwa2/tcmalloc
PBS_O_HOST=m6int03
DYLD_LIBRARY_PATH=/opt/intel/Compiler/11.1/072/tbb/intel64/cc4.1.0_libc2.4_kernel2.6.16.21/lib:/opt/intel/Compiler/11.1/072/tbb/intel64/cc4.1.0_libc2.4_kernel2.6.16.21/lib
PBS_VNODENUM=0
PYTHONPATH=/fslhome/mwa2/parallex/trunk/hpx/python/hpx:/fslhome/mwa2/python_modules:
LOGNAME=mwa2
BOOST_ROOT=/fslhome/mwa2/boost_1_47_0
CVS_RSH=ssh
PBS_QUEUE=batch
PBS_O_MAIL=/var/spool/mail/mwa2
LESSOPEN=|/usr/bin/lesspipe.sh %s
OMP_NUM_THREADS=1
PBS_NUM_PPN=8
INCLUDE=/opt/intel/mkl/10.2.5.035/include:/opt/intel/Compiler/11.1/072/ipp/em64t/include:/opt/intel/Compiler/11.1/072/ipp/em64t/include:
PBS_NODEFILE=/usr/spool/PBS/aux//4317327.fslsched.fsl.byu.edu
G_BROKEN_FILENAMES=1
PBS_O_PATH=/opt/pgi/linux86-64/10.5/bin:/usr/mpi/fsl_openmpi_intel-1.4.2/bin:/usr/kerberos/bin:/usr/java/latest/bin:/opt/intel/Compiler/11.1/072/bin/intel64:/opt/intel/Compiler/11.1/072/bin/intel64:/usr/local/cuda/bin:/usr/local/bin:/bin:/usr/bin:/opt/ganglia/bin:/opt/ganglia/sbin:/opt/454/bin:/opt/rocks/bin:/opt/rocks/sbin:/opt/dell/srvadmin/bin
_=/usr/bin/printenv
Thu Aug 25 14:04:01 MDT 2011

@brycelelbach
Copy link
Member Author

[comment by manderson] [Trac time Fri Aug 26 17:06:02 2011] Here are the environments and hostnames on each core:


#####################################
PBS_VERSION=TORQUE-2.5.5
PBS_JOBNAME=hpx_test
PBS_ENVIRONMENT=PBS_BATCH
PBS_O_WORKDIR=/fslhome/mwa2/compute/1dtest
PBS_TASKNUM=2
PBS_O_HOME=/fslhome/mwa2
PBS_GPUFILE=/usr/spool/PBS/aux//4343505.fslsched.fsl.byu.edugpu
PBS_MOMPORT=15003
PBS_O_QUEUE=batch
PBS_O_LOGNAME=mwa2
PATH=/bin:/usr/bin
PBS_O_LANG=en_US.iso885915
PBS_JOBCOOKIE=89FA13A76169D17B2482FA3C0D28C451
PWD=/bluehome/mwa2
PBS_NODENUM=0
LANG=C
PBS_NUM_NODES=2
PBS_O_SHELL=/usr/local/bin/bash
PBS_SERVER=fslsched.fsl.byu.edu
PBS_JOBID=4343505.fslsched.fsl.byu.edu
ENVIRONMENT=BATCH
SHLVL=1
HOME=/fslhome/mwa2
PBS_O_HOST=m6int02
PBS_VNODENUM=0
PBS_QUEUE=batch
PBS_O_MAIL=/var/spool/mail/mwa2
PBS_NUM_PPN=8
PBS_NODEFILE=/usr/spool/PBS/aux//4343505.fslsched.fsl.byu.edu
PBS_O_PATH=/opt/pgi/linux86-64/10.5/bin:/usr/mpi/fsl_openmpi_intel-1.4.2/bin:/usr/kerberos/bin:/usr/java/latest/bin:/opt/intel/Compiler/11.1/072/bin/intel64:/opt/intel/Compiler/11.1/072/bin/intel64:/usr/local/cuda/bin:/usr/local/bin:/bin:/usr/bin:/opt/ganglia/bin:/opt/ganglia/sbin:/opt/454/bin:/opt/rocks/bin:/opt/rocks/sbin:/opt/dell/srvadmin/bin
_=/usr/bin/printenv
#####################################
m6-1-1
m6-1-1
m6-1-1
m6-1-1
m6-1-1
m6-1-1
m6-1-1
m6-1-1
m6-1-2
m6-1-2
m6-1-2
m6-1-2
m6-1-2
m6-1-2
m6-1-2
m6-1-2
#####################################
m6-1-1.local
#####################################
#####################################
#####################################
PBS_VERSION=TORQUE-2.5.5
PBS_JOBNAME=hpx_test
PBS_ENVIRONMENT=PBS_BATCH
PBS_O_WORKDIR=/fslhome/mwa2/compute/1dtest
PBS_TASKNUM=3
PBS_O_HOME=/fslhome/mwa2
PBS_GPUFILE=/usr/spool/PBS/aux//4343505.fslsched.fsl.byu.edugpu
PBS_MOMPORT=15003
PBS_O_QUEUE=batch
PBS_O_LOGNAME=mwa2
PATH=/bin:/usr/bin
PBS_O_LANG=en_US.iso885915
PBS_JOBCOOKIE=89FA13A76169D17B2482FA3C0D28C451
PWD=/bluehome/mwa2
PBS_NODENUM=0
LANG=C
PBS_NUM_NODES=2
PBS_O_SHELL=/usr/local/bin/bash
PBS_SERVER=fslsched.fsl.byu.edu
PBS_JOBID=4343505.fslsched.fsl.byu.edu
ENVIRONMENT=BATCH
SHLVL=1
HOME=/fslhome/mwa2
PBS_O_HOST=m6int02
PBS_VNODENUM=1
PBS_QUEUE=batch
PBS_O_MAIL=/var/spool/mail/mwa2
PBS_NUM_PPN=8
PBS_NODEFILE=/usr/spool/PBS/aux//4343505.fslsched.fsl.byu.edu
PBS_O_PATH=/opt/pgi/linux86-64/10.5/bin:/usr/mpi/fsl_openmpi_intel-1.4.2/bin:/usr/kerberos/bin:/usr/java/latest/bin:/opt/intel/Compiler/11.1/072/bin/intel64:/opt/intel/Compiler/11.1/072/bin/intel64:/usr/local/cuda/bin:/usr/local/bin:/bin:/usr/bin:/opt/ganglia/bin:/opt/ganglia/sbin:/opt/454/bin:/opt/rocks/bin:/opt/rocks/sbin:/opt/dell/srvadmin/bin
_=/usr/bin/printenv
#####################################
m6-1-1
m6-1-1
m6-1-1
m6-1-1
m6-1-1
m6-1-1
m6-1-1
m6-1-1
m6-1-2
m6-1-2
m6-1-2
m6-1-2
m6-1-2
m6-1-2
m6-1-2
m6-1-2
#####################################
m6-1-1.local
#####################################
#####################################
#####################################
PBS_VERSION=TORQUE-2.5.5
PBS_JOBNAME=hpx_test
PBS_ENVIRONMENT=PBS_BATCH
PBS_O_WORKDIR=/fslhome/mwa2/compute/1dtest
PBS_TASKNUM=4
PBS_O_HOME=/fslhome/mwa2
PBS_GPUFILE=/usr/spool/PBS/aux//4343505.fslsched.fsl.byu.edugpu
PBS_MOMPORT=15003
PBS_O_QUEUE=batch
PBS_O_LOGNAME=mwa2
PATH=/bin:/usr/bin
PBS_O_LANG=en_US.iso885915
PBS_JOBCOOKIE=89FA13A76169D17B2482FA3C0D28C451
PWD=/bluehome/mwa2
PBS_NODENUM=0
LANG=C
PBS_NUM_NODES=2
PBS_O_SHELL=/usr/local/bin/bash
PBS_SERVER=fslsched.fsl.byu.edu
PBS_JOBID=4343505.fslsched.fsl.byu.edu
ENVIRONMENT=BATCH
SHLVL=1
HOME=/fslhome/mwa2
PBS_O_HOST=m6int02
PBS_VNODENUM=2
PBS_QUEUE=batch
PBS_O_MAIL=/var/spool/mail/mwa2
PBS_NUM_PPN=8
PBS_NODEFILE=/usr/spool/PBS/aux//4343505.fslsched.fsl.byu.edu
PBS_O_PATH=/opt/pgi/linux86-64/10.5/bin:/usr/mpi/fsl_openmpi_intel-1.4.2/bin:/usr/kerberos/bin:/usr/java/latest/bin:/opt/intel/Compiler/11.1/072/bin/intel64:/opt/intel/Compiler/11.1/072/bin/intel64:/usr/local/cuda/bin:/usr/local/bin:/bin:/usr/bin:/opt/ganglia/bin:/opt/ganglia/sbin:/opt/454/bin:/opt/rocks/bin:/opt/rocks/sbin:/opt/dell/srvadmin/bin
_=/usr/bin/printenv
#####################################
#####################################
m6-1-1
m6-1-1
m6-1-1
m6-1-1
m6-1-1
m6-1-1
m6-1-1
m6-1-1
m6-1-2
m6-1-2
m6-1-2
m6-1-2
m6-1-2
m6-1-2
m6-1-2
m6-1-2
#####################################
PBS_VERSION=TORQUE-2.5.5
PBS_JOBNAME=hpx_test
PBS_ENVIRONMENT=PBS_BATCH
PBS_O_WORKDIR=/fslhome/mwa2/compute/1dtest
PBS_TASKNUM=10
PBS_O_HOME=/fslhome/mwa2
PBS_MOMPORT=15003
PBS_O_QUEUE=batch
PBS_O_LOGNAME=mwa2
PATH=/bin:/usr/bin
PBS_O_LANG=en_US.iso885915
PBS_JOBCOOKIE=89FA13A76169D17B2482FA3C0D28C451
PWD=/bluehome/mwa2
PBS_NODENUM=1
LANG=C
PBS_NUM_NODES=2
PBS_O_SHELL=/usr/local/bin/bash
PBS_SERVER=fslsched.fsl.byu.edu
PBS_JOBID=4343505.fslsched.fsl.byu.edu
ENVIRONMENT=BATCH
SHLVL=1
HOME=/fslhome/mwa2
PBS_O_HOST=m6int02
PBS_VNODENUM=8
PBS_QUEUE=batch
PBS_O_MAIL=/var/spool/mail/mwa2
PBS_NUM_PPN=8
PBS_O_PATH=/opt/pgi/linux86-64/10.5/bin:/usr/mpi/fsl_openmpi_intel-1.4.2/bin:/usr/kerberos/bin:/usr/java/latest/bin:/opt/intel/Compiler/11.1/072/bin/intel64:/opt/intel/Compiler/11.1/072/bin/intel64:/usr/local/cuda/bin:/usr/local/bin:/bin:/usr/bin:/opt/ganglia/bin:/opt/ganglia/sbin:/opt/454/bin:/opt/rocks/bin:/opt/rocks/sbin:/opt/dell/srvadmin/bin
_=/usr/bin/printenv
#####################################
m6-1-1.local
#####################################
#####################################
#####################################
#####################################
m6-1-2.local
#####################################
PBS_VERSION=TORQUE-2.5.5
PBS_JOBNAME=hpx_test
PBS_ENVIRONMENT=PBS_BATCH
PBS_O_WORKDIR=/fslhome/mwa2/compute/1dtest
PBS_TASKNUM=11
PBS_O_HOME=/fslhome/mwa2
PBS_MOMPORT=15003
PBS_O_QUEUE=batch
PBS_O_LOGNAME=mwa2
PATH=/bin:/usr/bin
PBS_O_LANG=en_US.iso885915
PBS_JOBCOOKIE=89FA13A76169D17B2482FA3C0D28C451
PWD=/bluehome/mwa2
PBS_NODENUM=1
LANG=C
PBS_NUM_NODES=2
PBS_O_SHELL=/usr/local/bin/bash
PBS_SERVER=fslsched.fsl.byu.edu
PBS_JOBID=4343505.fslsched.fsl.byu.edu
ENVIRONMENT=BATCH
SHLVL=1
HOME=/fslhome/mwa2
PBS_O_HOST=m6int02
PBS_VNODENUM=9
PBS_QUEUE=batch
PBS_O_MAIL=/var/spool/mail/mwa2
PBS_NUM_PPN=8
PBS_O_PATH=/opt/pgi/linux86-64/10.5/bin:/usr/mpi/fsl_openmpi_intel-1.4.2/bin:/usr/kerberos/bin:/usr/java/latest/bin:/opt/intel/Compiler/11.1/072/bin/intel64:/opt/intel/Compiler/11.1/072/bin/intel64:/usr/local/cuda/bin:/usr/local/bin:/bin:/usr/bin:/opt/ganglia/bin:/opt/ganglia/sbin:/opt/454/bin:/opt/rocks/bin:/opt/rocks/sbin:/opt/dell/srvadmin/bin
_=/usr/bin/printenv
#####################################
#####################################
PBS_VERSION=TORQUE-2.5.5
PBS_JOBNAME=hpx_test
PBS_ENVIRONMENT=PBS_BATCH
PBS_O_WORKDIR=/fslhome/mwa2/compute/1dtest
PBS_TASKNUM=5
PBS_O_HOME=/fslhome/mwa2
PBS_GPUFILE=/usr/spool/PBS/aux//4343505.fslsched.fsl.byu.edugpu
PBS_MOMPORT=15003
PBS_O_QUEUE=batch
PB#####################################
S_O_LOGNAME=mwa2
PATH=/bin:/usr/bin
PBS_O_LANG=en_US.iso885915
PBS_JOBCOOKIE=89FA13A76169D17B2482FA3C0D28C451
PWD=/bluehome/mwa2
PBS_NODENUM=0
LANG=C
PBS_NUM_NODES=2
PBS_O_SHELL=/usr/local/bin/bash
PBS_SERVER=fslsched.fsl.byu.edu
PBS_JOBID=4343505.fslsched.fsl.byu.edu
ENVIRONMENT=BATCH
SHLVL=1
HOME=/fslhome/mwa2
PBS_O_HOST=m6int02
PBS_VNODENUM=3
PBS_QUEUE=batch
PBS_O_MAIL=/var/spool/mail/mwa2
PBS_NUM_PPN=8
PBS_NODEFILE=/usr/spool/PBS/aux//4343505.fslsched.fsl.byu.edu
PBS_O_PATH=/opt/pgi/linux86-64/10.5/bin:#####################################
/usr/mpi/fsl_openmpi_intel-1.4.2/bin:/usr/kerberos/bin:/usr/java/latest/bin:/opt/intel/Compiler/11.1/072/bin/intel64:/opt/intel/Compiler/11.1/072/bin/intel64:/usr/local/cuda/bin:/usr/local/bin:/bin:/usr/bin:/opt/ganglia/bin:/opt/ganglia/sbin:/opt/454/bin:/opt/rocks/bin:/opt/rocks/sbin:/opt/dell/srvadmin/bin
_=/usr/bin/printenv
#####################################
m6-1-2.local
#####################################
m6-1-1
m6-1-1
m6-1-1
m6-1-1
m6-1-1
m6-1-1
m6-1-1
m6-1-1
m6-1-2
m6-1-2
m6-1-2
m6-1-2
m6-1-2
m6-1-2
m6-1-2
m6-1-2
#####################################
#####################################
#####################################
m6-1-1.local
PBS_VERSION=TORQUE-2.5.5
PBS_JOBNAME=hpx_test
PBS_ENVIRONMENT=PBS_BATCH
PBS_O_WORKDIR=/fslhome/mwa2/compute/1dtest
PBS_TASKNUM=12
PBS_O_HOME=/fslhome/mwa2
PBS_MOMPORT=15003
PBS_O_QUEUE=batch
PBS_O_LOGNAME=mwa2
PATH=/bin:/usr/bin
PBS_O_LANG=en_US.iso885915
PBS_JOBCOOKIE=89FA13A76169D17B2482FA3C0D28C451
PWD=/bluehome/mwa2
PBS_NODENUM=1
LANG=C
PBS_NUM_NODES=2
PBS_O_SHELL=/usr/local/bin/bash
PBS_SERVER=fslsched.fsl.byu.edu
PBS_JOBID=4343505.fslsched.fsl.byu.edu
ENVIRONMENT=BATCH
SHLVL=1
HOME=/fslhome/mwa2
PBS_O#####################################
_HOST=m6int02
PBS_VNODENUM=10
PBS_QUEUE=batch
PBS_O_MAIL=/var/spool/mail/mwa2
PBS_NUM_PPN=8
PBS_O_PATH=/opt/pgi/linux86-64/10.5/bin:/usr/mpi/fsl_openmpi_intel-1.4.2/bin:/usr/kerberos/bin:/usr/java/latest/bin:/opt/intel/Compiler/11.1/072/bin/intel64:/opt/intel/Compiler/11.1/072/bin/intel64:/usr/local/cuda/bin:/usr/local/bin:/bin:/usr/bin:/opt/ganglia/bin:/opt/ganglia/sbin:/opt/454/bin:/opt/rocks/bin:/opt/rocks/sbin:/opt/dell/srvadmin/bin
_=/usr/bin/printenv
#####################################
#####################################
#####################################
m6-1-2.local
#####################################
#####################################
#####################################
PBS_VERSION=TORQUE-2.5.5
PBS_JOBNAME=hpx_test
PBS_ENVIRONMENT=PBS_BATCH
PBS_O_WORKDIR=/fslhome/mwa2/compute/1dtest
PBS_TASKNUM=6
PBS_O_HOME=/fslhome/mwa2
PBS_GPUFILE=/usr/spool/PBS/aux//4343505.fslsched.fsl.byu.edugpu
PBS_MOMPORT=15003
PBS_O_QUEUE=batch
PBS_O_LOGNAME=mwa2
PATH=/bin:/usr/bin
PBS_O_LANG=en_US.iso885915
PBS_JOBCOOKIE=89FA13A76169D17B2482FA3C0D28C451
PWD=/bluehome/mwa2
PBS_NODENUM=0
LANG=C
PBS_NUM_NODES=2
PBS_O_SHELL=/usr/local/bin/bash
PBS_SERVER=fslsched.fsl.byu.edu
PBS_JOBID=4343505.fslsched.fsl.byu.edu
ENVIRONMENT=BATCH
SHLVL=1
HOME=/fslhome/mwa2
PBS_O_HOST=m6int02
PBS_VNODENUM=4
PBS_QUEUE=batch
PBS_O_MAIL=/var/spool/mail/mwa2
PBS_NUM_PPN=8
PBS_NODEFILE=/usr/spool/PBS/aux//4343505.fslsched.fsl.byu.edu
PBS_O_PATH=/opt/pgi/linux86-64/10.5/bin:/usr/mpi/fsl_openmpi_intel-1.4.2/bin:/usr/kerberos/bin:/usr/java/latest/bin:/opt/intel/Compiler/11.1/072/bin/intel64:/opt/intel/Compiler/11.1/072/bin/intel64:/usr/local/cuda/bin:/usr/local/bin:/bin:/usr/bin:/opt/ganglia/bin:/opt/ganglia/sbin:/opt/454/bin:/opt/rocks/bin:/opt/rocks/sbin:/opt/dell/srvadmin/bin
_=/usr/bin/printenv
#####################################
m6-1-1
m6-1-1
m6-1-1
m6-1-1
m6-1-1
m6-1-1
m6-1-1
m6-1-1
m6-1-2
m6-1-2
m6-1-2
m6-1-2
m6-1-2
m6-1-2
m6-1-2
m6-1-2
#####################################
PBS_VERSION=TORQUE-2.5.5
PBS_JOBNAME=hpx_test
PBS_ENVIRONMENT=PBS_BATCH
PBS_O_WORKDIR=/fslhome/mwa2/compute/1dtest
PBS_TASKNUM=13
PBS_O_HOME=/fslhome/mwa2
PBS_MOMPORT=15003
PBS_O_QUEUE=batch
PBS_O_LOGNAME=mwa2
PATH=/bin:/usr/bin
PBS_O_LANG=en_US.iso885915
PBS_JOBCOOKIE=89FA13A76169D17B2482FA3C0D28C451
PWD=/bluehome/mwa2
PBS_NODENUM=1
LANG=C
PBS_NUM_NODES=2
PBS_O_SHELL=/usr/local/bin/bash
PBS_SERVER=fslsched.fsl.byu.edu
PBS_JOBID=4343505.fslsched.fsl.byu.edu
ENVIRONMENT=BATCH
SHLVL=1
HOME=/fslhome/mwa2
PBS_O_HOST=m6int02
PBS_VNODENUM=11
PBS_QUEUE=batch
PBS_O_MAIL=/var/spool/mail/mwa2
PBS_NUM_PPN=8
PBS_O_PATH=/opt/pgi/linux86-64/10.5/bin:/usr/mpi/fsl_openmpi_intel-1.4.2/bin:/usr/kerberos/bin:/usr/java/latest/bin:/opt/intel/Compiler/11.1/072/bin/intel64:/opt/intel/Compiler/11.1/072/bin/intel64:/usr/local/cuda/bin:/usr/local/bin:/bin:/usr/bin:/opt/ganglia/bin:/opt/ganglia/sbin:/opt/454/bin:/opt/rocks/bin:/opt/rocks/sbin:/opt/dell/srvadmin/bin
_=/usr/bin/printenv
#####################################
#####################################
#####################################
#####################################
PBS_VERSION=TORQUE-2.5.5
PBS_JOBNAME=hpx_test
PBS_ENVIRONMENT=PBS_BATCH
PBS_O_WORKDIR=/fslhome/mwa2/compute/1dtest
PBS_TASKNUM=14
PBS_O_HOME=/fslhome/mwa2
PBS_MOMPORT=15003
PBS_O_QUEUE=batch
PBS_O_LOGNAME=mwa2
PATH=/bin:/usr/bin
PBS_O_LANG=en_US.iso885915
PBS_JOBCOOKIE=89FA13A76169D17B2482FA3C0D28C451
PWD=/bluehome/mwa2
PBS_NODENUM=1
LANG=C
PBS_NUM_NODES=2
PBS_O_SHELL=/usr/local/bin/bash
PBS_SERVER=fslsched.fsl.byu.edu
PBS_JOBID=4343505.fslsched.fsl.byu.edu
ENVIRONMENT=BATCH
SHLVL=1
HOME=/fslhome/mwa2
PBS_O_HOST=m6int02
PBS_VNODENUM=12
PBS_QUEUE=batch
PBS_O_MAIL=/var/spool/mail/mwa2
PBS_NUM_PPN=8
PBS_O_PATH=/opt/pgi/linux86-64/10.5/bin:/usr/mpi/fsl_openmpi_intel-1.4.2/bin:/usr/kerberos/bin:/usr/java/latest/bin:/opt/intel/Compiler/11.1/072/bin/intel64:/opt/intel/Compiler/11.1/072/bin/intel64:/usr/local/cuda/bin:/usr/local/bin:/bin:/usr/bin:/opt/ganglia/bin:/opt/ganglia/sbin:/opt/454/bin:/opt/rocks/bin:/opt/rocks/sbin:/opt/dell/srvadmin/bin
_=/usr/bin/printenv
#####################################
#####################################
m6-1-1.local
m6-1-2.local
#####################################
#####################################
PBS_VERSION=TORQUE-2.5.5
PBS_JOBNAME=hpx_test
PBS_ENVIRONMENT=PBS_BATCH
PBS_O_WORKDIR=/fslhome/mwa2/compute/1dtest
PBS_TASKNUM=7
PBS_O_HOME=/fslhome/mwa2
PBS_GPUFILE=/usr/spool/PBS/aux//4343505.fslsched.fsl.byu.edugpu
PBS_MOMPORT=15003
PBS_O_QUEUE=batch
PBS_O_LOGNAME=mwa2
PATH=/bin:/usr/bin
PBS_O_LANG=en_US.iso885915
PBS_JOBCOOKIE=89FA13A76169D17B2482FA3C0D28C451
PWD=/bluehome/mwa2
PBS_NODENUM=0
LANG=C
PBS_NUM_NODES=2
PBS_O_SHELL=/usr/local/bin/bash
PBS_SERVER=fslsched.fsl.byu.edu
PBS_JOBID=4343505.fslsched.fsl.byu.edu
ENVIRONMENT=BATCH
SHLVL=1
HOME=/fslhome/mwa2
PBS_O_HOST=m6int02
PBS_VNODENUM=5
PBS_QUEUE=batch
PBS_O_MAIL=/var/spool/mail/mwa2
PBS_NUM_PPN=8
PBS_NODEFILE=/usr/spool/PBS/aux//4343505.fslsched.fsl.byu.edu
PBS_O_PATH=/opt/pgi/linux86-64/10.5/bin:/usr/mpi/fsl_openmpi_intel-1.4.2/bin:/usr/kerberos/bin:/usr/java/latest/bin:/opt/intel/Compiler/11.1/072/bin/intel64:/opt/intel/Compiler/11.1/072/bin/intel64:/usr/local/cuda/bin:/usr/local/bin:/bin:/usr/bin:/opt/ganglia/bin:/opt/ganglia/sbin:/opt/454/bin:/opt/rocks/bin:/opt/rocks/sbin:/opt/dell/srvadmin/bin
_=/usr/bin/printenv
#####################################
#####################################
#####################################
m6-1-1
m6-1-1
m6-1-1
m6-1-1
m6-1-1
m6-1-1
m6-1-1
m6-1-1
m6-1-2
m6-1-2
m6-1-2
m6-1-2
m6-1-2
m6-1-2
m6-1-2
m6-1-2
m6-1-2.local
#####################################
#####################################
#####################################
#####################################
PBS_VERSION=TORQUE-2.5.5
PBS_JOBNAME=hpx_test
PBS_ENVIRONMENT=PBS_BATCH
PBS_O_WORKDIR=/fslhome/mwa2/compute/1dtest
PBS_TASKNUM=15
PBS_O_HOME=/fslhome/mwa2
PBS_MOMPORT=15003
PBS_O_QUEUE=batch
PBS_O_LOGNAME=mwa2
PATH=/bin:/usr/bin
PBS_O_LANG=en_US.iso885915
PBS_JOBCOOKIE=89FA13A76169D17B2482FA3C0D28C451
PWD=/bluehome/mwa2
PBS_NODENUM=1
LANG=C
PBS_NUM_NODES=2
PBS_O_SHELL=/usr/local/bin/bash
PBS_SERVER=fslsched.fsl.byu.edu
PBS_JOBID=4343505.fslsched.fsl.byu.edu
ENVIRONMENT=BATCH
SHLVL=1
HOME=/fslhome/mwa2
PBS_O_HOST=m6int02
PBS_VNODENUM=13
PBS_QUEUE=batch
PBS_O_MAIL=/var/spool/mail/mwa2
PBS_NUM_PPN=8
PBS_O_PATH=/opt/pgi/linux86-64/10.5/bin:/usr/mpi/fsl_openmpi_intel-1.4.2/bin:/usr/kerberos/bin:/usr/java/latest/bin:/opt/intel/Compiler/11.1/072/bin/intel64:/opt/intel/Compiler/11.1/072/bin/intel64:/usr/local/cuda/bin:/usr/local/bin:/bin:/usr/bin:/opt/ganglia/bin:/opt/ganglia/sbin:/opt/454/bin:/opt/rocks/bin:/opt/rocks/sbin:/opt/dell/srvadmin/bin
_=/usr/bin/printenv
m6-1-1.local
#####################################
#####################################
#####################################
#####################################
#####################################
m6-1-2.local
#####################################
#####################################
#####################################
#####################################
PBS_VERSION=TORQUE-2.5.5
PBS_JOBNAME=hpx_test
PBS_ENVIRONMENT=PBS_BATCH
PBS_O_WORKDIR=/fslhome/mwa2/compute/1dtest
PBS_TASKNUM=16
PBS_O_HOME=/fslhome/mwa2
PBS_MOMPORT=15003
PBS_O_QUEUE=batch
PBS_O_LOGNAME=mwa2
PATH=/bin:/usr/bin
PBS_O_LANG=en_US.iso885915
PBS_JOBCOOKIE=89FA13A76169D17B2482FA3C0D28C451
PWD=/bluehome/mwa2
PBS_NODENUM=1
LANG=C
PBS_NUM_NODES=2
PBS_O_SHELL=/usr/local/bin/bash
PBS_SERVER=fslsched.fsl.byu.edu
PBS_JOBID=4343505.fslsched.fsl.byu.edu
ENVIRONMENT=BATCH
SHLVL=1
HOME=/fslhome/mwa2
PBS_OPBS_VERSION=TORQUE-2.5.5
PBS_JOBNAME=hpx_test
PBS_ENVIRONMENT=PBS_BATCH
PBS_O_WORKDIR=/fslhome/mwa2/compute/1dtest
PBS_TASKNUM=8
PBS_O_HOME=/fslhome/mwa2
PBS_GPUFILE=/usr/spool/PBS/aux//4343505.fslsched.fsl.byu.edugpu
PBS_MOMPORT=15003
PBS_O_QUEUE=batch
PB_HOST=m6int02
PBS_VNODENUM=14
PBS_QUEUE=batch
PBS_O_MAIL=/var/spool/mail/mwa2
PBS_NUM_PPN=8
PBS_O_PATH=/opt/pgi/linux86-64/10.5/bin:/usr/mpi/fsl_openmpi_intel-1.4.2/bin:/usr/kerberos/bin:/usr/java/latest/bin:/opt/intel/Compiler/11.1/072/bin/intel64:/opt/inS_O_LOGNAME=mwa2
PATH=/bin:/usr/bin
PBS_O_LANG=en_US.iso885915
PBS_JOBCOOKIE=89FA13A76169D17B2482FA3C0D28C451
PWD=/bluehome/mwa2
PBS_NODENUM=0
LANG=C
PBS_NUM_NODES=2
PBS_O_SHELL=/usr/local/bin/bash
PBS_SERVER=fslsched.fsl.byu.edu
PBS_JOBID=4343505.fslschedtel/Compiler/11.1/072/bin/intel64:/usr/local/cuda/bin:/usr/local/bin:/bin:/usr/bin:/opt/ganglia/bin:/opt/ganglia/sbin:/opt/454/bin:/opt/rocks/bin:/opt/rocks/sbin:/opt/dell/srvadmin/bin
_=/usr/bin/printenv
#####################################
.fsl.byu.edu
ENVIRONMENT=BATCH
SHLVL=1
HOME=/fslhome/mwa2
PBS_O_HOST=m6int02
PBS_VNODENUM=6
PBS_QUEUE=batch
PBS_O_MAIL=/var/spool/mail/mwa2
PBS_NUM_PPN=8
PBS_NODEFILE=/usr/spool/PBS/aux//4343505.fslsched.fsl.byu.edu
PBS_O_PATH=/opt/pgi/linux86-64/10.5/bin:/usr/mpi/fsl_openmpi_intel-1.4.2/bin:/usr/kerberos/bin:/usr/java/latest/bin:/opt/intel/Compiler/11.1/072/bin/intel64:/opt/intel/Compiler/11.1/072/bin/intel64:/usr/local/cuda/bin:/usr/local/bin:/bin:/usr/bin:/opt/ganglia/bin:/opt/ganglia/sbin:/opt/454/bin:/opt/rocks/bin:/opt/rocks/sbin:/opt/dell/srvadmin/bin
_=/usr/bin/printenv
#####################################
#####################################
m6-1-1
m6-1-1
m6-1-1
m6-1-1
m6-1-1
m6-1-1
m6-1-1
m6-1-1
m6-1-2
m6-1-2
m6-1-2
m6-1-2
m6-1-2
m6-1-2
m6-1-2
m6-1-2
#####################################
m6-1-2.local
#####################################
m6-1-1.local
#####################################
#####################################
#####################################
#####################################
PBS_VERSION=TORQUE-2.5.5
PBS_JOBNAME=hpx_test
PBS_ENVIRONMENT=PBS_BATCH
PBS_O_WORKDIR=/fslhome/mwa2/compute/1dtest
PBS_TASKNUM=17
PBS_O_HOME=/fslhome/mwa2
PBS_MOMPORT=15003
PBS_O_QUEUE=batch
PBS_O_LOGNAME=mwa2
PATH=/bin:/usr/bin
PBS_O_LANG=en_US.iso885915
PBS_JOBCOOKIE=89FA13A76169D17B2482FA3C0D28C451
PWD=/bluehome/mwa2
PBS_NODENUM=1
LANG=C
PBS_NUM_NODES=2
PBS_O_SHELL=/usr/local/bin/bash
PBS_SERVER=fslsched.fsl.byu.edu
PBS_JOBID=4343505.fslsched.fsl.byu.edu
ENVIRONMENT=BATCH
SHLVL=1
HOME=/fslhome/mwa2
PBS_O_HOST=m6int02
PBS_VNODENUM=15
PBS_QUEUE=batch
PBS_O_MAIL=/var/spool/mail/mwa2
PBS_NUM_PPN=8
PBS_O_PATH=/opt/pgi/linux86-64/10.5/bin:/usr/mpi/fsl_openmpi_intel-1.4.2/bin:/usr/kerberos/bin:/usr/java/latest/bin:/opt/intel/Compiler/11.1/072/bin/intel64:/opt/intel/Compiler/11.1/072/bin/intel64:/usr/local/cuda/bin:/usr/local/bin:/bin:/usr/bin:/opt/ganglia/bin:/opt/ganglia/sbin:/opt/454/bin:/opt/rocks/bin:/opt/rocks/sbin:/opt/dell/srvadmin/bin
_=/usr/bin/printenv
#####################################
#####################################
#####################################
m6-1-2.local
#####################################
PBS_VERSION=TORQUE-2.5.5
PBS_JOBNAME=hpx_test
PBS_ENVIRONMENT=PBS_BATCH
PBS_O_WORKDIR=/fslhome/mwa2/compute/1dtest
PBS_TASKNUM=9
PBS_O_HOME=/fslhome/mwa2
PBS_GPUFILE=/usr/spool/PBS/aux//4343505.fslsched.fsl.byu.edugpu
PBS_MOMPORT=15003
PBS_O_QUEUE=batch
PBS_O_LOGNAME=mwa2
PATH=/bin:/usr/bin
PBS_O_LANG=en_US.iso885915
PBS_JOBCOOKIE=89FA13A76169D17B2482FA3C0D28C451
PWD=/bluehome/mwa2
PBS_NODENUM=0
LANG=C
PBS_NUM_NODES=2
PBS_O_SHELL=/usr/local/bin/bash
PBS_SERVER=fslsched.fsl.byu.edu
PBS_JOBID=4343505.fslsched.fsl.byu.edu
ENVIRONMENT=BATCH
SHLVL=1
HOME=/fslhome/mwa2
PBS_O_HOST=m6int02
PBS_VNODENUM=7
PBS_QUEUE=batch
PBS_O_MAIL=/var/spool/mail/mwa2
PBS_NUM_PPN=8
PBS_NODEFILE=/usr/spool/PBS/aux//4343505.fslsched.fsl.byu.edu
PBS_O_PATH=/opt/pgi/linux86-64/10.5/bin:/usr/mpi/fsl_openmpi_intel-1.4.2/bin:/usr/kerberos/bin:/usr/java/latest/bin:/opt/intel/Compiler/11.1/072/bin/intel64:/opt/intel/Compiler/11.1/072/bin/intel64:/usr/local/cuda/bin:/usr/local/bin:/bin:/usr/bin:/opt/ganglia/bin:/opt/ganglia/sbin:/opt/454/bin:/opt/rocks/bin:/opt/rocks/sbin:/opt/dell/srvadmin/bin
_=/usr/bin/printenv
#####################################
#####################################
m6-1-1
m6-1-1
m6-1-1
m6-1-1
m6-1-1
m6-1-1
m6-1-1
m6-1-1
m6-1-2
m6-1-2
m6-1-2
m6-1-2
m6-1-2
m6-1-2
m6-1-2
m6-1-2
#####################################
m6-1-1.local
#####################################
#####################################
Fri Aug 26 11:02:16 MDT 2011

@brycelelbach
Copy link
Member Author

[comment by hkaiser] [Trac time Mon Sep 12 22:59:55 2011] This is resolved now. Use the following commandline in the PBS script:

pbsdsh -u /your/app/name --nodes=`echo $PBS_NODEFILE` [...more app options...]

Using the command line option --ifsuffix=<...> allows to specify a common suffix, which when appended to the hostname, selects the proper interface to be used (i.e. infiniband, myrinet, etc.)

Alternatively (instead of --nodes=) you can first copy the node file to a place where it can be read from all nodes. Specify the name of the copied nodefile to HPX using --nodefile=....

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant