parcel port serialization error -- graph500

[brycelelbach]

[reported by manderson] [Trac time Wed Mar 14 22:54:31 2012] While running the graph500 problem 1b5a6d0 in distributed, I get the following error ONLY when running IB over IP:

[what]: types are not compatible: destination_type(21) action_type(4) parcel (({0000000100000001, 0000000000090231}:(192.168.170.172:7910:component_base_lco_with_value[4]:0x24ef1b0):set_result_action_bool)): HPX(bad_component_type): HPX(serialization_error)
[version]: V0.8.0-trunk (AGAS: V2.1), SVN: 7426M

[date]: Mar 14 2012 10:55:02

[compiler]: GNU C++ version 4.4.0 20090514 (Red Hat 4.4.0-6)
[stdlib]: GNU libstdc++ version 20090514

When running without IB over IP, everything works fine.

Here is the pbs submission script with IB:

!/bin/bash

---------------------------

PBS -l nodes=4:ppn=8:ib,pmem=2gb,walltime=00:05:00

PBS -N hpx_test

PBS -m bea

PBS -M matt@phys.lsu.edu

---------------------------

date
source ~/.bashrc
cd $PBS_O_WORKDIR
pbsdsh -v -u /fslhome/mwa2/compute/graph/graph500_client --hasedgeroot 1 --hpx:nodes=cat $PBS_NODEFILE --hpx:ifsuffix=ib --hpx:iftransform=s/.local$// --hpx:debug-clp
date
exit 0

Here it is without:

!/bin/bash

---------------------------

PBS -l nodes=4:ppn=8,pmem=2gb,walltime=00:05:00

PBS -N hpx_test

PBS -m bea

PBS -M matt@phys.lsu.edu

---------------------------

date
source ~/.bashrc
cd $PBS_O_WORKDIR
pbsdsh -v -u /fslhome/mwa2/compute/graph/graph500_client --hasedgeroot 1 --hpx:nodes=cat $PBS_NODEFILE --hpx:debug-clp
date
exit 0

This exact same IB same test worked on Saturday. Any suggestions?

[brycelelbach]

[comment by hkaiser] [Trac time Thu Mar 15 00:03:04 2012] You said it happens with 1b5a6d0, however the exception syas, it's f8ff93a. Not that I believe this to be the cause for the error, but you might want to try a full rebuild.

[brycelelbach]

[comment by manderson] [Trac time Thu Mar 15 00:36:21 2012] full rebuild fixes the problem -- sorry for the noise