combblas symbols multiple definition in z_c2cpp_GetHWPM.cpp.o and d_c2cpp_GetHWPM.cpp.o #60
Comments
|
The same problem still occurs if I leave |
|
I don't see the problem. At the top of CombBLAS.h, there is the following guard: #ifndef COMBBLAS_H This guarantees only one copy is included (the first time it is #include'ed by some file). Similarly, at the top of ApproxWeightPerfectMatching.h, there is the followng: #ifndef ApproxWeightPerfectMatching_h |
|
It is a bit strange. Those guards in the header files would only avoid double definition in the one .o file. The problem here is linking two .o files. Still, that kind of linking is perfectly usual, doesn't generate "multiple definition" errors from linking other object files. Will have to dig deeper. |
|
Most of the symbols in the error message are marked as type B in the object files, e.g. That must be a clue. Usually symbols from external libraries are marked U not B. Looks like my CombBLAS build might be faulty. In fact looks those symbols are not actually in my libCombBLAS.so at all (they're in the CombBLAS/BipartiteMatchings header files but not in the .so), so I'll need to fix that. |
|
I think it's a peculiarity of these BipartiteMatchings headers. BipartiteMatchings is a separate application using CombBLAS, so the symbols should not be in libCombBLAS.so itself. The conflicting symbols are defined in ApproxWeightPerfectMatching.h and BPMaximalMatching.h, not merely declared. Hence those definitions are embedding into both d_c2cpp_GetHWPM.cpp.o and z_c2cpp_GetHWPM.cpp.o, causing the multiple definition error when the two object files are linked. Should we check versions? I'm using CombBLAS 1.6.2 (CombBLAS_beta_16_2.tgz from https://people.eecs.berkeley.edu/~aydin/CombBLAS/html/index.html). Is that the version you're working with? |
|
@RizzerOnGitHub Were you able to compile SuperLU_Dist v 6.3.1 with CombBLAS v1.6.2? I'm on an Ubuntu system but am running into the same compilation errors that you showed above. I've tried different versions of both superlu_dist and CombBLAS. For all of them, I can compile CombBLAS without issue. For the versions of superlu_dist, I either cannot compile them or all of ctest's test fail due to seg-faults. I can write out the details of what worked with what if that's helpful. |
|
Still not building here. I tried the |
|
@JustinClough which permutation gave you a successful build (albeit with segfaults) ? |
|
The conflicting variables are related to performance monitoring. The following patch is applied to the Install BipartiteMatchings headers for SuperLU_DIST. Removes global variables
and code related to performance measurement that is not useful when used in a
library setting.
--- a/BipartiteMatchings/ApproxWeightPerfectMatching.h
+++ b/BipartiteMatchings/ApproxWeightPerfectMatching.h
@@ -9,7 +9,7 @@
#ifndef ApproxWeightPerfectMatching_h
#define ApproxWeightPerfectMatching_h
-#include "../CombBLAS.h"
+#include "CombBLAS.h"
#include "BPMaximalMatching.h"
#include "BPMaximumMatching.h"
#include <parallel/algorithm>
@@ -39,9 +39,6 @@
std::shared_ptr<CommGrid> commGrid;
};
-double t1Comp, t1Comm, t2Comp, t2Comm, t3Comp, t3Comm, t4Comp, t4Comm, t5Comp, t5Comm, tUpdateMateComp;
-
-
template <class IT, class NT>
std::vector<std::tuple<IT,IT,NT>> ExchangeData(std::vector<std::vector<std::tuple<IT,IT,NT>>> & tempTuples, MPI_Comm World)
{
@@ -391,7 +388,7 @@
-int ThreadBuffLenForBinning(int itemsize, int nbins)
+inline int ThreadBuffLenForBinning(int itemsize, int nbins)
{
// 1MB shared cache (per 2 cores) in KNL
#ifndef L2_CACHE_SIZE
@@ -417,7 +414,6 @@
- double tstart = MPI_Wtime();
MPI_Comm World = param.commGrid->GetWorld();
@@ -528,9 +524,6 @@
}
}
- t1Comp = MPI_Wtime() - tstart;
- tstart = MPI_Wtime();
-
// Step 3: Communicate data
std::vector<int> recvcnt (param.nprocs);
@@ -548,7 +541,6 @@
std::vector< std::tuple<IT,IT,NT> > recvTuples1(totrecv);
MPI_Alltoallv(sendTuples.data(), sendcnt.data(), sdispls.data(), MPI_tuple, recvTuples1.data(), recvcnt.data(), rdispls.data(), MPI_tuple, World);
MPI_Type_free(&MPI_tuple);
- t1Comm = MPI_Wtime() - tstart;
return recvTuples1;
}
@@ -730,9 +722,6 @@
// Step 4: Communicate data
- t2Comp = MPI_Wtime() - tstart;
- tstart = MPI_Wtime();
-
std::vector<int> recvcnt (param.nprocs);
std::vector<int> rdispls (param.nprocs, 0);
@@ -748,7 +737,6 @@
std::vector< std::tuple<IT,IT,IT,NT> > recvTuples1(totrecv);
MPI_Alltoallv(sendTuples.data(), sendcnt.data(), sdispls.data(), MPI_tuple, recvTuples1.data(), recvcnt.data(), rdispls.data(), MPI_tuple, World);
MPI_Type_free(&MPI_tuple);
- t2Comm = MPI_Wtime() - tstart;
return recvTuples1;
}
@@ -836,7 +824,6 @@
param.myrank = myrank;
param.commGrid = commGrid;
- double t1CompAll = 0, t1CommAll = 0, t2CompAll = 0, t2CommAll = 0, t3CompAll = 0, t3CommAll = 0, t4CompAll = 0, t4CommAll = 0, t5CompAll = 0, t5CommAll = 0, tUpdateMateCompAll = 0, tUpdateWeightAll = 0;
// -----------------------------------------------------------
// replicate mate vectors for mateCol2Row
@@ -975,11 +962,7 @@
}
//vector< tuple<IT,IT,IT, NT> >().swap(recvTuples1);
- double t3Comp = MPI_Wtime() - tstart;
- tstart = MPI_Wtime();
recvTuples1 = ExchangeData1(tempTuples1, World);
- double t3Comm = MPI_Wtime() - tstart;
- tstart = MPI_Wtime();
std::vector<std::tuple<IT,IT,IT,IT, NT>> bestTuplesPhase4 (lncol);
// we could have used lnrow in both bestTuplesPhase3 and bestTuplesPhase4
@@ -1041,14 +1024,9 @@
//vector< tuple<IT,IT,IT, NT> >().swap(recvTuples1);
- double t4Comp = MPI_Wtime() - tstart;
- tstart = MPI_Wtime();
std::vector<std::tuple<IT,IT,IT,IT>> recvWinnerTuples = ExchangeData1(winnerTuples, World);
- double t4Comm = MPI_Wtime() - tstart;
- tstart = MPI_Wtime();
-
// at the owner of (mj,j)
std::vector<std::tuple<IT,IT>> rowBcastTuples(recvWinnerTuples.size()); //(mi,mj)
std::vector<std::tuple<IT,IT>> colBcastTuples(recvWinnerTuples.size()); //(j,i)
@@ -1065,15 +1043,10 @@
colBcastTuples[k] = std::make_tuple(j,i);
rowBcastTuples[k] = std::make_tuple(mj,mi);
}
- double t5Comp = MPI_Wtime() - tstart;
- tstart = MPI_Wtime();
std::vector<std::tuple<IT,IT>> updatedR2C = MateBcast(rowBcastTuples, RowWorld);
std::vector<std::tuple<IT,IT>> updatedC2R = MateBcast(colBcastTuples, ColWorld);
- double t5Comm = MPI_Wtime() - tstart;
- tstart = MPI_Wtime();
-
#ifdef THREADED
#pragma omp parallel for
#endif
@@ -1095,13 +1068,9 @@
}
- double tUpdateMateComp = MPI_Wtime() - tstart;
- tstart = MPI_Wtime();
// update weights of matched edges
// we can do better than this since we are doing sparse updates
ReplicateMateWeights(param, dcsc, colptr, RepMateC2R, RepMateWR2C, RepMateWC2R);
- double tUpdateWeight = MPI_Wtime() - tstart;
-
weightPrev = weightCur;
weightCur = MatchingWeight(RepMateWC2R, RowWorld, minw);
@@ -1110,32 +1079,8 @@
//UpdateMatching(mateRow2Col, mateCol2Row, RepMateR2C, RepMateC2R);
//CheckMatching(mateRow2Col,mateCol2Row);
- if(myrank==0)
- {
- std::cout << t1Comp << " " << t1Comm << " "<< t2Comp << " " << t2Comm << " " << t3Comp << " " << t3Comm << " " << t4Comp << " " << t4Comm << " " << t5Comp << " " << t5Comm << " " << tUpdateMateComp << " " << tUpdateWeight << std::endl;
-
- t1CompAll += t1Comp;
- t1CommAll += t1Comm;
- t2CompAll += t2Comp;
- t2CommAll += t2Comm;
- t3CompAll += t3Comp;
- t3CommAll += t3Comm;
- t4CompAll += t4Comp;
- t4CommAll += t4Comm;
- t5CompAll += t5Comp;
- t5CommAll += t5Comm;
- tUpdateMateCompAll += tUpdateMateComp;
- tUpdateWeightAll += tUpdateWeight;
-
- }
}
- if(myrank==0)
- {
- std::cout << "=========== overal timing ==========" << std::endl;
- std::cout << t1CompAll << " " << t1CommAll << " " << t2CompAll << " " << t2CommAll << " " << t3CompAll << " " << t3CommAll << " " << t4CompAll << " " << t4CommAll << " " << t5CompAll << " " << t5CommAll << " " << tUpdateMateCompAll << " " << tUpdateWeightAll << std::endl;
- }
-
// update the distributed mate vectors from replicated mate vectors
UpdateMatching(mateRow2Col, mateCol2Row, RepMateR2C, RepMateC2R);
//weightCur = MatchingWeight(RepMateWC2R, RowWorld);
--- a/BipartiteMatchings/BPMaximalMatching.h
+++ b/BipartiteMatchings/BPMaximalMatching.h
@@ -1,7 +1,7 @@
#ifndef BP_MAXIMAL_MATCHING_H
#define BP_MAXIMAL_MATCHING_H
-#include "../CombBLAS.h"
+#include "CombBLAS.h"
#include <iostream>
#include <functional>
#include <algorithm>
@@ -14,8 +14,6 @@
#define GREEDY 1
#define KARP_SIPSER 2
#define DMD 3
-MTRand GlobalMT(123); // for reproducible result
-double tTotalMaximal;
namespace combblas {
@@ -25,7 +25,7 @@
void MaximalMatching(Par_DCSC_Bool & A, Par_DCSC_Bool & AT, FullyDistVec<IT, IT>& mateRow2Col,
FullyDistVec<IT, IT>& mateCol2Row, FullyDistVec<IT, IT>& degColRecv, int type, bool rand=true)
{
-
+ static MTRand GlobalMT(123); // for reproducible result
typedef VertexTypeML < IT, IT> VertexType;
int nprocs, myrank;
MPI_Comm_size(MPI_COMM_WORLD,&nprocs);
@@ -354,8 +354,6 @@
}
- tTotalMaximal = MPI_Wtime() - tStart;
-
IT cardinality = mateRow2Col.Count([](IT mate){return mate!=-1;});
std::vector<double> totalTimes(timing[0].size(),0);
for(int i=0; i<timing.size(); i++)
--- a/BipartiteMatchings/BPMaximumMatching.h
+++ b/BipartiteMatchings/BPMaximumMatching.h
@@ -1,7 +1,7 @@
#ifndef BP_MAXIMUM_MATCHING_H
#define BP_MAXIMUM_MATCHING_H
-#include "../CombBLAS.h"
+#include "CombBLAS.h"
#include <mpi.h>
#include <sys/time.h>
#include <iostream>
@@ -11,7 +11,6 @@
#include <string>
#include <sstream>
#include "MatchingDefs.h"
-double tTotalMaximum;
namespace combblas {
@@ -231,7 +231,7 @@
void maximumMatching(SpParMat < IT, NT, DER > & A, FullyDistVec<IT, IT>& mateRow2Col,
FullyDistVec<IT, IT>& mateCol2Row, bool prune=true, bool randMM = false, bool maximizeWeight = false)
{
-
+ static MTRand GlobalMT(123); // for reproducible result
typedef VertexTypeMM <IT> VertexType;
int nthreads=1;
@@ -420,8 +420,6 @@
MPI_Win_free(&winLeaves);
- tTotalMaximum = MPI_Wtime() - tstart;
-
//isMaximalmatching(A, mateRow2Col, mateCol2Row, unmatchedRow, unmatchedCol);
//isMatching(mateCol2Row, mateRow2Col); //todo there is a better way to check this
--- a/BipartiteMatchings/MatchingDefs.h
+++ b/BipartiteMatchings/MatchingDefs.h
@@ -9,7 +9,7 @@
#ifndef MatchingDefs_h
#define MatchingDefs_h
-#include "../CombBLAS.h"
+#include "CombBLAS.h"
#include <iostream>
namespace combblas {
--- a/BipartiteMatchings/Utility.h
+++ b/BipartiteMatchings/Utility.h
@@ -1,7 +1,7 @@
#ifndef BP_UTILITY_H
#define BP_UTILITY_H
-#include "../CombBLAS.h"
+#include "CombBLAS.h"
namespace combblas {
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -68,6 +68,7 @@ set_property(TARGET CombBLAS PROPERTY VERSION ${CombBLAS_VERSION})
# installation
install(DIRECTORY include/ DESTINATION include)
install(DIRECTORY psort-1.0/include/ DESTINATION include)
+install(DIRECTORY BipartiteMatchings DESTINATION include FILES_MATCHING PATTERN "*.h")
install(TARGETS CombBLAS EXPORT CombBLASTargets
LIBRARY DESTINATION lib
ARCHIVE DESTINATION lib
|
Fixes use in latest versions of SuperLU_DIST. See e.g. xiaoyeli/superlu_dist#60 * gnu/packages/patches/combinatorial-blas-awpm.patch: Remove globals related to performance measurement. Declare non-template function inline.
Sorry for the delayed response, @drew-parsons. I was able to compile the following SLUD and CombBlas versions but tests would fail with seg faults:
I did also try SLUD 6.3.1 but ran into compiler issues (multiple definitions of I did get my seg fault issue sorted out and now have SLUD 5.4.0 with CB 2.0.0 installed on my machine. The issue was that I had different index sizes defined for different libraries. In my working configuration, I have |
|
Thanks @bavier , @mbakke , your guix patch on CombBlas works a charm. After rebuilding a patched combblas, superlu-dist 6.4 now builds for me. @JustinClough , looks like bavier's patch is what we need. |
|
@JustinClough With 64-bit integers, you only need them for extremely large systems (billions of degrees of freedom). Many systems are smaller than that (millions of DOFs or less), in which case conventional 32-bit integers may give better performance (In terms of memory bandwidth, two 32-bit index values transfer for every one 64-bit index). Certainly your libraries need to be consistent with each other, either conventional 32-bit indexing (LP64) or 64-bit indexing (ILP64). Some supercomputing facilities offer both, you can load one or the other. In Debian/Ubuntu we've starting building 64-bit variants for some libraries, but we haven't done that for SuperLU-Dist yet. |
|
The CombBLAS patch is holding up fine. It's passing it own tests, https://ci.debian.net/packages/c/combblas/ SuperLU-Dist 6.4.0 is reliably building against the patched CombBLAS, https://tests.reproducible-builds.org/debian/rb-pkg/unstable/amd64/superlu-dist.html I'm happy to accept @bavier's CombBLAS patch as a resolution. There's nothing more to do in SuperLU-Dist, so I'll close this Issue now. |
superlu-dist 6.3.1 is failing to build on Debian unstable, failing to link because of multiple definitions, e.g.
I can see why its happening: AWPM_CombBLAS.hpp has been split into precision-dependent z_c2cpp_GetHWPM.cpp and d_c2cpp_GetHWPM.cpp, and each of them includes combblas and ApproxWeightPerfectMatching.h.
But I'm not certain what should be done about it. What do you recommend?
Why would it be happening on my system and not yours? Would it be compiler dependent? I'm using gcc 9.3.0.
I've applied a patch to swap
#include "ApproxWeightPerfectMatching.h"with#include "BipartiteMatchings/ApproxWeightPerfectMatching.h"with include flag given as-I/usr/include/CombBLAS/, since the BipartiteMatchings dir is placed under /usr/include/CombBLAS/ in the Debian installation. I wouldn't have thought that would cause this problem but would appreciate being corrected if BipartiteMatchings placement is more delicate than I realise.The failing link command is
build-superlu-dist.log
Also attaching the build log.
The text was updated successfully, but these errors were encountered: