This repository was archived by the owner on Mar 20, 2023. It is now read-only.
-
Notifications
You must be signed in to change notification settings - Fork 42
Make MPI functions dynamically loadable #641
Merged
Merged
Changes from all commits
Commits
Show all changes
82 commits
Select commit
Hold shift + click to select a range
403e876
Let MPI function can be loaded dynamically
alkino 1c29034
Continue fixing
alkino 1e93850
One more time
alkino ae19353
Fix everything
alkino 2b001d9
clang-format
alkino ed23112
Fix include
alkino 9091bb7
One more
alkino c71f315
Fix GPU build: wrappers for nrnmpi_local_rank and nrnmpi_local_size
pramodk 1acc8b7
last part
alkino 27ee610
missing include
alkino 12cb38c
Fix?!?
alkino 675cb61
Fix again
alkino 328d111
Remove unused functions?
alkino 5672115
fix the include path
pramodk 39a1209
Resolved symbols only once: keep status in mpi_manager_t
pramodk 410cf22
generate lincorenrn_mpi.so in the lib directory
pramodk ae1ea48
Set set_tests_properties for DYLD/LD_LIBRARY_PATH
pramodk dfe7210
make cmake-format happy
pramodk 201f0f7
Enable ctest verbose and list libraries under lib
pramodk e3b6645
Use libcorenrn_mpi.dylib instead of libcorenrn_mpi.so on Mac OS
pramodk 594d733
use RTD_GLOBAL with dlopen, see details at https://users.open-mpi.nar…
pramodk e544ac7
disable lfp test for dynamic MPI; add status in the cmake build summa…
pramodk 9d84870
Clean cmake
alkino f623ef2
Load only once
alkino f01f2ee
Load only if --mpi is given
alkino 54b5945
mk_mech_init is called before solve_core
alkino a1c94a6
Protect mpi call at runtine if --mpi is not given of command line
alkino 50a668b
clang-format
alkino 039f00c
More clang-format
alkino 90b0007
nrnmpi_use is useless
alkino 4c807cd
don't simplify if mpi_nrank < 2
alkino c7422db
Delete everything about bbs
alkino 3100f4e
Remove _worlds
alkino 1cff4de
Cleaning
alkino 75610c4
clang-format
alkino 3f7577d
Compile as STATIC with dynamic mpi ;)
alkino 11c965e
No more problem with localgid_size_
alkino f0a03d2
Stop sharing spfixin_ovfl_
alkino 58e4814
numprocs
alkino d9cb700
Use local nrnmpi_initialized_impl
alkino 26996a2
nrnmpi_myid
alkino 397ea51
ag_send_nspike
alkino ac21ffb
clang-format ftw
alkino 4492a27
Split myid and nprocs
alkino 32919e4
Merge remote-tracking branch 'origin/master'
alkino 665755b
Merge remote-tracking branch 'origin/master' into HEAD
alkino 9ad356e
nin_
alkino 34cb0a9
ovfl_capacity
alkino aa63ce0
spikeout
alkino 29102b6
icapacity
alkino a2225c4
spikein
alkino 8b73b7b
spikeout_fixed
alkino 1be5e1e
ad_seng_size
alkino 02177c2
spikein_fixed
alkino b1f4f83
ovfl
alkino 86db950
nout
alkino e84d87c
extern corenrn_param already in header
alkino 01dcdad
less GLOB in cmake
alkino fd30b2b
Clean enginemech
alkino 5d8efa8
Error if dynamic=on when mpi=off
alkino 52b0b9c
No more symbols_resolved because includes are cleaner
alkino 7816f2f
No more debug output on stdio
alkino cec8423
memcpy => std::memcpy
alkino 55bee34
More explanation for lfp test disabled
alkino 950d5d1
no more mkdynam.sh
alkino 0783cae
Fix bad index
alkino 994897a
Fix typo
alkino a39f239
Use static_cast instead of c-cast
alkino b895f5e
no more make_tuple
alkino 74c6294
move nrnmpi_def_cinc.h to nrnmpi_dev_cinc.cpp
alkino d52dd56
More fixes
alkino d8e2710
clang-format
alkino ec03d18
Always use _impl version of functions inside mpi/lib
alkino 1f4de0d
cmake-format
0527574
DYNAMIC_MPI need to be well separated and linked when libcoreneuron i…
alkino 7d56b2f
Let's work around problem of weird building for ever
alkino 16aad7a
std::tuple inside extern "C" is not correct
alkino facb737
Merge remote-tracking branch 'origin/master'
alkino c784d1f
Delete unused pgvts_op
alkino f2c63fd
Take suffix of libcorenrn_mpi from cmake
alkino d3f99fa
clang-format cmake-format
alkino c809890
protect a call to nrnmpi_int_alltoallv
alkino File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -14,6 +14,7 @@ | |
|
|
||
| #include <cstring> | ||
| #include <climits> | ||
| #include <dlfcn.h> | ||
|
pramodk marked this conversation as resolved.
|
||
| #include <memory> | ||
| #include <vector> | ||
|
|
||
|
|
@@ -45,6 +46,7 @@ | |
| #include "coreneuron/io/file_utils.hpp" | ||
| #include "coreneuron/io/nrn2core_direct.h" | ||
| #include "coreneuron/io/core2nrn_data_return.hpp" | ||
| #include "coreneuron/utils/utils.hpp" | ||
|
|
||
| extern "C" { | ||
| const char* corenrn_version() { | ||
|
|
@@ -201,7 +203,7 @@ void nrn_init_and_load_data(int argc, | |
|
|
||
| // if multi-threading enabled, make sure mpi library supports it | ||
| #if NRNMPI | ||
| if (corenrn_param.threading) { | ||
| if (corenrn_param.mpi_enable && corenrn_param.threading) { | ||
| nrnmpi_check_threading_support(); | ||
| } | ||
| #endif | ||
|
|
@@ -448,14 +450,35 @@ std::unique_ptr<ReportHandler> create_report_handler(ReportConfiguration& config | |
|
|
||
| using namespace coreneuron; | ||
|
|
||
| #if NRNMPI | ||
| #define STRINGIFY(x) #x | ||
| #define TOSTRING(x) STRINGIFY(x) | ||
| static void* load_dynamic_mpi() { | ||
| dlerror(); | ||
| void* handle = dlopen("libcorenrn_mpi" TOSTRING(CMAKE_SHARED_LIBRARY_SUFFIX), | ||
| RTLD_NOW | RTLD_GLOBAL); | ||
| const char* error = dlerror(); | ||
| if (error) { | ||
| std::string err_msg = std::string("Could not open dynamic MPI library: ") + error + "\n"; | ||
| throw std::runtime_error(err_msg); | ||
| } | ||
| return handle; | ||
| } | ||
| #endif | ||
|
|
||
| extern "C" void mk_mech_init(int argc, char** argv) { | ||
| // read command line parameters and parameter config files | ||
| corenrn_param.parse(argc, argv); | ||
|
|
||
| #if NRNMPI | ||
| if (corenrn_param.mpi_enable) { | ||
| nrnmpi_init(&argc, &argv); | ||
| #ifdef CORENRN_ENABLE_DYNAMIC_MPI | ||
| auto mpi_handle = load_dynamic_mpi(); | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Fine for now, but this part will need better integration with NEURON. |
||
| mpi_manager().resolve_symbols(mpi_handle); | ||
| #endif | ||
| auto ret = nrnmpi_init(&argc, &argv); | ||
| nrnmpi_numprocs = ret.numprocs; | ||
| nrnmpi_myid = ret.myid; | ||
| } | ||
| #endif | ||
|
|
||
|
|
@@ -514,7 +537,9 @@ extern "C" int run_solve_core(int argc, char** argv) { | |
| mkdir_p(output_dir.c_str()); | ||
| } | ||
| #if NRNMPI | ||
| nrnmpi_barrier(); | ||
| if (corenrn_param.mpi_enable) { | ||
| nrnmpi_barrier(); | ||
| } | ||
| #endif | ||
| bool compute_gpu = corenrn_param.gpu; | ||
| bool skip_mpi_finalize = corenrn_param.skip_mpi_finalize; | ||
|
|
@@ -643,7 +668,7 @@ extern "C" int run_solve_core(int argc, char** argv) { | |
|
|
||
| // mpi finalize | ||
| #if NRNMPI | ||
| if (!skip_mpi_finalize) { | ||
| if (corenrn_param.mpi_enable && !skip_mpi_finalize) { | ||
| nrnmpi_finalize(); | ||
| } | ||
| #endif | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.