Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NRN_ENABLE_SHARED=OFF builds static library (for GPU support) #682

Merged
merged 4 commits into from
Jul 27, 2020

Conversation

pramodk
Copy link
Member

@pramodk pramodk commented Jul 26, 2020

  • libnrnmech is now build as static library if NRN_ENABLE_SHARED=OFF
  • this is required for GPU support with OpenACC because global
    variables from shared library are not initialized

Also related to issues mentioned in #646

  - libnrnmech is now build as static library if NRN_ENABLE_SHARED=OFF
  - this is required for GPU support with OpenACC because global
    variables from shared library are not initialized

Also related to issues mentioned in #646
@pramodk pramodk requested a review from nrnhines July 26, 2020 14:03
@pramodk
Copy link
Member Author

pramodk commented Jul 26, 2020

This now allows to embed GPU enabled CoreNEURON into NEURON as:

# build mechanisms for coreneuron
/gpfs/bbp.cscs.ch/project/proj16/kumbhar/pramod_scratch/nrn/build_gpu_mod2c_python2/install/bin/nrnivmodl-core .

# link coreneuron with mechanisms into neuron
/gpfs/bbp.cscs.ch/project/proj16/kumbhar/pramod_scratch/nrn/build_gpu_mod2c_python2/install/bin/nrnivmodl \
-loadflags \
"-acc -rdynamic -ldl -lrt \
-Wl,--whole-archive -Lx86_64/ -lcorenrnmech \
/gpfs/bbp.cscs.ch/ssd/apps/hpc/jenkins/deploy/external-libraries/2020-02-01/linux-rhel7-x86_64/gcc-8.3.0/cuda-10.1.243-aekzngafo3/lib64/libcudart_static.a \
-L/gpfs/bbp.cscs.ch/project/proj16/kumbhar/pramod_scratch/nrn/build_gpu_mod2c_python2/install/lib \
-lcoreneuron -lcudacoreneuron  \
-Wl,--no-whole-archive \
-I/gpfs/bbp.cscs.ch/ssd/apps/hpc/jenkins/deploy/tools/2020-02-01/linux-rhel7-x86_64/gcc-8.3.0/hpe-mpi-2.21-7pbszh6v5u/include \
-L/gpfs/bbp.cscs.ch/ssd/apps/hpc/jenkins/deploy/tools/2020-02-01/linux-rhel7-x86_64/gcc-8.3.0/hpe-mpi-2.21-7pbszh6v5u/lib \
-lmpi++ -lmpi" .

cc: @iomaganaris

This is still not clean but now have all information required to implement clean solution (as separate PR).

bin/nrnivmodl.in Outdated
@@ -172,7 +172,7 @@ fi
@NRN_BINARY_SPECIAL_FALSE@ mdir='${NRNHOME}'/share/nrn/demo/release/${MODSUBDIR}
@NRN_BINARY_SPECIAL_FALSE@fi
@NRN_BINARY_SPECIAL_FALSE@@USING_CMAKE_FALSE@make -j 4 -f "$MAKEFILEDIR/nrnmech_makefile" "ROOT=${prefix}" "MODOBJFILES=$MODLO" "COBJFILES=$CLO" "UserLDFLAGS=$UserLDFLAGS" "UserINCFLAGS=$UserINCFLAGS" libnrnmech.la
@NRN_BINARY_SPECIAL_FALSE@@USING_CMAKE_TRUE@make -j 4 -f "$MAKEFILEDIR/nrnmech_makefile" "ROOT=${prefix}" "MODOBJFILES=$MODOBJS" "COBJFILES=$COBJS" "UserLDFLAGS=$UserLDFLAGS" "UserINCFLAGS=$UserINCFLAGS" libnrnmech.la VERBOSE=1
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@nrnhines : was there any specific need to have libnrnmech.la target here?

@pramodk
Copy link
Member Author

pramodk commented Jul 26, 2020

With this PR I have seen:

(cd .. ;  g++ -O2 -DVERSION_INFO='8.0.dev160' -std=c++11  -I /home/vsts/work/1/s/nrn_test_venv_35/lib/python3.5/site-packages/neuron/.data/include -I /home/vsts/work/1/s/nrn_test_venv_35/lib/python3.5/site-packages/neuron/.data/include/nrncvode -DAUTO_DLOPEN_NRNMECH=0 /home/vsts/work/1/s/nrn_test_venv_35/lib/python3.5/site-packages/neuron/.data/share/nrn/nrnmain.cpp -o x86_64/special \
  -L x86_64 -lnrnmech -L/home/vsts/work/1/s/nrn_test_venv_35/lib/python3.5/site-packages/neuron/.data/lib -lnrniv -lnrnmech -L/home/vsts/work/1/s/nrn_test_venv_35/lib/python3.5/site-packages/neuron/.data/lib -lnrniv -Wl,-rpath,'$ORIGIN' -Wl,-rpath,/home/vsts/work/1/s/nrn_test_venv_35/lib/python3.5/site-packages/neuron/.data/lib   )
/home/vsts/work/1/s/nrn_test_venv_35/lib/python3.5/site-packages/neuron/.data/lib/libnrniv.so: undefined reference to `pthread_create'
/home/vsts/work/1/s/nrn_test_venv_35/lib/python3.5/site-packages/neuron/.data/lib/libnrniv.so: undefined reference to `pthread_join'

and also below for static library build:

 => LINKING static library x86_64/libnrnmech.a Mod files: CaDynamics_E2.mod
 => LINKING executable x86_64/special LDFLAGS are:    -lreadline -lcurses -lform /gpfs/bbp.cscs.ch/project/proj16/kumbhar/pramod_scratch/soft/install/linux-rhel7-x86_64/gcc-8.3.0/python-2.7.16-5x2gzf/lib/libpython2.7.so -Wl,-rpath,/gpfs/bbp.cscs.ch/project/proj16/kumbhar/pramod_scratch/soft/install/linux-rhel7-x86_64/gcc-8.3.0/python-2.7.16-5x2gzf/lib -lpthread /gpfs/bbp.cscs.ch/ssd/apps/hpc/jenkins/deploy/tools/2020-02-01/linux-rhel7-x86_64/gcc-8.3.0/hpe-mpi-2.21-7pbszh6v5u/lib/libmpi.so -Wl,-rpath,/gpfs/bbp.cscs.ch/ssd/apps/hpc/jenkins/deploy/tools/2020-02-01/linux-rhel7-x86_64/gcc-8.3.0/hpe-mpi-2.21-7pbszh6v5u/lib
/gpfs/bbp.cscs.ch/project/proj16/kumbhar/pramod_scratch/nrn/build_gpu_mod2c_python2/install/share/nrn/nrnmain.cpp:
/usr/bin/ld: /gpfs/bbp.cscs.ch/project/proj16/kumbhar/pramod_scratch/nrn/build_gpu_mod2c_python2/install/lib/libnrniv.a(init.c.o): undefined reference to symbol 'dlclose@@GLIBC_2.2.5'
//usr/lib64/libdl.so.2: error adding symbols: DSO missing from command line

Working on fix locally.

Copy link
Member

@nrnhines nrnhines left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This kind of thing seems to be almost entirely experimental and one never knows a priori when a different linker or machine or style (wheel, package, etc.) will need a slightly differen tweak. I don't know whether it would ever be possible or worthwhile to (try to) eliminate certain types of library dependency.

@nrnhines nrnhines merged commit 9782fcc into master Jul 27, 2020
@nrnhines nrnhines deleted the pramodk/nrnivmodl-static-lib branch July 27, 2020 10:45
olupton added a commit that referenced this pull request Dec 7, 2022
* Enable GitLab logfile upload.
* Install GCC-8 on Mac OS to fix the Azure CI
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants