-
Notifications
You must be signed in to change notification settings - Fork 218
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
not compiling with CUDA 11.3 #3591
Comments
Hello @cbontoiu thanks for your report. We've just got a similar report for alpaka (which is used inside PIConGPU) with CUDA 11.3. So it may be that CUDA 11.3 is the issue (you are also using it according to the log), not gcc 9.3. Note that alpaka only officially supports up to CUDA 11.2 so far. Could you try with an earlier CUDA version? |
@cbontoiu If you have installed the CUDA 11.3 driver on your system you can simply compile PIConGPU with CUDA 11.2 and run it on a system with CUDA 11.3 driver. |
@psychocoderHPC This option is interesting, but I don't know how should I apply it. I thought compilation goes with the CUDA version used at the installation of PIConGPU (in my case with Spack but using CUDA and CUDA aware openMPI from the system). In the meantime I had a fresh install of PIConGPU, dev version, using CUDA 11.2 and openMPI 4.1.1 (from the system) but the latter one gives some errors at the compilations shown in the text file attached. You may want to investigate this incompatibility as well. Thank you. |
Hello @cbontoiu . The issue is that your build uses two MPI libraries and that causes a conflict. One is your system openMPI. And another one is MPI at anaconda, that is pulled by ADIOS in anaconda. I think it is reasonable to approach it gradually. First disable ADIOS and try without it. |
After that works, you could either rebuild that ADIOS thing or tell it to use your system openMPI and thus avoid the conflict |
@sbastrakov Thank you. Indeed there was a clash between conda and spack and I managed to compile disabling the lines written by anaconda in my bashrc file. I don't know another way. I also managed to run the LWFA model and I am surprised how slow it was. This model used to complete in 1 min and 10 secs on this machine before the openpmd plugin was included. But now out of the box the running with the 1.cfg file took 3 mins and 25 seconds. |
PIConGPU: 0.6.0-dev Third party: |
source $HOME/src/spack/share/spack/setup-env.sh && spack load picongpu && spack load openpmd-api && export PIC_BACKEND="cuda:75" && export OMPI_MCA_io=^ompio |
If you want to see how the output affects run time, you could disable it in your .cfg file. Or change the period of output. Look for the TBG_openPMD variable there |
Indeed, 1 min and 25 secs can be spared when running without the creation of check points. |
Do you keep a change log file from version 0.5.0 such that we could have an idea of where the developing is pointing and maybe suggest features? |
Yes, here. Feature suggestions and external contributions are welcome. Ideally please create an new issue per suggestion. |
We work on a fix for CUDA 11.3 alpaka-group/alpaka#1295 |
Hello,
The
picongpu@develop
does not compile withgcc@9.3.0
. The compilation of the LWFA example ended up with this error:There are some incompatibilities when using the newest gcc 9 compilers and the error message is also mentioned here: https://gcc.gnu.org/gcc-9/changes.html
P.S. In fact I discovered later that the same error appears when compiling
picongpu
(master version) withgcc@7.5.0
log_out.txt
Regards,
Cristian
The text was updated successfully, but these errors were encountered: