-
Notifications
You must be signed in to change notification settings - Fork 113
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
bugfix: Distribute mulitcompartment reaction fluxes in MPI. #728
Conversation
Previously only currents where being gathered by all nodes. In order to avoid double counting rxd currents, the currents generated by rxd are also distributed to all nodes.
@adamjhn : seems like builds are failing because of IV submodule update. If it's not needed in this PR, you can revert IV to older commit and we can fix IV specific build issues separately. |
@adamjhn : I didn't change anything. Seems like it was temporary issue on travis which got resolved. |
In travis CI log I am randomly seeing:
I thought we discussed and workaround it in #559 (comment). For now, I am just re-triggering failed build. |
* bugfix: Distribute mulitcompartment reaction fluxes in MPI. Previously only currents were being gathered by all nodes. In order to avoid double counting rxd currents, the currents generated by rxd are also distributed to all nodes. * Fixed some initialization problems. * Update test results due to a change in when multicompartment fluxes are added. * Fixed a memory problem in the variable step method. * Fix an initialization problem. * Update test data. * Revert external/iv changes.
* bugfix: Distribute mulitcompartment reaction fluxes in MPI. Previously only currents were being gathered by all nodes. In order to avoid double counting rxd currents, the currents generated by rxd are also distributed to all nodes. * Fixed some initialization problems. * Update test results due to a change in when multicompartment fluxes are added. * Fixed a memory problem in the variable step method. * Fix an initialization problem. * Update test data. * Revert external/iv changes.
* Two factors contribute to the above solution: - New eigen branch (version 3.5 and above). Currently we are using a mirrored version of Eigen in BlueBrain organisation https://github.com/BlueBrain/eigen/releases/tag/v3.5-alpha - An API that makes possible to call any Eigen `__device__` function from within OpenACC regions. * More details: Eigen-3.5+ provides better GPU support; however, some functions cannot be called directly from within OpenACC regions. Therefore, we need to wrap them in a special API (decorate them with `__device__` & `acc routine` tokens), which allows us to eventually call them from OpenACC. Calling these functions from CUDA kernels presents no issue. * From #726: Avoid use `[]` operator with eigen Matrix objects. This results into runtime error with OpenACC and PGI compiler. * Note that this should works in combination with BlueBrain/CoreNeuron/pull/624 fixes #311 fixes #135
Previously only currents where being gathered by all nodes. In order to avoid double counting rxd currents, the currents generated by rxd are also distributed to all nodes.