Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MPI test fails locally #406

Closed
cjayb opened this issue Jul 30, 2021 · 6 comments · Fixed by #545
Closed

MPI test fails locally #406

cjayb opened this issue Jul 30, 2021 · 6 comments · Fixed by #545
Labels
bug Something isn't working maint This issue or pull request already exists
Milestone

Comments

@cjayb
Copy link
Collaborator

cjayb commented Jul 30, 2021

I've run across this in current master when running test_parallel_backends.py:test_run_mpibackend_oversubscribed, can any other macOS users confirm?

E           RuntimeError: MPI simulation failed. Return code: 143

hnn_core/parallel_backends.py:235: RuntimeError
----------------------------- Captured stdout call -----------------------------
MPI will run over 12 processes
Running 2 trials...
numprocs=12

This is running on my 4-core Air, so I'd expect 1.5 * 4 == 6 cores, not 12. My environment is:

mpi                       1.0                     openmpi    conda-forge
mpi4py                    3.0.3            py38h368b8e5_5    conda-forge
openmpi                   4.1.0                h5807002_1    conda-forge
neuron                    7.8.2                    pypi_0    pypi
@jasmainak
Copy link
Collaborator

I have this issue too. Not sure what's the right fix, we might have dig up the pull request by Blake and see what was the intention.

Also, are you sure it's 4 logical cores?

@jasmainak jasmainak added bug Something isn't working maint This issue or pull request already exists labels Jul 30, 2021
@cjayb
Copy link
Collaborator Author

cjayb commented Jul 30, 2021

Also, are you sure it's 4 logical cores?

DUH! No, it's a 1.2 GHz Quad-Core Intel Core i7, so the 12 virtual cores is correct.

@jasmainak
Copy link
Collaborator

@cjayb I can't seem to reproduce this error any more. Do you still have this issue? I am now installing mpi4py on Mac by doing the following:

First install openmpi-mpicc:

$ conda install -c conda-forge openmpi-mpicc

Next, do a pip install:

$ pip install mpi4py

The old command:

$ conda install openmpi mpi4py

doesn't seem to work for me any more (conda claims it's installed but pip and ipython don't see it)

@rythorpe
Copy link
Contributor

I still have this issue on Ubuntu. openmpi is installed globally at root but is not installed as a python package.

@cjayb
Copy link
Collaborator Author

cjayb commented Sep 3, 2021

Still have the problem with mpi and mpi4py from conda-forge... I need to upgrade to Big Sur soon, so will need to create new envs anyway...

@jasmainak
Copy link
Collaborator

moving to 0.3 ... let's wait for some users to hit this issue

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working maint This issue or pull request already exists
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants