Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

openmpi: 3.1.3 -> 4.0.0 #54877

Merged
merged 10 commits into from Feb 22, 2019

Conversation

@markuskowa
Copy link
Member

commented Jan 29, 2019

Motivation for this change

This update removes some deprecated MPI-1 symbols. This breaks some applications still using these outdated symbols. Fortunately it is a simple function name change in most cases.

Upgrade guide: https://www.open-mpi.org/software/ompi/major-changes.php
Removed symbols: https://www.open-mpi.org/faq/?category=mpi-removed

CC potentially affected maintainers @costrouc @abbradar @bzizou

Things done
  • Add upstream patch to mpi4py
  • patch deprecated functions in scalapack
  • Add upstream patch to globalarrays
  • Add upstream patches to hfd5-mpi
  • added an entry to the release notes
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
    -> ~+3MB
  • Assured whether relevant documentation is up to date
  • Fits CONTRIBUTING.md.

@markuskowa

This comment has been minimized.

Copy link
Member Author

commented Jan 29, 2019

@GrahamcOfBorg build python36Packages.mpi4py python36Packages.tensorflow
@GrahamcOfBorg build lammps-mpi dl-poly-classic-mpi siesta-mpi quantum-espresso-mpi

@dotlambda
Copy link
Member

left a comment

You should add the pythonPackages. prefix to the last commit message.

@markuskowa markuskowa force-pushed the markuskowa:upd-openmpi4 branch from a694e95 to 75e59ec Jan 29, 2019

@markuskowa

This comment has been minimized.

Copy link
Member Author

commented Jan 29, 2019

@GrahamcOfBorg build python36Packages.mpi4py
@GrahamcOfBorg build python36Packages.tensorflow
@GrahamcOfBorg build lammps-mpi
@GrahamcOfBorg build dl-poly-classic-mpi
@GrahamcOfBorg build siesta-mpi
@GrahamcOfBorg build quantum-espresso-mpi
@GrahamcOfBorg build gromacsDoubleMpi

@markuskowa markuskowa requested review from matthewbauer and Mic92 Feb 1, 2019

@markuskowa markuskowa force-pushed the markuskowa:upd-openmpi4 branch from 75e59ec to d5a6573 Feb 5, 2019

@markuskowa

This comment has been minimized.

Copy link
Member Author

commented Feb 6, 2019

Maybe I beat the timeout this time
@GrahamcOfBorg build scalapack

@markuskowa markuskowa force-pushed the markuskowa:upd-openmpi4 branch from b7549da to fe19395 Feb 7, 2019

@nixos-discourse

This comment has been minimized.

Copy link

commented Feb 19, 2019

This pull request has been mentioned on Nix community. There might be relevant details there:

https://discourse.nixos.org/t/nixos-19-03-feature-freeze/1950/30

@markuskowa markuskowa force-pushed the markuskowa:upd-openmpi4 branch from fe19395 to d48465c Feb 19, 2019

@markuskowa

This comment has been minimized.

Copy link
Member Author

commented Feb 20, 2019

@GrahamcOfBorg build scalapack

@FRidh

FRidh approved these changes Feb 21, 2019

Copy link
Member

left a comment

Looks good to me. In case we find packages that we can't trivially fix we can still include 3.1.3, but that's for then.

@markuskowa markuskowa force-pushed the markuskowa:upd-openmpi4 branch from d48465c to f3c1b3e Feb 21, 2019

openmpi: 3.1.3 -> 4.0.0
openmpi-4.0.0 removes deprecated MPI-1 symbols.
For upgrade guide see:
https://www.open-mpi.org/faq/?category=mpi-removed

markuskowa added some commits Jan 28, 2019

release-notes: add note about openmpi-4.0.0 upgrade.
Add link to openmpi upgrade guide, regarding deprecated MPI-1 symbols.
scalapack: update meta data
* Set platform from linux -> x86_64-linux
  Scalapack does not seem to run properly on ARM (failing tests)

* Add markuskowa as maintainer
openmpi: add patch for mpi tests on machines with less than 4 cores
Fix a bug that ignores OMPI_MCA_rmaps_base_oversubscribe (upstream patch).
This bug breaks the test from libs, such as scalapack,
on machines with less than 4 cores.

@markuskowa markuskowa force-pushed the markuskowa:upd-openmpi4 branch from 3ff8c45 to 99ca454 Feb 21, 2019

@markuskowa

This comment has been minimized.

Copy link
Member Author

commented Feb 21, 2019

@FRidh That was my first thought to include the old version as well but I am optimistic that it should not be necessary.

@markuskowa

This comment has been minimized.

Copy link
Member Author

commented Feb 21, 2019

@GrahamcOfBorg build hdf5-mpi

@markuskowa

This comment has been minimized.

Copy link
Member Author

commented Feb 21, 2019

The PR would be ready for merging.

@markuskowa markuskowa merged commit 5c8c258 into NixOS:master Feb 22, 2019

14 of 19 checks passed

globalarrays, hdf5, openmpi, scalapack on aarch64-linux Failure
Details
hdf5-mpi on aarch64-linux Failure
Details
hf5-mpi on aarch64-linux No attempt
Details
hf5-mpi on x86_64-darwin No attempt
Details
hf5-mpi on x86_64-linux No attempt
Details
globalarrays, hdf5, openmpi, scalapack on x86_64-darwin Success
Details
globalarrays, hdf5, openmpi, scalapack on x86_64-linux Success
Details
grahamcofborg-eval ^.^!
Details
grahamcofborg-eval-check-maintainers matching changed paths to changed attrs...
Details
grahamcofborg-eval-check-meta config.nix: checkMeta = true
Details
grahamcofborg-eval-nixos-manual nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="ofborg"; } ./nixos/release.nix -A manual
Details
grahamcofborg-eval-nixos-options nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="ofborg"; } ./nixos/release.nix -A options
Details
grahamcofborg-eval-nixpkgs-manual nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="ofborg"; } ./pkgs/top-level/release.nix -A manual
Details
grahamcofborg-eval-nixpkgs-tarball nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="ofborg"; } ./pkgs/top-level/release.nix -A tarball
Details
grahamcofborg-eval-nixpkgs-unstable-jobset nix-instantiate --arg nixpkgs { outPath=./.; revCount=999999; shortRev="ofborg"; } ./pkgs/top-level/release.nix -A unstable
Details
grahamcofborg-eval-package-list nix-env -qa --json --file .
Details
grahamcofborg-eval-package-list-no-aliases nix-env -qa --json --file . --arg config { allowAliases = false; }
Details
hdf5-mpi on x86_64-darwin Success
Details
hdf5-mpi on x86_64-linux Success
Details

@markuskowa markuskowa deleted the markuskowa:upd-openmpi4 branch Feb 22, 2019

@costrouc

This comment has been minimized.

Copy link
Contributor

commented Feb 22, 2019

@markuskowa thank you thank you for this work. I will greatly benefit from this PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.