Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

openmpi: 3.1.3 -> 4.0.0 #50675

Closed
wants to merge 1 commit into from

Conversation

Projects
None yet
4 participants
@r-ryantm
Copy link
Contributor

commented Nov 19, 2018

Semi-automatic update generated by https://github.com/ryantm/nixpkgs-update tools. This update was made based on information from https://repology.org/metapackage/openmpi/versions.

meta.description for openmpi is: '"Open source MPI-3 implementation"'.

Checks done (click to expand)
  • built on NixOS
  • /nix/store/qg1vdpwrwk39j7xpnp4sr2fd2g4x7w80-openmpi-4.0.0/bin/ortecc passed the binary check.
  • /nix/store/qg1vdpwrwk39j7xpnp4sr2fd2g4x7w80-openmpi-4.0.0/bin/mpirun passed the binary check.
  • /nix/store/qg1vdpwrwk39j7xpnp4sr2fd2g4x7w80-openmpi-4.0.0/bin/mpiexec passed the binary check.
  • /nix/store/qg1vdpwrwk39j7xpnp4sr2fd2g4x7w80-openmpi-4.0.0/bin/ompi-clean passed the binary check.
  • /nix/store/qg1vdpwrwk39j7xpnp4sr2fd2g4x7w80-openmpi-4.0.0/bin/ompi-server passed the binary check.
  • /nix/store/qg1vdpwrwk39j7xpnp4sr2fd2g4x7w80-openmpi-4.0.0/bin/mpicc passed the binary check.
  • /nix/store/qg1vdpwrwk39j7xpnp4sr2fd2g4x7w80-openmpi-4.0.0/bin/mpic++ passed the binary check.
  • /nix/store/qg1vdpwrwk39j7xpnp4sr2fd2g4x7w80-openmpi-4.0.0/bin/mpicxx passed the binary check.
  • Warning: no invocation of /nix/store/qg1vdpwrwk39j7xpnp4sr2fd2g4x7w80-openmpi-4.0.0/bin/mpifort had a zero exit code or showed the expected version
  • Warning: no invocation of /nix/store/qg1vdpwrwk39j7xpnp4sr2fd2g4x7w80-openmpi-4.0.0/bin/mpif77 had a zero exit code or showed the expected version
  • Warning: no invocation of /nix/store/qg1vdpwrwk39j7xpnp4sr2fd2g4x7w80-openmpi-4.0.0/bin/mpif90 had a zero exit code or showed the expected version
  • /nix/store/qg1vdpwrwk39j7xpnp4sr2fd2g4x7w80-openmpi-4.0.0/bin/mpiCC passed the binary check.
  • Warning: no invocation of /nix/store/qg1vdpwrwk39j7xpnp4sr2fd2g4x7w80-openmpi-4.0.0/bin/opal_wrapper had a zero exit code or showed the expected version
  • /nix/store/qg1vdpwrwk39j7xpnp4sr2fd2g4x7w80-openmpi-4.0.0/bin/orte-clean passed the binary check.
  • Warning: no invocation of /nix/store/qg1vdpwrwk39j7xpnp4sr2fd2g4x7w80-openmpi-4.0.0/bin/orted had a zero exit code or showed the expected version
  • /nix/store/qg1vdpwrwk39j7xpnp4sr2fd2g4x7w80-openmpi-4.0.0/bin/orterun passed the binary check.
  • /nix/store/qg1vdpwrwk39j7xpnp4sr2fd2g4x7w80-openmpi-4.0.0/bin/orte-info passed the binary check.
  • /nix/store/qg1vdpwrwk39j7xpnp4sr2fd2g4x7w80-openmpi-4.0.0/bin/orte-server passed the binary check.
  • /nix/store/qg1vdpwrwk39j7xpnp4sr2fd2g4x7w80-openmpi-4.0.0/bin/ompi_info passed the binary check.
  • 14 of 19 passed binary check by having a zero exit code.
  • 0 of 19 passed binary check by having the new version present in output.
  • found 4.0.0 with grep in /nix/store/qg1vdpwrwk39j7xpnp4sr2fd2g4x7w80-openmpi-4.0.0
  • directory tree listing: https://gist.github.com/186b23dec4b049140af52259dd30d0eb
  • du listing: https://gist.github.com/74325e493284757f37cc01f9b195a5b7
Rebuild report (click to expand)

106 total rebuild path(s)

32 package rebuild(s)

32 x86_64-linux rebuild(s)
27 i686-linux rebuild(s)
20 x86_64-darwin rebuild(s)
27 aarch64-linux rebuild(s)

First fifty rebuilds by attrpath
cntk
dl-poly-classic-mpi
freecad
globalarrays
gromacsDoubleMpi
gromacsMpi
haskellPackages.haskell-mpi
hdf5-mpi
ior
lammps-mpi
netcdf-mpi
neuron-full
neuron-mpi
openmolcas
openmpi
parmetis
python27Packages.cntk
python27Packages.h5py-mpi
python27Packages.mpi4py
python27Packages.neuron-mpi
python27Packages.neurotools
python36Packages.cntk
python36Packages.dftfit
python36Packages.h5py-mpi
python36Packages.lammps-cython
python36Packages.mpi4py
python36Packages.neuron-mpi
quantum-espresso-mpi
raxml-mpi
scalapack
scotch
siesta-mpi

Instructions to test this update (click to expand)

Either download from Cachix:

nix-store -r /nix/store/qg1vdpwrwk39j7xpnp4sr2fd2g4x7w80-openmpi-4.0.0 \
  --option binary-caches 'https://cache.nixos.org/ https://r-ryantm.cachix.org/' \
  --option trusted-public-keys '
  r-ryantm.cachix.org-1:gkUbLkouDAyvBdpBX0JOdIiD2/DP1ldF3Z3Y6Gqcc4c=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(r-ryantm's Cachix cache is only trusted for this store-path realization.)

Or, build yourself:

nix-build -A openmpi https://github.com/r-ryantm/nixpkgs/archive/01fa37bd79db8bb8eb53afc64994a7a5f5ee7a5a.tar.gz

After you've downloaded or built it, look at the files and if there are any, run the binaries:

ls -la /nix/store/qg1vdpwrwk39j7xpnp4sr2fd2g4x7w80-openmpi-4.0.0
ls -la /nix/store/qg1vdpwrwk39j7xpnp4sr2fd2g4x7w80-openmpi-4.0.0/bin

cc @markuskowa for testing.

openmpi: 3.1.3 -> 4.0.0
Semi-automatic update generated by
https://github.com/ryantm/nixpkgs-update tools. This update was made
based on information from
https://repology.org/metapackage/openmpi/versions
@GrahamcOfBorg

This comment has been minimized.

Copy link

commented Nov 19, 2018

Success on x86_64-linux (full log)

Attempted: openmpi

Partial log (click to expand)

shrinking /nix/store/qg1vdpwrwk39j7xpnp4sr2fd2g4x7w80-openmpi-4.0.0/lib/libmpi_usempif08.so.40.20.0
shrinking /nix/store/qg1vdpwrwk39j7xpnp4sr2fd2g4x7w80-openmpi-4.0.0/lib/openmpi/libompi_dbg_msgq.so
gzipping man pages under /nix/store/qg1vdpwrwk39j7xpnp4sr2fd2g4x7w80-openmpi-4.0.0/share/man/
strip is /nix/store/vcc4svb8gy29g4pam2zja6llkbcwsyiq-binutils-2.30/bin/strip
stripping (with command strip and flags -S) in /nix/store/qg1vdpwrwk39j7xpnp4sr2fd2g4x7w80-openmpi-4.0.0/lib  /nix/store/qg1vdpwrwk39j7xpnp4sr2fd2g4x7w80-openmpi-4.0.0/bin
patching script interpreter paths in /nix/store/qg1vdpwrwk39j7xpnp4sr2fd2g4x7w80-openmpi-4.0.0
checking for references to /build in /nix/store/qg1vdpwrwk39j7xpnp4sr2fd2g4x7w80-openmpi-4.0.0...
cannot link '/nix/store/.links/17s8pmqxpb98r5jb9nl4nl492xzivma1k8i63hr5r6cry0cqz8bq' to '/nix/store/qg1vdpwrwk39j7xpnp4sr2fd2g4x7w80-openmpi-4.0.0/share/man/man3/MPI_Win_flush_local.3.gz': No space left on device
cannot link '/nix/store/.links/0k0b5jr516jlj1xrzn4vanb2z2jqbrvadcgzcsyl7gjg6paw4fpf' to '/nix/store/qg1vdpwrwk39j7xpnp4sr2fd2g4x7w80-openmpi-4.0.0/bin/orte-clean': No space left on device
/nix/store/qg1vdpwrwk39j7xpnp4sr2fd2g4x7w80-openmpi-4.0.0

@GrahamcOfBorg

This comment has been minimized.

Copy link

commented Nov 19, 2018

Success on x86_64-darwin (full log)

Attempted: openmpi

Partial log (click to expand)

make[3]: Leaving directory '/private/tmp/nix-build-openmpi-4.0.0.drv-0/openmpi-4.0.0'
make[2]: Nothing to be done for 'install-data-am'.
make[2]: Leaving directory '/private/tmp/nix-build-openmpi-4.0.0.drv-0/openmpi-4.0.0'
make[1]: Leaving directory '/private/tmp/nix-build-openmpi-4.0.0.drv-0/openmpi-4.0.0'
post-installation fixup
gzipping man pages under /nix/store/hjkwrb7jhyczsxl58mxix399faz3hh42-openmpi-4.0.0/share/man/
strip is /nix/store/g5r4apl0za012ffs6ladinwa5w0m1l3k-cctools-binutils-darwin/bin/strip
stripping (with command strip and flags -S) in /nix/store/hjkwrb7jhyczsxl58mxix399faz3hh42-openmpi-4.0.0/lib  /nix/store/hjkwrb7jhyczsxl58mxix399faz3hh42-openmpi-4.0.0/bin
patching script interpreter paths in /nix/store/hjkwrb7jhyczsxl58mxix399faz3hh42-openmpi-4.0.0
/nix/store/hjkwrb7jhyczsxl58mxix399faz3hh42-openmpi-4.0.0

@GrahamcOfBorg

This comment has been minimized.

Copy link

commented Nov 19, 2018

Failure on aarch64-linux (full log)

Attempted: openmpi

Partial log (click to expand)

cannot build derivation '/nix/store/8y1jjk4zi6qn8wxlrgin67wa9mpsl8ng-x509-validation-1.6.11.drv': 13 dependencies couldn't be built
cannot build derivation '/nix/store/by5p7vlw6p1hsixyliy85dim5r8xqzq5-conduit-1.3.1.drv': 14 dependencies couldn't be built
cannot build derivation '/nix/store/q3pv07v48jyh8qzgpm1r8wjv6yc5va57-tls-1.4.1.drv': 17 dependencies couldn't be built
cannot build derivation '/nix/store/x94qhdgjjn6pi5zhcnbxp6csmndvyvrh-connection-0.2.8.drv': 11 dependencies couldn't be built
cannot build derivation '/nix/store/793ymrxpa1nmc0h9qbyb705hzaqpc6aa-yaml-0.8.32.drv': 15 dependencies couldn't be built
cannot build derivation '/nix/store/qcpay92v35qx4jgpckq7krz513ah3sdv-http-client-tls-0.3.5.3.drv': 13 dependencies couldn't be built
cannot build derivation '/nix/store/a1snaj0bpwzl00skil0a6xz97i0mkilf-pandoc-2.2.1.drv': 40 dependencies couldn't be built
cannot build derivation '/nix/store/zsx00jmjpbisrlk7yhaiigbwd4ss9dl8-rdma-core-21.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/79d399gl2qqhc1v0g21aaw482a7psnwp-openmpi-4.0.0.drv': 1 dependencies couldn't be built
error: build of '/nix/store/79d399gl2qqhc1v0g21aaw482a7psnwp-openmpi-4.0.0.drv' failed

@markuskowa markuskowa self-assigned this Nov 19, 2018

@markuskowa

This comment has been minimized.

Copy link
Member

commented Nov 19, 2018

@GrahamcOfBorg build scalapack openmolcas python36Packages.mpi4py

@GrahamcOfBorg

This comment has been minimized.

Copy link

commented Nov 19, 2018

Failure on x86_64-linux (full log)

Attempted: scalapack, openmolcas, python36Packages.mpi4py

Partial log (click to expand)

make[2]: *** [BLACS/TESTING/CMakeFiles/xFbtest.dir/build.make:115: TESTING/xFbtest] Error 1
make[1]: *** [CMakeFiles/Makefile2:1102: BLACS/TESTING/CMakeFiles/xFbtest.dir/all] Error 2
../../lib/libscalapack.so: undefined reference to `MPI_Type_struct'
../../lib/libscalapack.so: undefined reference to `MPI_Attr_get'
collect2: error: ld returned 1 exit status
make[2]: *** [BLACS/TESTING/CMakeFiles/xCbtest.dir/build.make:130: TESTING/xCbtest] Error 1
make[1]: *** [CMakeFiles/Makefile2:1065: BLACS/TESTING/CMakeFiles/xCbtest.dir/all] Error 2
make: *** [Makefile:163: all] Error 2
builder for '/nix/store/hqjszn0jcbqgh80bybva4rq7hksp1nhl-scalapack-2.0.2.drv' failed with exit code 2
error: build of '/nix/store/021gsxzzqv8dysv4h2gsfk8xyrc61mjb-openmolcas-18.09.drv', '/nix/store/802va58fqa0rzps82acr6svqp6lvvw3a-python3.6-mpi4py-3.0.0.drv', '/nix/store/hqjszn0jcbqgh80bybva4rq7hksp1nhl-scalapack-2.0.2.drv' failed

@GrahamcOfBorg

This comment has been minimized.

Copy link

commented Nov 19, 2018

Failure on x86_64-darwin (full log)

Attempted: python36Packages.mpi4py

The following builds were skipped because they don't evaluate on x86_64-darwin: scalapack, openmolcas

Partial log (click to expand)

  __pyx_t_3 = ((PyObject *)__pyx_f_6mpi4py_3MPI_new_Datatype(MPI_LB)); if (unlikely(!__pyx_t_3)) __PYX_ERR(21, 863, __pyx_L1_error)
                                                             ^~~~~~
                                                             MPI_IO
/nix/store/hjkwrb7jhyczsxl58mxix399faz3hh42-openmpi-4.0.0/include/mpi.h:513:5: note: 'MPI_IO' declared here
    MPI_IO,
    ^
6 warnings and 8 errors generated.
error: command '/nix/store/hjkwrb7jhyczsxl58mxix399faz3hh42-openmpi-4.0.0/bin/mpicc' failed with exit status 1
builder for '/nix/store/j5zkpkpz2vr61wsz3330xx0ikvw87fvq-python3.6-mpi4py-3.0.0.drv' failed with exit code 1
error: build of '/nix/store/j5zkpkpz2vr61wsz3330xx0ikvw87fvq-python3.6-mpi4py-3.0.0.drv' failed

@GrahamcOfBorg

This comment has been minimized.

Copy link

commented Nov 19, 2018

Timed out, unknown build status on aarch64-linux (full log)

Attempted: scalapack, openmolcas, python36Packages.mpi4py

Partial log (click to expand)

cannot build derivation '/nix/store/793ymrxpa1nmc0h9qbyb705hzaqpc6aa-yaml-0.8.32.drv': 15 dependencies couldn't be built
cannot build derivation '/nix/store/qcpay92v35qx4jgpckq7krz513ah3sdv-http-client-tls-0.3.5.3.drv': 13 dependencies couldn't be built
cannot build derivation '/nix/store/a1snaj0bpwzl00skil0a6xz97i0mkilf-pandoc-2.2.1.drv': 40 dependencies couldn't be built
cannot build derivation '/nix/store/zsx00jmjpbisrlk7yhaiigbwd4ss9dl8-rdma-core-21.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/79d399gl2qqhc1v0g21aaw482a7psnwp-openmpi-4.0.0.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/x4yq7qzpjsq6nmpjjkwywvsl2dlk0bdp-globalarrays-5.7.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/sh8a4bisi1rbin97mnabfhmvhij1jsan-python3.6-mpi4py-3.0.0.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/1fmfvhg0qg27z6c7zs3z7n777f7bagzp-scalapack-2.0.2.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/xlkaknk8mkpzkdvy0f0qay16fdx4w4fk-openmolcas-18.09.drv': 2 dependencies couldn't be built
error: build of '/nix/store/1fmfvhg0qg27z6c7zs3z7n777f7bagzp-scalapack-2.0.2.drv', '/nix/store/sh8a4bisi1rbin97mnabfhmvhij1jsan-python3.6-mpi4py-3.0.0.drv', '/nix/store/xlkaknk8mkpzkdvy0f0qay16fdx4w4fk-openmolcas-18.09.drv' failed

@markuskowa

This comment has been minimized.

Copy link
Member

commented Nov 19, 2018

This is major update that seems to break a bunch of packages (e.g. scalapack, and python36Packages.mpi4py).
CC @costrouc for more testing.

@costrouc

This comment has been minimized.

Copy link
Contributor

commented Nov 19, 2018

I don't have the time right now to heavily look into this but I would be happy to work on this in the next week or so. Looks like openmpi 4.0.0 was released 7 days ago and other package mangers have not made the transition yet (looked at anaconda and arch). Would be nice to see if other package mangers are having issues too

@markuskowa

This comment has been minimized.

Copy link
Member

commented Nov 19, 2018

I am OK if we wait and keep this PR open until there are more upstream updates out. Updating openmpi is not a pressing issue (would be nice to have for the 19.03 release).

@markuskowa markuskowa added this to the 19.03 milestone Nov 19, 2018

@markuskowa

This comment has been minimized.

Copy link
Member

commented Jan 29, 2019

Closing this PR in favor of #54877

@markuskowa markuskowa closed this Jan 29, 2019

@r-ryantm r-ryantm deleted the r-ryantm:auto-update/openmpi branch Feb 3, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.