ADIOS2 Feedstock #8526

ax3l · 2019-06-08T18:28:17Z

Prepare a feedstock for ADIOS2, including MPI variants.

We will enable aarch64 and ppc64le support later on in the feedstock.

Checklist

License file is packaged (see here for an example)
Source is from official source
Package does not vendor other packages (it does for the start, but only private dependencies. we will reduce those over time)
Build number is 0
GitHub users listed in the maintainer section have posted a comment confirming they are willing to be listed there

conda-forge-linter · 2019-06-08T18:28:26Z

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipes/adios2) and found it was in an excellent condition.

ax3l · 2019-06-09T14:37:58Z

@chuckatkins in coordination with @williamfgc, I have adopted your conda scripts for conda-forge.

There are still some smaller hiccups, but at least on Linux and OSX the build passes already.
I am currently using ADIOS2 v2.4.0rc1, so that we can carve out issues towards the 2.4.0 release.

Both of you, please confirm that you want to be listed as co-maintainers of the feedstock and please tell me anyone you want to add as well.

Prepare a feedstock for ADIOS2, including MPI variants.

chuckatkins · 2019-06-30T17:20:22Z

I agree to be listed as co-maintainer of this feedstock

ax3l · 2019-07-01T07:58:16Z

@chuckatkins thanks!

@williamfgc it will be conda install -c conda-forge adios2 for the start. We also build MPI-variants which allows users to make the following installation choices:

conda install adios2 installs by default the nompi variant
conda install adios2=*=mpi* installs any mpi-compatible implementation
conda install adios2=*=mpi_mpich* explicitly picks mpich variant.
This will have the same result as conda install mpich adios2=*=mpi*
conda install adios2=2.4.0=nompi* explicitly forbids mpi. Usually not necessary, as the build number offset will cause nompi to be preferred unless mpi is explicitly requested.

ax3l · 2019-07-01T08:12:04Z

I'll move Windows HDF5 support to an in-feedstock update since I cannot spot the linker error here: https://ci.appveyor.com/project/conda-forge/staged-recipes/builds/25643672

chrisburr

Few cosmetic things then LGTM 👍

recipes/adios2/meta.yaml

chrisburr · 2019-07-01T12:15:53Z

recipes/adios2/LICENSE

@@ -0,0 +1,201 @@
+                                 Apache License


Do you need this file? I think it should be in the source tarball

I thought we should always add the LICENSE file separately?
(link in PR description template)

We always need to package it but we prefer to get it from source tarball(s) instead of including it in the recipe directory if we can to ensure it's correct and up to date.

Clean up: suggestions by Chris (thanks!) Co-Authored-By: Chris Burr <chrisburr@users.noreply.github.com>

ax3l · 2019-07-01T13:08:33Z

Thanks for the review @chrisburr, I added your suggestions and CI passed again :)

ax3l · 2019-07-01T14:39:44Z

ping @chrisburr :)

scopatz · 2019-07-01T15:52:36Z

recipes/adios2/build.sh

+    CMAKE_PLATFORM_FLAGS+=(-DADIOS2_USE_BZip2=OFF)
+elif [[ ${target_platform} =~ linux.* ]]; then
+    # link transitive libraries during build of intermediate wrapper lib
+    # export LDFLAGS="${LDFLAGS} -Wl,-rpath-link,${PREFIX}/lib"


Are these comments still needed.

Luckily not, will remove.

scopatz · 2019-07-01T15:53:53Z

recipes/adios2/meta.yaml

+# prioritize nompi variant via build number
+{% if mpi == 'nompi' %}
+{% set build = build + 100 %}
+{% endif %}


This doesn't seem like the right way to do this.

Not? That's how @minrk showed it to me (hdf5, h5py et al.). We increase the build number for nompi variants to select them by default.

Really? That is kinda nuts.

We don't have documentation for this do we?

Somewhat, I am just following recently explored best practices:

https://hackmd.io/ry4uI0thTs2q_b4mAQd_qg

MPI Variants adios-feedstock#12 (comment)

https://github.com/conda-forge/hdf5-feedstock/blob/master/recipe/meta.yaml

https://github.com/conda-forge/h5py-feedstock/blob/master/recipe/meta.yaml

Bumping the build number with an offset actually sounds quite legit to me.

I think this way of doing things is causing issues in the solver with multi output recipes that depend on packages with MPI variants. I've not managed to make sense of it yet but these builds show it:

https://dev.azure.com/conda-forge/feedstock-builds/_build/results?buildId=45775

https://dev.azure.com/conda-forge/feedstock-builds/_build/results?buildId=45278

We don't have documentation for this do we?

It's in the docs

Build offset is the only way to indicate preference in the solver that I'm aware of that works for existing packages. Note that the original and first-implemented plan was to use track_features to deprioritize, but this does not work for packages that have ever been published before, but I think it might work for new packages, or packages where all previous builds have been fully revoked or hotfixed.

In most cases, packages that require MPI express no preference, but it's more complicated when you have three variants and want users to opt-in to MPI (e.g. hdf5). Conda still has no ability to express preferences in a natural way, so any solution is going to be a bit wonky until there's a real solution. For that reason, I might recommend avoiding a 'preference' unless it's important to avoid picking up a variant in certain circumstances, at least until there's a better way.

ax3l force-pushed the topic-adios2 branch 14 times, most recently from a618cf3 to 1849f63 Compare June 9, 2019 12:41

ax3l mentioned this pull request Jun 9, 2019

[WIP] CMake 3.9+: RPATH on OSX does not affect install_name ornladios/ADIOS2#1484

Closed

ax3l force-pushed the topic-adios2 branch 6 times, most recently from e87f43d to faf4b1b Compare June 9, 2019 13:34

ADIOS2 Feedstock

2d4cf85

Prepare a feedstock for ADIOS2, including MPI variants.

ax3l force-pushed the topic-adios2 branch from faf4b1b to 2d4cf85 Compare June 9, 2019 20:01

ax3l mentioned this pull request Jun 9, 2019

Compile Error with MSVC 19.04 ornladios/ADIOS2#1485

Closed

williamfgc mentioned this pull request Jun 24, 2019

summit anaconda build error ornladios/ADIOS2#1524

Closed

Patch: MSVC

661d876

ax3l force-pushed the topic-adios2 branch from 10a5cd6 to 661d876 Compare June 29, 2019 10:00

ax3l mentioned this pull request Jun 29, 2019

fix MSVC/conda build ornladios/ADIOS2#1575

Closed

Cleanup and no -GL on CFLAGS, too

8bf6102

ax3l force-pushed the topic-adios2 branch from 164594b to 8bf6102 Compare July 1, 2019 08:02

ax3l force-pushed the topic-adios2 branch 8 times, most recently from 1a1eef6 to a7f0d49 Compare July 1, 2019 09:41

Patch: Python Path on Windows

202dcba

ax3l force-pushed the topic-adios2 branch from a7f0d49 to dca4a95 Compare July 1, 2019 09:43

Version: 2.4.0

06bb01c

ax3l force-pushed the topic-adios2 branch from dca4a95 to 06bb01c Compare July 1, 2019 11:18

chrisburr approved these changes Jul 1, 2019

View reviewed changes

Apply suggestions from code review

624fe99

Clean up: suggestions by Chris (thanks!) Co-Authored-By: Chris Burr <chrisburr@users.noreply.github.com>

scopatz reviewed Jul 1, 2019

View reviewed changes

ax3l force-pushed the topic-adios2 branch 3 times, most recently from 5e48a18 to d654b04 Compare July 1, 2019 17:29

Clean-Up More

78cf899

ax3l force-pushed the topic-adios2 branch from d654b04 to 78cf899 Compare July 1, 2019 17:46

chrisburr merged commit 41589a2 into conda-forge:master Jul 1, 2019

ax3l deleted the topic-adios2 branch July 1, 2019 19:41

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ADIOS2 Feedstock #8526

ADIOS2 Feedstock #8526

ax3l commented Jun 8, 2019 •

edited

conda-forge-linter commented Jun 8, 2019

ax3l commented Jun 9, 2019

chuckatkins commented Jun 30, 2019

ax3l commented Jul 1, 2019 •

edited

ax3l commented Jul 1, 2019

chrisburr left a comment

chrisburr Jul 1, 2019

ax3l Jul 1, 2019 •

edited

chrisburr Jul 1, 2019

ax3l commented Jul 1, 2019

ax3l commented Jul 1, 2019

scopatz Jul 1, 2019

ax3l Jul 1, 2019

scopatz Jul 1, 2019

ax3l Jul 1, 2019 •

edited

scopatz Jul 1, 2019

scopatz Jul 1, 2019

ax3l Jul 1, 2019 •

edited

chrisburr Jul 1, 2019

minrk Jul 2, 2019 •

edited

ADIOS2 Feedstock #8526

ADIOS2 Feedstock #8526

Conversation

ax3l commented Jun 8, 2019 • edited

conda-forge-linter commented Jun 8, 2019

ax3l commented Jun 9, 2019

chuckatkins commented Jun 30, 2019

ax3l commented Jul 1, 2019 • edited

ax3l commented Jul 1, 2019

chrisburr left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ax3l Jul 1, 2019 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ax3l commented Jul 1, 2019

ax3l commented Jul 1, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ax3l Jul 1, 2019 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ax3l Jul 1, 2019 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

minrk Jul 2, 2019 • edited

Choose a reason for hiding this comment

ax3l commented Jun 8, 2019 •

edited

ax3l commented Jul 1, 2019 •

edited

ax3l Jul 1, 2019 •

edited

ax3l Jul 1, 2019 •

edited

ax3l Jul 1, 2019 •

edited

minrk Jul 2, 2019 •

edited