ADIOS2 2.9.0 release

[vicentebolea]

Instructions for performing an ADIOS2 release:

• Make sure that the milestone for 2.9.0 has no pending issues/PRs.
• Update your remotes

git fetch origin
git checkout release_29
git reset --hard origin/release_29

• Create a branch that updates the version

git checkout -b bump-release-version origin/release_29
git grep --name-only 2.9.0-rc1  | xargs -n1 sed -i 's/2.9.0-rc1/2.9.0/g'

• Create PR
• Ask for review
• Merge PR
• Create tag
• Create Release in GitHub page
  • Use the following script for getting the PR of this release
    • ./scripts/developer/create-changelog.sh v2.9.0 v2.8.3
  • Copy its content to the release description
• Create PR that merges release_29 into master
• Submit a PR in Spack that adds this new version of ADIOS (if not RC mark this new version as preferred)
• Write an announcement in the ADIOS-ECP mail-list:
  • https://groups.google.com/a/kitware.com/g/adios-ecp

[vicentebolea]

#3568

[vicentebolea]

cc: @pnorbert

[vicentebolea]

I intent to add this template to master after this release to standardize the release process PR #3570. Its the first time doing something like this thus there might be some changes here and there.

[MichaelLaufer]

@vicentebolea just curious, what about updating the Spack package.py?

[vicentebolea]

@vicentebolea just curious, what about updating the Spack package.py?

Thanks for pointing out, indeed, we need to add that step

[vicentebolea]

@MichaelLaufer

spack/spack#36563

[eisenhauer]

Can we also add updating the conda-feedstock recipe?

[vicentebolea]

Can we also add updating the conda-feedstock recipe?

We do treat Spack as a first citizen since it is an ECP project and help us to deploy ADIOS2 in Doe supercomputers. For other package managers, we can delegate on their community with occasional help from us.

[williamfgc]

Thanks @vicentebolea .

Can we also add updating the conda-feedstock recipe?

There is an automated PR. Unfortunately, it's catching several test failures it might be of interest here. CC @ax3l

[vicentebolea]

On another note the Spack build is failing due to our removal of the deprecated adios_init function in both Sensei and ParaView . I will update both projects and patch the spack package so that it offers those deprecated overloads in the meanwhile.

Keeping here the error message for future reference:

Sensei

/tmp/root/spack-stage/spack-stage-sensei-4.0.0-vp45t2jdvmzm6qpqbannlajsbyqmhajn/spack-src/sensei/ADIOS2Schema.cxx:425:69: error: macro "adios2_init" passed 2 arguments, but takes just 1
  425 |   this->Adios = adios2_init(comm, adios2_debug_mode(this->DebugMode));
      |                                                                     ^
In file included from /home/software/spack/[padded-to-256-chars]/linux-ubuntu20.04-x86_64_v3/gcc-11.1.0/adios2-2.9.0-nt6lmyc7sndvra7fit55fb5d3bqzxqsy/include/adios2_c.h:16,
                 from /tmp/root/spack-stage/spack-stage-sensei-4.0.0-vp45t2jdvmzm6qpqbannlajsbyqmhajn/spack-src/sensei/ADIOS2Schema.h:10,
                 from /tmp/root/spack-stage/spack-stage-sensei-4.0.0-vp45t2jdvmzm6qpqbannlajsbyqmhajn/spack-src/sensei/ADIOS2Schema.cxx:1:
/home/software/spack/[padded-to-256-chars]/linux-ubuntu20.04-x86_64_v3/gcc-11.1.0/adios2-2.9.0-nt6lmyc7sndvra7fit55fb5d3bqzxqsy/include/adios2/c/adios2_c_adios.h:25: note: macro "adios2_init" defined here
   25 | #define adios2_init(comm) adios2_init_mpi(comm)
      | 
/tmp/root/spack-stage/spack-stage-sensei-4.0.0-vp45t2jdvmzm6qpqbannlajsbyqmhajn/spack-src/sensei/ADIOS2Schema.cxx: In member function 'int senseiADIOS2::InputStream::Initialize(MPI_Comm)':
/tmp/root/spack-stage/spack-stage-sensei-4.0.0-vp45t2jdvmzm6qpqbannlajsbyqmhajn/spack-src/sensei/ADIOS2Schema.cxx:425:17: error: 'adios2_init' was not declared in this scope
  425 |   this->Adios = adios2_init(comm, adios2_debug_mode(this->DebugMode));
      |                 ^~~~~~~~~~~
/tmp/root/spack-stage/spack-stage-sensei-4.0.0-vp45t2jdvmzm6qpqbannlajsbyqmhajn/spack-src/sensei/ADIOS2Schema.cxx:420:38: warning: unused parameter 'comm' [-Wunused-parameter]
  420 | int InputStream::Initialize(MPI_Comm comm)
      |                             ~~~~~~~~~^~~~
make[2]: *** [sensei/CMakeFiles/senseiCore.dir/build.make:457: sensei/CMakeFiles/senseiCore.dir/ADIOS2Schema.cxx.o] Error 1

ParaView

/tmp/root/spack-stage/spack-stage-paraview-5.11.0-hirjfrpoia27cdimnr3ki3x7kwaq3t2z/spack-src/VTK/IO/ADIOS2/vtkADIOS2CoreImageReader.cxx: In member function 'bool vtkADIOS2CoreImageReader::OpenAndReadMetaData()':
/tmp/root/spack-stage/spack-stage-paraview-5.11.0-hirjfrpoia27cdimnr3ki3x7kwaq3t2z/spack-src/VTK/IO/ADIOS2/vtkADIOS2CoreImageReader.cxx:361:89: error: 'DebugON' is not a member of 'adios2'
  361 |     this->Impl->Adios.reset(new adios2::ADIOS(*comm->GetMPIComm()->GetHandle(), adios2::DebugON));
      |                                                                                         ^~~~~~~

@eisenhauer do you think that adding a temporary (until next releases of ParaView and maybe Sensei) patch the reverts of #3494 in Spack will be able to build? We would want to build ADIOS2 with latest release of ParaView.

[williamfgc]

@vicentebolea on ParaView I don't know the status of the vtkCoreImageReader. I think it hasn't been maintained in a long time (hence the failure). Better to check with you and @caitlinross

[vicentebolea]

Closing this as the release has been already done.

[vicentebolea]

@vicentebolea on ParaView I don't know the status of the vtkCoreImageReader. I think it hasn't been maintained in a long time (hence the failure). Better to check with you and @caitlinross

I am not familiar with it, maybe @caitlinross

[caitlinross]

I don't really know much about it either, or how much it's used. I'm pretty sure it's tested in VTK CI, but I believe that uses ADIOS 2.8.x (can't remember which one), so that's why it wasn't caught that way. I think Vicente already took care of updating that in VTK. I guess we should also update VTK CI to use ADIOS 2.9 now too. And I guess ParaView superbuild?

[vicentebolea]

Sorry, I read this too fast, as @caitlinross said we already have a MR in VTK that updates it. After it makes it to VTK master it will take a while to make it to ParaView master. Thus there will be a period of time where people can potentially hit this error if building ParaView with ADIOS 2.9. For people using Spack they soon will be able to do it as soon as this PR gets merged: spack/spack#36563

I think Vicente already took care of updating that in VTK. I guess we should also update VTK CI to use ADIOS 2.9 now too.

Good idea lets do it soon

And I guess ParaView superbuild?

After the MR fixing this makes it to ParaView

[liangwang0734]

Thanks @vicentebolea .

Can we also add updating the conda-feedstock recipe?

There is an automated PR. Unfortunately, it's catching several test failures it might be of interest here. CC @ax3l

I hope this helps, as half of the failed builds seem to have trouble with a known change in multiprocessing in Python > 3.8 on OS X.