Upgrade minimum support GCC from 4.7.2 with C++11 to something higher (4.8.4 or 4.9.3)?

[bartlettroscoe]

Next Action Status:

Looks like some customers need GCC 4.8.4 with C++11 Trilinos currently. Next: Decide to make the downgrade the CI server from GCC 4.9.3 back to GCC 4.8.4 (which currently passes the full CI test suite) ...

CC: @trilinos/framework

Description:

Maintaining support for GCC 4.7.2 puts a lot of constraints on the C++11 code that you can write and is holding back Trilinos development (see #1002). This story is to remove testing and support for GCC 4.7.2 and upgrade to GCC 4.8.x or newer (GCC 4.9.3 would be better given that is what Sierra uses).

[bartlettroscoe]

@rppawlo requested the disable of Panzer and Phalanx in the GCC 4.7.2 build:

• http://testing.sandia.gov/cdash/index.php?project=Trilinos&display=project&filtercount=2&showfilters=1&filtercombine=and&field1=buildname&compare1=61&value1=Linux-GCC-4.7.2-CONTINUOUS_MPI_OPT_DEV_SHARED&field2=buildstarttime&compare2=84&value2=now

I will go ahead and take care of that.

@jwillenbring, can you please archive the info you have on what version we can upgrade the minimum too? Ideally this would be GCC 4.9.3 but if we need to support GCC 4.8.4 then so be it. Let's just do this analysis quickly and make this upgrade happen.

NOTE: @rrdrake said that Sierra is now clear of GCC 4.7.2 so that will not hold us back.

[bartlettroscoe]

So far, the only example of the need for less than GCC 4.9.3 is GCC 4.8.4 on Mira (see email below from Paul Lin).

Paul Lin mentioned that there is a machine he still runs on that uses GCC 4.7.2 but that he keeps a set of patches for dealing with issues.

---

Not too long ago Paul Lin mentioned machines such as Mira at the facilities that have compilers older than 4.9. This is a machine that xSDK cares about. I looked and we use 4.8.4 for that build.

https://xsdk.info/release-0-2-0-alpha/

I am wondering then if we can drop 4.7.2, as it seems problematic, use 4.9.3 for continuous builds, but keep 4.8.4 for the minimum?

Jim

[bartlettroscoe]

At the request of Mike H. I moved this "In Progress" in the SART board.

[jwillenbring]

I am investigating possibilities for supporting @pwxy requirements as well as xSDK requirements.

[nmhamster]

@jwillenbring can you sync with @dmvigi on requirements as part of the broader SEMS requirements effort? Thank you.

[pwxy]

I am trying to get better performance from the drekar/trilinos clang 3.9 binary on sequoia. This really the only viable alternative. It is not practical to support gnu 4.7.2 the next 1.5 years.

[pwxy]

ops forgot to mention in the previous post that due to the negative impact on developers to maintain 4.7.2 compatibility, it is best to just move to 4.9.3.

[nmhamster]

@pwxy - I don't think we have final agreement from the IC teams yet to move do we?

[pwxy]

@nmhamster you are correct, I don't know about the other IC teams or xSDK. I'm just speaking for myself. I'm no longer a holdout.

[bartlettroscoe]

@nmhamster:

I don't think we have final agreement from the IC teams yet to move do we?

Who do you do you mean by "IC"? I know that Sierra is free of GCC 4.7.2 and now GCC 4.9.3 is the minimum required GCC version for C++11 support. I will send out an email to the SART email list and ask if there is anyone else who needs C++11 for anything less than GCC 4.9.3. If we don't get strong pushback in a couple of weeks or so, then it is time to upgrade and disable all the type-2 templated packages from GCC 4.7.2 testing (starting with Kokkos on down) or change all of the GCC 4.7.2 automated builds posting to CDash to GCC 4.9.3.

We also need to quickly make a decision on dropping support for C++98 all together for the 'develop' and 'master' branches in #1390. This is related but technically these two issues can be decided independently.

[pwxy]

Actually develop Trilinos won't build for gnu 4.7.2 anymore (builds fine for 4.9.3). I do not plan to ask anyone to fix this so that it will build for 4.7.2.

[jwillenbring]

Significant progress has been made by @pwxy to remove 4.7.2 needs. Questions still exist for 4.8 builds. One important one is that 4.8 is the default for RHEL 7, and default configurations on RHEL are currently important to Xyce. If SEMS deploys compilers more widely, this importance will decrease. Hope to discuss timeline for that later today.

[ibaned]

According to this page and this page, GCC 4.8.4 should have all the pure-language-level C++11 features. There is one thing I know it is missing relative to GCC 4.9.0, and that is a correct implementation of <regex> (see here). I don't think that is a big deal as far as we're concerned.

[mhoemmen]

GCC 4.8.4 + CUDA 7.5 works fine for me with Tpetra. I don't see any of the lambda capture bugs that I see in GCC 4.7.2 (that I can work around by explicitly listing all variables that I want to capture).

[bartlettroscoe]

I ran the full standard CI build with the SEMS GCC 4.8.4 stack:

Currently Loaded Modulefiles:
  1) sems-env                       4) sems-git/2.10.1                7) sems-boost/1.63.0/base        10) sems-hdf5/1.8.12/parallel     13) sems-superlu/4.3/base
  2) sems-python/2.7.9              5) sems-gcc/4.8.4                 8) sems-yaml_cpp/0.5.3/base      11) sems-netcdf/4.3.2/parallel
  3) sems-cmake/3.5.2               6) sems-openmpi/1.6.5             9) sems-zlib/1.2.8/base          12) sems-parmetis/4.0.3/parallel

on RHEL 6 and it passed. (Results below).

Given that it looks like there are still some platforms and customers that require GCC 4.8.4 and it is reported above that GCC 4.8.4 should have all of the C++11 language features, I think we should make GCC 4.8.4 the minimum required GCC version for Trilinos C++11 support and change the standard CI build from GCC 4.9.3 to GCC 4.8.4 in order to help maintain that.

Any objections?

---

READY TO PUSH: Trilinos: crf450.srn.sandia.gov

Thu Jun 22 16:08:20 MDT 2017

Enabled Packages: 
Disabled Packages: PyTrilinos,Claps,TriKota
Enabled all Packages

Build test results:
-------------------
0) MPI_RELEASE_DEBUG_SHARED_PT => passed: passed=2340,notpassed=0 (89.81 min)

*** Commits for repo :

0) MPI_RELEASE_DEBUG_SHARED_PT Results:
---------------------------------------

  passed: Trilinos/MPI_RELEASE_DEBUG_SHARED_PT: passed=2340,notpassed=0
  
  Thu Jun 22 16:08:20 MDT 2017
  
  Enabled Packages: 
  Disabled Packages: PyTrilinos,Claps,TriKota
  Enabled all Packages
  Hostname: crf450.srn.sandia.gov
  Source Dir: /home/rabartl/Trilinos.base/Trilinos/cmake/tribits/ci_support/../../..
  Build Dir: /home/rabartl/Trilinos.base/BUILDS/CHECKIN/MPI_RELEASE_DEBUG_SHARED_PT
  
  CMake Cache Varibles: -DTrilinos_TRIBITS_DIR:PATH=/home/rabartl/Trilinos.base/Trilinos/cmake/tribits -DTrilinos_ENABLE_TESTS:BOOL=ON -DTrilinos_TEST_CATEGORIES:STRING=BASIC -DTrilinos_ALLOW_NO_PACKAGES:BOOL=OFF -DDART_TESTING_TIMEOUT:STRING=300.0 -DBUILD_SHARED_LIBS=ON -DTrilinos_DISABLE_ENABLED_FORWARD_DEP_PACKAGES=ON -DTrilinos_TRACE_ADD_TEST=ON -DTrilinos_ENABLE_SECONDARY_TESTED_CODE:BOOL=OFF -DTrilinos_CONFIGURE_OPTIONS_FILE:STRING=cmake/std/MpiReleaseDebugSharedPtSettings.cmake,cmake/std/BasicCiTestingSettings.cmake -DTrilinos_ENABLE_ALL_OPTIONAL_PACKAGES:BOOL=ON -DTrilinos_ENABLE_ALL_PACKAGES:BOOL=ON -DTrilinos_ENABLE_ALL_FORWARD_DEP_PACKAGES:BOOL=ON -DTrilinos_ENABLE_PyTrilinos:BOOL=OFF -DTrilinos_ENABLE_Claps:BOOL=OFF -DTrilinos_ENABLE_TriKota:BOOL=OFF
  Make Options: -j16 
  CTest Options: -j16 
  
  Pull: Not Performed
  Configure: Passed (2.28 min)
  Build: Passed (75.59 min)
  Test: Passed (11.93 min)
  
  100% tests passed, 0 tests failed out of 2340
  
  Label Time Summary:
  Amesos               =  19.66 sec (14 tests)
  Amesos2              =   9.88 sec (8 tests)
  Anasazi              = 102.72 sec (71 tests)
  AztecOO              =  16.12 sec (17 tests)
  Belos                =  95.87 sec (67 tests)
  Domi                 = 149.15 sec (125 tests)
  Epetra               =  44.77 sec (61 tests)
  EpetraExt            =  13.40 sec (10 tests)
  FEI                  =  44.36 sec (43 tests)
  Galeri               =   4.16 sec (9 tests)
  GlobiPack            =   1.72 sec (6 tests)
  Ifpack               =  59.90 sec (53 tests)
  Ifpack2              =  36.44 sec (33 tests)
  Intrepid             = 191.33 sec (152 tests)
  Intrepid2            =  33.62 sec (74 tests)
  Isorropia            =   7.47 sec (6 tests)
  Kokkos               =  44.77 sec (21 tests)
  KokkosKernels        =   3.47 sec (9 tests)
  ML                   =  45.40 sec (34 tests)
  MiniTensor           =   0.41 sec (2 tests)
  MueLu                = 222.01 sec (56 tests)
  NOX                  = 140.53 sec (101 tests)
  OptiPack             =   5.88 sec (5 tests)
  Panzer               = 268.62 sec (129 tests)
  Phalanx              =   5.28 sec (19 tests)
  Pike                 =   3.50 sec (7 tests)
  Piro                 =  27.98 sec (12 tests)
  ROL                  = 754.60 sec (133 tests)
  RTOp                 =  10.83 sec (24 tests)
  Rythmos              = 144.19 sec (83 tests)
  SEACAS               =   9.57 sec (14 tests)
  STK                  =  27.81 sec (11 tests)
  Sacado               =  45.63 sec (292 tests)
  Shards               =   1.42 sec (4 tests)
  ShyLU                =   7.54 sec (5 tests)
  Stokhos              = 108.86 sec (75 tests)
  Stratimikos          =  33.79 sec (39 tests)
  Teko                 = 101.58 sec (19 tests)
  Tempus               = 493.30 sec (9 tests)
  Teuchos              =  52.79 sec (126 tests)
  ThreadPool           =   7.12 sec (10 tests)
  Thyra                =  61.99 sec (80 tests)
  Tpetra               = 134.30 sec (129 tests)
  TrilinosCouplings    =  39.99 sec (19 tests)
  Triutils             =   2.49 sec (2 tests)
  Xpetra               =  41.11 sec (17 tests)
  Zoltan               = 200.47 sec (16 tests)
  Zoltan2              = 133.53 sec (97 tests)
  
  Total Test time (real) = 715.64 sec
  
  Total time for MPI_RELEASE_DEBUG_SHARED_PT = 89.81 min

[bartlettroscoe]

The branch that makes this change is:

• https://github.com/bartlettroscoe/Trilinos/tree/switch-to-gcc-4.8.4-1363

We just need to merge (or rebase) that branch to 'develop' and push and update the GitHub wiki pages that mention GCC 4.9.3.

Should we pull the trigger?

[pwxy]

As I mentioned in previous comments, if I am the final user that prevents switching the minimum gcc version from 4.7.2 to 4.9.3, then please make 4.9.3 the default. No point having developers expend effort to maintain backward compatibilty with gnu 4.7.2.

bgclang++11 (clang 3.9) on sequoia compiles develop Trilinos without issue. By forcing lots of inlining (and long compile times and a accepting a large binary), for the two Drekar test cases I have been working with, the clang-built binary is competitive with the gnu 4.7.2-built binary, which the exception of the solve being 60-75% slower when using ifpack2 triangular solves (native RILUK and Gauss-Seidel). For situations where there is sufficient memory (e.g. strong-scaling runs), @ambrad showed me how to switch from ifpack2 native RILUK to HTS. For bgclang++11 (clang 3.9), HTS solve is twice as fast as ifpack2 native RILUK, but with a penalty of larger memory requirement (50% increase). When Trilino makes 4.9.3 the minimum gcc version, then I will use bgclang++11.

However, it seems that there are other users than require gcc 4.8.4 and cannot go to 4.9.3.
I have revisited the gcc 4.8.4 sequoia build environment, and the Drekar build with gcc 4.8.4 seems to run fine (things like C++ exception handling isn't supposed to work, but that is okay).

[mhoemmen]

@pwxy My experience is that GCC 4.7.2 is broken in ways that make using Kokkos harder, while GCC 4.8.4 doesn't have these issues.

[pwxy]

I agree; unless others need 4.7.2, we should ditch it.

[jwillenbring]

I spoke with @pwxy this morning. I need to clear with @maherou yet, but it appears we can move to 4.8.4 as the minimum supported compiler version. There are several reasons to stick with 4.8.4 for the time being.

1. It simplifies moving beyond the C++98 standard for Xyce support, as described in Create plan to drop Trilinos support for C++98 preventing the usage of C++11 #1390.
2. Several important platforms have a max compiler of 4.8.4. This impacts customers both internal and external to Sandia.
3. 4.8.5 is the default compiler for RHEL 7.

This is not to say we should support 4.8.4 until end of life for RHEL 7, but rather that it seems to be a good version to support for now.

[bartlettroscoe]

I am in the process of pushing a commit to change from GCC 4.9.3 to 4.8.4 for the standard CI build (see #1453). Also, I am archiving Jim's email below.

---

From: Trilinos-developers [mailto:trilinos-developers-bounces@trilinos.org] On Behalf Of Willenbring, James M
Sent: Monday, June 26, 2017 12:38 PM
To: trilinos-users@trilinos.org; trilinos-announce@trilinos.org; Trilinos Developers List trilinos-developers@trilinos.org
Subject: [EXTERNAL] [Trilinos-developers] Trilinos compiler support announcement

All,

On behalf of Trilinos Project Lead Mike Heroux, I want to inform you of some modifications to the minimum supported GCC compiler version for most of Trilinos. Effective immediately, the minimum supported GCC compiler version for most of Trilinos is 4.8.4. This applies to future development on the develop and master branches. The exceptions to this minimum version include the packages used in this build:

https://testing.sandia.gov/cdash/index.php?project=Trilinos&parentid=2967870

Namely,

Teuchos
Sacado
Epetra
Zoltan
Triutils
TrilinosSS
EpetraExt
Isorropia
AztecOO
Amesos
Ifpack
Belos
NOX
TrilinosCouplings

For these packages, the minimum supported compiler version remains GCC 4.4.7.

Thanks

Jim

[jwillenbring]

In support of upgrading the minimum compiler version, the @trilinos/framework is setting up 2 new continuous builds to cover the most important GCC compiler versions: 4.8.4 (minimum) and 4.9.3 (target for some customers). That activity will be tracked here: #1457.

In addition, we are adding a 4.8.4 clean build (see #1445).

I believe that as described, this ticket can now be closed. Other related activities are covered by other tickets. If that is not accurate, feel free to reopen.