List of currently failing tests #13703

peterrum · 2022-05-09T20:03:25Z

(Matthias) Failing tests with RC1:

Host	Configuration	Commit	Build warnings	Failing tests	Passing tests
tester	Clang-14.0.5-dealii-9.4-unity_build	5303c8	0	0	0
tester	GNU-12.1.1-dealii-9.4-all_components	5303c8	6	0	6
tester	Clang-14.0.5-dealii-9.4-no_mpi	5303c8	0	0	0
tester	GNU-12.1.1-dealii-9.4-cpp20	5303c8	7	0	0
tester	Clang-14.0.5-dealii-9.4-cpp17	5303c8	0	0	0
tester	Clang-14.0.5-dealii-9.4-cpp14	5303c8	0	0	0
tester	Clang-14.0.5-dealii-9.4-autodetection	5303c8	0	22	13656
tester	Clang-13.0.1-dealii-9.4-autodetection	5303c8	0	21	13657
tester	Clang-12.0.0-dealii-9.4-autodetection	5303c8	0	0	0
tester	Clang-11.1.0-dealii-9.4-autodetection	5303c8	0	0	0
tester	Clang-10.0.0-dealii-9.4-autodetection	5303c8	0	0	0
tester	Clang-9.0.1-dealii-9.4-autodetection	5303c8	0	0	0
tester	Clang-8.0.1-dealii-9.4-autodetection	5303c8	0	0	0
tester	Clang-7.1.0-dealii-9.4-autodetection	5303c8	0	0	0
tester	Clang-6.0.1-dealii-9.4-autodetection	5303c8	0	0	0
tester	Clang-5.0.2-dealii-9.4-autodetection	5303c8	0	0	0
tester	GNU-12.1.1-dealii-9.4-autodetection	5303c8	6	4	13698
tester	GNU-11.3.0-dealii-9.4-autodetection	5303c8	0	6	13684
tester	GNU-10.3.0-dealii-9.4-autodetection	5303c8	0	0	0
tester	GNU-9.3.0-dealii-9.4-autodetection	5303c8	0	0	0
tester	GNU-8.4.0-dealii-9.4-autodetection	5303c8	0	0	0
tester	GNU-7.4.0-dealii-9.4-autodetection	5303c8	0	0	0
tester	GNU-6.5.0-dealii-9.4-autodetection	5303c8	0	0	0
tester	GNU-11.2.0-dealii-9.4-ubuntu-lts-22	5303c8	0	12	13459
tester	GNU-9.4.0-dealii-9.4-ubuntu-lts-20	5303c8	0	0	0
tester	GNU-7.5.0-dealii-9.4-ubuntu-lts-18	5303c8	0	0	0
tester	GNU-10.2.1-dealii-9.4-debian-11	5303c8	300	12	13252
tester	GNU-8.3.0-dealii-9.4-debian-10	5303c8	0	0	0
tester	Clang-14.0.5-dealii-9.4-avx2-O3	5303c8	0	34	13574
tester	Clang-14.0.5-dealii-9.4-avx2-Ofast	5303c8	0	73	13535
tester	GNU-12.1.1-dealii-9.4-avx2-O3	5303c8	300	9	13623
tester	GNU-12.1.0-dealii-9.4-64bit_indices	5303c8	2	28	12969
tester	GNU-10.3.0-dealii-9.4-petsc_complex	5303c8	0	0	13158
tester	GNU-12.1.1-dealii-9.4-tets	5303c8	6	28	12985
e5d1d9e6850b	GNU-9.3.0-dealii-9.4	5303c8	0	8	11988

marcfehling · 2022-05-09T20:09:56Z

I will have a look at the sharedtria/refine_and_coarsen test (provided I can reproduce the problem).

EDIT: My guess is that we should pick a partitioner here, rather than using the auto setting.

kronbichler · 2022-05-09T20:54:03Z

@drwells how should be handle the nodal_renumbering cases? They fail on my machine because my compiler generates a different error message, namely

DEAL:0::void dealii::DoFRenumbering::compute_support_point_wise(std::vector<types::global_dof_index> &, const DoFHandler<dim, spacedim> &) [dim = 2, spacedim = 2]

rather than the one you added:

DEAL:0::void dealii::DoFRenumbering::compute_support_point_wise(std::vector<unsigned int>&, const dealii::DoFHandler<dim, spacedim>&) [with int dim = 2; int spacedim = 2]

Since the exact error message is not critical, and it seems to be a lot of manual work to get this robust on all different compilers and compiler versions, I suggest to simply remove the lines

dealii/tests/dofs/nodal_renumbering_01.cc

Lines 313 to 314 in 1d883b6

    
           for (std::size_t i = 3; i < 6; ++i) 
        
             deallog << lines[i] << std::endl;

and

dealii/tests/dofs/nodal_renumbering_01.cc

Lines 342 to 343 in 1d883b6

    
           for (std::size_t i = 3; i < 6; ++i) 
        
             deallog << lines[i] << std::endl;

(and the part extracting the lines).

marcfehling · 2022-05-09T23:42:06Z

@peterrum sharedtria/refine_and_coarsen_01 works on my machine. Before changing the partitioner as a shot in the dark, could you provide a verbose error message here? Thanks.

drwells · 2022-05-10T00:01:02Z

Yes, I think the nodal renumbering test fails because the assertions try to be too clever. I'll fix it.

peterrum · 2022-05-10T09:09:03Z

sharedtria/refine_and_coarsen_01 works on my machine. Before changing the partitioner as a shot in the dark, could you provide a verbose error message here? Thanks.

For some reason, CDash does not show the error messages. What happens? Which assert is triggered?

marcfehling · 2022-05-10T20:19:07Z

tjhei · 2022-05-10T20:22:35Z

mpi/create_mpi_datatype_01

I assume this is because your MPI version is very, very old. Not sure what to do about it.

tjhei · 2022-05-10T20:23:01Z

Also see #13638

peterrum · 2022-05-14T12:20:01Z

@marcfehling

matrix_free/ecl_03.mpirun=5.debug
matrix_free/ecl_03.mpirun=5.release
matrix_free/ecl_04.mpirun=5.debug
matrix_free/ecl_04.mpirun=5.release

What is the output here?

tjhei · 2022-05-15T15:25:45Z

also see the failing tests in #13458

marcfehling · 2022-05-15T23:00:05Z

matrix_free/ecl_03.mpirun=5.debug
matrix_free/ecl_03.mpirun=5.release
matrix_free/ecl_04.mpirun=5.debug
matrix_free/ecl_04.mpirun=5.release

What is the output here?

It looks like these tests are flagged Unstable on CDash https://cdash.dealii.43-1.org/test/8538014

I will check the output on our machines in a moment and provide them here.

marcfehling · 2022-05-16T05:27:00Z

mpi/create_mpi_datatype_01

I assume this is because your MPI version is very, very old. Not sure what to do about it.

matrix_free/ecl_03.mpirun=5.debug
matrix_free/ecl_03.mpirun=5.release
matrix_free/ecl_04.mpirun=5.debug
matrix_free/ecl_04.mpirun=5.release

What is the output here?

We have OpenMPI 1.10.7 on our machines. It believe this is the reason why these tests fail. Do we have an elegant way how to disable tests if the MPI library is too old? We could add an #ifdef in the main function, but maybe you have a better idea :)

You can find the ctest output to the tests here:

tjhei · 2022-05-16T15:43:19Z

Do we have an elegant way how to disable tests if the MPI library is too old?

That would hide the fact that your system is broken, so I am not sure this is a good idea. One option would be to check "MPI VERSION <=3.0" (see https://github.com/tjhei/BigMPICompat#test-results

kronbichler · 2022-05-20T07:19:39Z

I also see test failures on the test matrix_free/ecl_01, see e.g. https://cdash.dealii.43-1.org/test/10060936 and https://cdash.dealii.43-1.org/test/10048075 (the latter is when we build with SSE2 vectorization only, so that might give a hint). @peterrum can you take a look?

peterrum · 2022-05-22T20:31:17Z

@kronbichler I am able to reproduce the error locally when compiling with -msse2 (and also with VectorizedArray<double,2>). I'll try to fix this, but not sure when.

peterrum · 2022-05-22T21:01:54Z

@kronbichler I am pretty sure that the problem is the following lines

dealii/include/deal.II/matrix_free/mapping_info.templates.h

Line 3013 in a56c2ee

if (cell_type[cell] <= affine)

dealii/include/deal.II/matrix_free/mapping_info.templates.h

Lines 3036 to 3046 in a56c2ee

    
           DerivativeForm<1, dim, dim> inv_jac = 
        
             fe_val_neigh.jacobian(q).covariant_form(); 
        
           for (unsigned int d = 0; d < dim; ++d) 
        
             for (unsigned int e = 0; e < dim; ++e) 
        
               { 
        
                 const unsigned int ee = ExtractFaceHelper:: 
        
                   reorder_face_derivative_indices<dim>( 
        
                     fe_val_neigh.get_face_number(), e); 
        
                 face_data_by_cells[my_q] 
        
                   .jacobians[1][offset][d][e][v] = 
        
                   inv_jac[d][ee];

The reduced mapping of the neighbors is set up based on the current cell. In the test, we have a hyperball (for some reason I disabled in the test the manifolds) so that we have an affine cell in the center and the other cells are all general. When enabling the manifolds so that the inner cells are not affine the results are identical independent of the vectorization.

Do you have an idea how to address this? I would say that we need MappingInfo::neighbor_cell_type.

kronbichler · 2022-05-27T09:40:48Z

I had a look at the failing test multigrid-global-coarsening/interpolate_01.mpirun=1.release, which only fails in release mode and only recently, at some point during the last week. I see with valgrind:

==1000711== Invalid read of size 4
==1000711==    at 0x10FF2F10: dealii::internal::FineDoFHandlerView<2>::reinit(dealii::IndexSet const&, dealii::IndexSet const&, dealii::IndexSet const&, bool) (in /home/kronbichler/deal/mpi_build/lib/libdeal_II.so.9.4.0-pre)
==1000711==    by 0x10FFDAD8: dealii::internal::PermutationFineDoFHandlerView<2>::PermutationFineDoFHandlerView(dealii::DoFHandler<2, 2> const&, dealii::DoFHandler<2, 2> const&, unsigned int, unsigned int) (in /home/kronbichler/deal/mpi_build/lib/libdeal_II.so.9.4.0-pre)
==1000711==    by 0x10F901D9: void dealii::internal::MGTwoLevelTransferImplementation::reinit_polynomial_transfer<2, double>(dealii::DoFHandler<2, 2> const&, dealii::DoFHandler<2, 2> const&, dealii::AffineConstraints<double> const&, dealii::AffineConstraints<double> const&, unsigned int, unsigned int, dealii::MGTwoLevelTransfer<2, dealii::LinearAlgebra::distributed::Vector<double, dealii::MemorySpace::Host> >&) (in /home/kronbichler/deal/mpi_build/lib/libdeal_II.so.9.4.0-pre)
==1000711==    by 0x10F92F2D: dealii::MGTwoLevelTransfer<2, dealii::LinearAlgebra::distributed::Vector<double, dealii::MemorySpace::Host> >::reinit(dealii::DoFHandler<2, 2> const&, dealii::DoFHandler<2, 2> const&, dealii::AffineConstraints<double> const&, dealii::AffineConstraints<double> const&, unsigned int, unsigned int) (in /home/kronbichler/deal/mpi_build/lib/libdeal_II.so.9.4.0-pre)
==1000711==    by 0x125F50: void test<2, double>(unsigned int, unsigned int, bool) (in /home/kronbichler/deal/mpi_build/tests/multigrid-global-coarsening/interpolate_01.release/interpolate_01.release)
==1000711==    by 0x124F5A: main (in /home/kronbichler/deal/mpi_build/tests/multigrid-global-coarsening/interpolate_01.release/interpolate_01.release)
==1000711==  Address 0x0 is not stack'd, malloc'd or (recently) free'd

While I can't see line numbers (and I can't run valgrind on the debug version right now, it seems our library is too big for valgrind), I believe the most likely cause is some work related on the consensus algorithms or, as a second possibility, my work on the number cache or dof indices inside the triangulation. I can try to investigate in more detail later, but I just wanted to hear if anyone sees it immediately.

marcfehling · 2022-05-27T19:47:13Z

For the petsc/preconditioner_tvmult_01 test, some exception is caused with the latest version of petsc 3.17.1. It also occurs if only a single process is used (no MPI debugging!). With gdb I get the following info...

Program received signal SIGFPE, Arithmetic exception.
0x00007fffbc5d29f5 in hypre_BoomerAMGSolveT (amg_vdata=0x9fd920, A=0x90bc50, f=0xa027e0, u=0x909ca0) at par_amg_solveT.c:220
220	      conv_factor = resid_nrm / old_resid;
Missing separate debuginfos, use: debuginfo-install blas-3.4.2-8.el7.x86_64 glibc-2.17-326.el7_9.x86_64 hwloc-libs-1.11.8-4.el7.x86_64 infinipath-psm-3.3-26_g604758e_open.2.el7.x86_64 lapack-3.4.2-8.el7.x86_64 libfabric-1.7.2-1.el7.x86_64 libgfortran-4.8.5-44.el7.x86_64 libibumad-22.4-6.el7_9.x86_64 libibverbs-22.4-6.el7_9.x86_64 libnl3-3.2.28-4.el7.x86_64 libpsm2-11.2.78-1.el7.x86_64 librdmacm-22.4-6.el7_9.x86_64 libtool-ltdl-2.4.2-22.el7_3.x86_64 libuuid-2.23.2-65.el7_9.1.x86_64 numactl-libs-2.0.12-5.el7.x86_64 openmpi-1.10.7-5.el7.x86_64 opensm-libs-3.3.21-4.el7_9.x86_64 ucx-1.5.2-1.el7.x86_64 zlib-1.2.7-20.el7_9.x86_64

...and the following backtrace

#0  0x00007fffbc5d29f5 in hypre_BoomerAMGSolveT (amg_vdata=0x9fd920, A=0x90bc50, f=0xa027e0, u=0x909ca0) at par_amg_solveT.c:220
#1  0x00007fffbc58ffe8 in HYPRE_BoomerAMGSolveT (solver=0x9fd920, A=0x90bc50, b=0xa027e0, x=0x909ca0) at HYPRE_parcsr_amg.c:83
#2  0x00007fffbe857116 in PCApplyTranspose_HYPRE_BoomerAMG (pc=0xa02f50, b=0x8e9820, x=0x8ecc00) at petsc-3.17.1/src/ksp/pc/impls/hypre/hypre.c:631
#3  0x00007fffbe93afb2 in PCApplyTranspose (pc=0xa02f50, x=0x8e9820, y=0x8ecc00) at petsc-3.17.1/src/ksp/pc/interface/precon.c:611
#4  0x00007fffeb9d39ec in dealii::PETScWrappers::PreconditionBase::Tvmult (this=0x7fffffffd650, dst=..., src=...) at dealii/source/lac/petsc_precondition.cc:80
#5  0x0000000000432dc7 in test<dealii::PETScWrappers::PreconditionBoomerAMG> () at dealii/tests/petsc/preconditioner_tvmult_01.cc:85
#6  0x0000000000423d7e in main (argc=1, argv=0x7fffffffdfb8) at dealii/tests/petsc/preconditioner_tvmult_01.cc:102

This looks like a bug in hypre. We should come up with a minimal testcase and report it to them.

tjhei · 2022-06-09T14:52:01Z

Suggestions on how I should fix #13638 ?

tjhei · 2022-06-17T15:42:24Z

On my jenkins we are down to 3 failing tests:

	1355 - distributed_grids/3d_coarse_grid_02.debug (Failed)
	5975 - simplex/poisson_01.mpirun=1.debug (Failed)
	6003 - simplex/poisson_01.mpirun=4.debug (Failed)
	6319 - lac/distributed_vector_sm.mpirun=4.debug (Failed)

lac/distributed_vector_sm.
/jenkins/workspace/dealii_PR-13458/tests/lac/distributed_vector_sm.cc: In lambda function:

/jenkins/workspace/dealii_PR-13458/tests/lac/distributed_vector_sm.cc:76:7: internal compiler error: Segmentation fault

       vector.local_element(i) = 10 * my_rank + i;

       ^

Please submit a full bug report,

/jenkins/workspace/dealii_PR-13458/tests/simplex/poisson_01.cc: In function ‘void test(const dealii::Triangulation<dim, spacedim>&, const dealii::FiniteElement<dimension_, space_dimension_>&, const dealii::Quadrature<dim>&, const dealii::hp::QCollection<(dim - 1)>&, const dealii::Mapping<dim, spacedim>&, double, bool)’:

/jenkins/workspace/dealii_PR-13458/tests/simplex/poisson_01.cc:262:28: error: the value of ‘dealii::ReferenceCells::Triangle’ is not usable in a constant expression

       case ReferenceCells::Triangle:

                            ^

In file included from /jenkins/workspace/dealii_PR-13458/include/deal.II/grid/tria_description.h:24:0,

                 from /jenkins/workspace/dealii_PR-13458/include/deal.II/grid/tria.h:29,

                 from /jenkins/workspace/dealii_PR-13458/include/deal.II/distributed/repartitioning_policy_tools.h:19,

                 from /jenkins/workspace/dealii_PR-13458/include/deal.II/distributed/fully_distributed_tria.h:22,

                 from /jenkins/workspace/dealii_PR-13458/tests/simplex/poisson_01.cc:24:

/jenkins/workspace/dealii_PR-13458/include/deal.II/grid/reference_cell.h:792:41: note: ‘dealii::ReferenceCells::Triangle’ was not declared ‘constexpr’

   DEAL_II_CONSTEXPR const ReferenceCell Triangle =

see https://jenkins.tjhei.info/blue/organizations/jenkins/dealii/detail/PR-13458/10/pipeline/

peterrum added this to the Release 9.4 milestone May 9, 2022

This was referenced May 9, 2022

Fix test failure: Close the right AffineConstraints object #13701

Merged

Adjust test output for new FE_RaviartThomasNodal #13702

Merged

drwells self-assigned this May 10, 2022

drwells mentioned this issue May 10, 2022

Fix the nodal renumbering tests. #13711

Merged

kronbichler mentioned this issue May 10, 2022

Restrict feinterface/stokes test #13710

Merged

marcfehling mentioned this issue May 10, 2022

Fix test simplex/step-55. #13714

Merged

marcfehling mentioned this issue May 15, 2022

Added missing headers to failing test. #13739

Merged

drwells mentioned this issue May 16, 2022

Fix sundials 6 tests, part 1 #13741

Merged

This was referenced May 17, 2022

Update petsc test output. #13750

Merged

Check for range of function calls in bfgs tests. #13751

Merged

Fix compute_index_owner_01. #13752

Merged

kronbichler mentioned this issue May 18, 2022

MatrixFree: Add instantiation for ConstraintValues::insert_entries #13756

Merged

This comment was marked as outdated.

Sign in to view

marcfehling mentioned this issue May 27, 2022

Fix petsc/solver_08 for petsc 3.17.1 #13839

Merged

marcfehling mentioned this issue May 27, 2022

Fix test output for trilinos 13.2.0 #13840

Merged

kronbichler mentioned this issue May 30, 2022

Fix bug for global coarsening: Must allow invalid proc id #13858

Merged

drwells mentioned this issue Jun 1, 2022

Fix arkode 3 and 4 #13893

Merged

kronbichler mentioned this issue Jun 10, 2022

MatrixFree: Fix element-centric loops with different cell geometry compression #13952

Merged

This was referenced Jun 11, 2022

CGAL: fix header includes #13955

Merged

test cgal/cgal_surface_mesh_01: Update output #13956

Merged

contrib/python_bindings: set PYTHON_EXECUTABLE #13964

Merged

tamiko added the High Priority ⚠️ label Jun 14, 2022

This was referenced Jun 16, 2022

deal.II Release 9.4 - post-release tasks #13574

Closed

Fix failing symengine tests #14002

Merged

bangerth mentioned this issue Jun 18, 2022

Can't use ReferenceCells in switch-case. #14010

Merged

marcfehling mentioned this issue Jun 23, 2022

Disable HYPRE in the Tvmult test. #14032

Merged

tamiko closed this as completed Jun 24, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

List of currently failing tests #13703

List of currently failing tests #13703

peterrum commented May 9, 2022 •

edited by tamiko

marcfehling commented May 9, 2022 •

edited

kronbichler commented May 9, 2022

marcfehling commented May 9, 2022

drwells commented May 10, 2022

peterrum commented May 10, 2022

marcfehling commented May 10, 2022 •

edited

tjhei commented May 10, 2022

tjhei commented May 10, 2022

peterrum commented May 14, 2022

tjhei commented May 15, 2022

marcfehling commented May 15, 2022

marcfehling commented May 16, 2022 •

edited

tjhei commented May 16, 2022

This comment was marked as outdated.

kronbichler commented May 20, 2022

peterrum commented May 22, 2022 •

edited

peterrum commented May 22, 2022 •

edited

kronbichler commented May 27, 2022

marcfehling commented May 27, 2022 •

edited

tjhei commented Jun 9, 2022

tjhei commented Jun 17, 2022

List of currently failing tests #13703

List of currently failing tests #13703

Comments

peterrum commented May 9, 2022 • edited by tamiko

(Matthias) Failing tests with RC1:

marcfehling commented May 9, 2022 • edited

kronbichler commented May 9, 2022

marcfehling commented May 9, 2022

drwells commented May 10, 2022

peterrum commented May 10, 2022

marcfehling commented May 10, 2022 • edited

tjhei commented May 10, 2022

tjhei commented May 10, 2022

peterrum commented May 14, 2022

tjhei commented May 15, 2022

marcfehling commented May 15, 2022

marcfehling commented May 16, 2022 • edited

tjhei commented May 16, 2022

This comment was marked as outdated.

kronbichler commented May 20, 2022

peterrum commented May 22, 2022 • edited

peterrum commented May 22, 2022 • edited

kronbichler commented May 27, 2022

marcfehling commented May 27, 2022 • edited

tjhei commented Jun 9, 2022

tjhei commented Jun 17, 2022

peterrum commented May 9, 2022 •

edited by tamiko

marcfehling commented May 9, 2022 •

edited

marcfehling commented May 10, 2022 •

edited

marcfehling commented May 16, 2022 •

edited

peterrum commented May 22, 2022 •

edited

peterrum commented May 22, 2022 •

edited

marcfehling commented May 27, 2022 •

edited