Assertion during refinement of parallel distributed GridGenerator::quarter_hyper_ball

[stefan-kaessmair]

Dear all,
I encountered an error during refinement of a quarter hyper ball, therefore Daniel Arndt asked me to open a new issue. In several cycles I refine the boundary cells of the triangulation.
This simple example fails:

#include <deal.II/distributed/tria.h>
#include <deal.II/grid/grid_generator.h>
#include <deal.II/grid/grid_out.h>

int main (int argc, char *argv[]) {

    try {
        using namespace dealii;

        Utilities::MPI::MPI_InitFinalize mpi_initialization(argc, argv, 1);

        const unsigned int dim = 3;

        MPI_Comm mpi_communicator = MPI_COMM_WORLD;

        parallel::distributed::Triangulation<dim> tria (
                          mpi_communicator,
                          typename Triangulation<dim>::MeshSmoothing(Triangulation<dim>::smoothing_on_refinement),
                          parallel::distributed::Triangulation<dim>::default_setting);

        GridGenerator::quarter_hyper_ball(tria);

        for (unsigned int i_refinement = 0; i_refinement < 6; ++i_refinement)
        {
            auto cell = tria.begin_active();
            auto endc = tria.end();

            for (; cell!=endc; ++cell)
                if(cell->is_locally_owned() && cell->at_boundary())
                    cell->set_refine_flag ();

            tria.prepare_coarsening_and_refinement ();
            tria.execute_coarsening_and_refinement ();
        }
    }
    catch (std::exception &exec)
    {
        std::cout << std::flush;
        std::cerr << "\n\n------------------------------------------------\n"
                  << "Exception thrown :\n" << exec.what() << std::endl
                  << "Aborting!\n"
                  << "----------------------------------------------------" << std::endl;
        return 1;
    }
    catch (...)
    {
        std::cout << std::flush;
        std::cerr << "\n\n------------------------------------------------\n"
                  << "Unknown exception!\n"
                  << "Aborting!\n"
                  << "----------------------------------------------------" << std::endl;
        return 1;
    }

    return 0;
}

I receive the following error when running on a single core:

An error occurred in line <2764> of file </path/to/dealii-9.0.1/source/distributed/tria.cc> in function
    void dealii::parallel::distributed::Triangulation<dim, spacedim>::copy_local_forest_to_triangulation() [with int dim = 3; int spacedim = 3]
The violated condition was:
    static_cast<unsigned int>(parallel_forest->local_num_quadrants) == total_local_cells

When running on multiple cores I hit another error message (probably because there are no ghosts when using only one core):

An error occurred in line <2670> of file </path/to/dealii-9.0.1/source/distributed/tria.cc> in function
    void dealii::parallel::distributed::Triangulation<dim, spacedim>::copy_local_forest_to_triangulation() [with int dim = 3; int spacedim = 3]
The violated condition was:
    num_ghosts == parallel_ghost->ghosts.elem_count

When running my code, Daniel got the following error:

Abort: Expected '(int) edge_trees - (int) ta->elem_count': 'p8est_find_edge_transform_internal (conn, itree, iedge, ei, conn->edge_to_tree + ettae, conn->edge_to_edge + ettae, edge_trees)'
Abort: /home/darndt/Sources/p4est-2.2/src/p8est_connectivity.c:1111
Abort
[lap115343:18334] *** Process received signal ***
[lap115343:18334] Signal: Aborted (6)
[lap115343:18334] Signal code:  (-6)
[lap115343:18334] [ 0] /lib/x86_64-linux-gnu/libc.so.6(+0x3ef20)[0x7ff7d31b2f20]
[lap115343:18334] [ 1] /lib/x86_64-linux-gnu/libc.so.6(gsignal+0xc7)[0x7ff7d31b2e97]
[lap115343:18334] [ 2] /lib/x86_64-linux-gnu/libc.so.6(abort+0x141)[0x7ff7d31b4801]
[lap115343:18334] [ 3] /home/darndt/p4est-2.2/DEBUG/lib/libsc-2.2.so(sc_abort_verbose+0x0)[0x7ff7da6a1070]
[lap115343:18334] [ 4] /home/darndt/p4est-2.2/DEBUG/lib/libsc-2.2.so(sc_abort+0xd)[0x7ff7da6a0e5b]
[lap115343:18334] [ 5] /home/darndt/p4est-2.2/DEBUG/lib/libsc-2.2.so(sc_abort_verbosef+0x0)[0x7ff7da6a1104]
[lap115343:18334] [ 6] /home/darndt/p4est-2.2/DEBUG/lib/libp4est-2.2.so(p8est_find_edge_transform+0x1f4)[0x7ff7da9604b9]
[lap115343:18334] [ 7] /home/darndt/p4est-2.2/DEBUG/lib/libp4est-2.2.so(p8est_connectivity_is_valid+0x6a0)[0x7ff7da9547ed]
[lap115343:18334] [ 8] /home/darndt/dealii/build/lib/libdeal_II.g.so.9.2.0-pre(_ZN6dealii8parallel11distributed13TriangulationILi3ELi3EE31copy_new_triangulation_to_p4estESt17integral_constantIiLi3EE+0x3db)[0x7ff7e57cf53b]
[lap115343:18334] [ 9] /home/darndt/dealii/build/lib/libdeal_II.g.so.9.2.0-pre(_ZN6dealii8parallel11distributed13TriangulationILi3ELi3EE20create_triangulationERKSt6vectorINS_5PointILi3EdEESaIS6_EERKS4_INS_8CellDataILi3EEESaISC_EERKNS_11SubCellDataE+0x27)[0x7ff7e57edad7]
[lap115343:18334] [10] /home/darndt/dealii/build/lib/libdeal_II.g.so.9.2.0-pre(_ZN6dealii13GridGenerator18quarter_hyper_ballILi3EEEvRNS_13TriangulationIXT_EXT_EEERKNS_5PointIXT_EdEEd+0x647)[0x7ff7e4e20f47]
[lap115343:18334] [11] ./mwe(main+0x6c)[0x406ecc]
[lap115343:18334] [12] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xe7)[0x7ff7d3195b97]
[lap115343:18334] [13] ./mwe(_start+0x2a)[0x406d9a]
[lap115343:18334] *** End of error message ***

He pointed out that this might be related to #7428.

[masterleinad]

Also, see #8603.

[tamiko]

See also #7428