Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Changed extract_dofs so that it works in parallel #8302

Merged
merged 14 commits into from
Jun 11, 2019
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
29 changes: 15 additions & 14 deletions source/dofs/dof_tools.cc
Original file line number Diff line number Diff line change
Expand Up @@ -451,40 +451,41 @@ namespace DoFTools
const ComponentMask & component_mask,
std::vector<bool> & selected_dofs)
{
const FiniteElement<dim, spacedim> &fe = dof.begin_active()->get_fe();
(void)fe;

Assert(component_mask.represents_n_components(fe.n_components()),
Assert(component_mask.represents_n_components(
dof.get_fe_collection().n_components()),
ExcMessage(
"The given component mask is not sized correctly to represent the "
"components of the given finite element."));
Assert(selected_dofs.size() == dof.n_dofs(),
ExcDimensionMismatch(selected_dofs.size(), dof.n_dofs()));
Assert(selected_dofs.size() == dof.n_locally_owned_dofs(),
ExcDimensionMismatch(selected_dofs.size(),
dof.n_locally_owned_dofs()));

// two special cases: no component is selected, and all components are
// selected; both rather stupid, but easy to catch
if (component_mask.n_selected_components(n_components(dof)) == 0)
if (component_mask.n_selected_components(
dof.get_fe_collection().n_components()) == 0)
{
std::fill_n(selected_dofs.begin(), dof.n_dofs(), false);
std::fill_n(selected_dofs.begin(), dof.n_locally_owned_dofs(), false);
return;
}
else if (component_mask.n_selected_components(n_components(dof)) ==
n_components(dof))
else if (component_mask.n_selected_components(
dof.get_fe_collection().n_components()) ==
dof.get_fe_collection().n_components())
{
std::fill_n(selected_dofs.begin(), dof.n_dofs(), true);
std::fill_n(selected_dofs.begin(), dof.n_locally_owned_dofs(), true);
return;
}


// preset all values by false
std::fill_n(selected_dofs.begin(), dof.n_dofs(), false);
std::fill_n(selected_dofs.begin(), dof.n_locally_owned_dofs(), false);

// get the component association of each DoF and then select the ones
// that match the given set of components
std::vector<unsigned char> dofs_by_component(dof.n_dofs());
std::vector<unsigned char> dofs_by_component(dof.n_locally_owned_dofs());
internal::get_component_association(dof, component_mask, dofs_by_component);

for (types::global_dof_index i = 0; i < dof.n_dofs(); ++i)
for (types::global_dof_index i = 0; i < dof.n_locally_owned_dofs(); ++i)
if (component_mask[dofs_by_component[i]] == true)
selected_dofs[i] = true;
}
Expand Down
133 changes: 133 additions & 0 deletions tests/dofs/dof_tools_12a.cc
Original file line number Diff line number Diff line change
@@ -0,0 +1,133 @@
// ---------------------------------------------------------------------
//
// Copyright (C) 2003 - 2018 by the deal.II authors
//
// This file is part of the deal.II library.
//
// The deal.II library is free software; you can use it, redistribute
// it, and/or modify it under the terms of the GNU Lesser General
// Public License as published by the Free Software Foundation; either
// version 2.1 of the License, or (at your option) any later version.
// The full text of the license can be found in the file LICENSE.md at
// the top level directory of deal.II.
//
// ---------------------------------------------------------------------
#include <deal.II/base/logstream.h>

#include <deal.II/distributed/tria.h>

#include <deal.II/dofs/dof_accessor.h>
#include <deal.II/dofs/dof_handler.h>
#include <deal.II/dofs/dof_tools.h>

#include <deal.II/fe/fe_q.h>
#include <deal.II/fe/mapping_q_generic.h>

#include <deal.II/grid/grid_generator.h>
#include <deal.II/grid/grid_out.h>
#include <deal.II/grid/tria_iterator.h>

#include <algorithm>
#include <fstream>
#include <iomanip>
#include <string>

#include "../tests.h"

// check
// DoFTools::extract_dofs as in dof_tools_12 in parallel for fewer elements

void
output_bool_vector(std::vector<bool> &v)
{
for (unsigned int i = 0; i < v.size(); ++i)
deallog << (v[i] ? '1' : '0');
deallog << std::endl;
}


template <int dim, typename DoFHandlerType>
void
check_this(const DoFHandlerType &dof_handler)
{
IndexSet locally_relevant_dofs;
DoFTools::extract_locally_relevant_dofs(dof_handler, locally_relevant_dofs);

std::vector<bool> selected_dofs(dof_handler.n_locally_owned_dofs());
std::vector<bool> mask(dof_handler.get_fe().n_components(), false);

// only select first component
mask[0] = true;
DoFTools::extract_dofs(dof_handler, ComponentMask(mask), selected_dofs);
if (Utilities::MPI::this_mpi_process(MPI_COMM_WORLD) == 0)
{
output_bool_vector(selected_dofs);
}

// also select last component
mask.back() = true;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Currently, you are only ever checking FE_Q elements, i.e. scalar ones with only one component. Thus, the size of mask is one and you are testing the same thing twice.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the review! You are absolutely right, I added vector valued elements and (most) of the elements of dof_tools_common.
I also use now "MPILogInitAll" to get the output of all processes, but, as you said, the selected_dofs vectors are just the locally owned vectors, which are not mapped to global DoF indices. I don't think that this is a problem in this testing scenario, or am I wrong there? Therefore I wouldn't change this, if there is no objection.

Greetings
Mathias

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, it's not a problem that you are considering the locally owned degrees of freedom. At least for me, it is just unintuitive that the vector returned doesn't correspond to actual degrees of freedoms but has to be mapped using the IndexSet of locally owned DoFs to make sense (with a distributed mesh). We often define a separate function returning an IndexSet instead of a std::vector<bool> when porting functions from serial Triangulations to distributed ones. Of course, this behavior was pre-existing for the non-hp version and it's fine to extend that behavior to the hp case. Nevertheless, I would appreciate a comment in the documentation on how to interpret the returned object.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

masterleinad: OK, I understand. I explained this fact in the comment of the beginning of the test, or is there another place, where this should go?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for not being precise. Such a comment should go to

/**
* Extract the indices of the degrees of freedom belonging to certain vector
* components of a vector-valued finite element. The @p component_mask
* defines which components or blocks of an FESystem are to be extracted
* from the DoFHandler @p dof. The entries in the output array @p
* selected_dofs corresponding to degrees of freedom belonging to these
* components are then flagged @p true, while all others are set to @p
* false.
*
* If the finite element under consideration is not primitive, i.e., some or
* all of its shape functions are non-zero in more than one vector component
* (which holds, for example, for FE_Nedelec or FE_RaviartThomas elements),
* then shape functions cannot be associated with a single vector component.
* In this case, if <em>one</em> shape vector component of this element is
* flagged in @p component_mask (see
* @ref GlossComponentMask),
* then this is equivalent to selecting <em>all</em> vector components
* corresponding to this non-primitive base element.
*
* @param[in] dof_handler The DoFHandler whose enumerated degrees of freedom
* are to be filtered by this function.
* @param[in] component_mask A mask that states which components you want
* to select. The size of this mask must be compatible with the number of
* components in the FiniteElement used by the @p dof_handler. See
* @ref GlossComponentMask "the glossary entry on component masks"
* for more information.
* @param[out] selected_dofs A vector that will hold @p true or @p false
* values for each degree of freedom depending on whether or not it
* corresponds to a vector component selected by the mask above. The size
* of this array must equal DoFHandler::n_locally_owned_dofs(), which for
* sequential computations of course equals DoFHandler::n_dofs(). The
* previous contents of this array are overwritten.
*/

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, I didn't understand this, it was just "obvious" to me, that this is, what the function does. I have added a little note in the description and hope this is OK.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, that looks OK. How did you verify the correctness of the test output? 😉

Copy link
Contributor Author

@mathsen mathsen Jun 5, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't... correct until someone else proofs the opposite ;)

//Edit: To be more precise, the main code change (extract_dofs()) works in a convergence test for me for a real world incompressible flow problem with a vector valued FE. So I strongly assume that the main change here does what it should do. Nevertheless I didn't check the output of the test file.

DoFTools::extract_dofs(dof_handler, ComponentMask(mask), selected_dofs);
if (Utilities::MPI::this_mpi_process(MPI_COMM_WORLD) == 0)
{
output_bool_vector(selected_dofs);
}
}


template <int dim>
void
check(const FiniteElement<dim> &fe, const std::string &name)
{
deallog << "Checking " << name << " in " << dim << "d:" << std::endl;

// create tria and dofhandler
// objects. set different boundary
// and sub-domain ids
parallel::distributed::Triangulation<dim> tria(MPI_COMM_WORLD);
GridGenerator::hyper_cube(tria, 0., 1.);
tria.refine_global(1);
for (unsigned int ref = 0; ref < 2; ++ref)
{
for (auto &cell : tria.active_cell_iterators())
if (cell->is_locally_owned() && cell->center()(0) < .5 &&
cell->center()(1) < .5)
cell->set_refine_flag();
tria.execute_coarsening_and_refinement();
}

DoFHandler<dim> dof_handler(tria);
dof_handler.distribute_dofs(fe);

// setup hp DoFHandler
hp::FECollection<dim> fe_collection(fe);
hp::DoFHandler<dim> hp_dof_handler(tria);
hp_dof_handler.distribute_dofs(fe_collection);

check_this<dim, DoFHandler<dim>>(dof_handler);
check_this<dim, hp::DoFHandler<dim>>(hp_dof_handler);
}


#define CHECK(EL, deg, dim) \
{ \
FE_##EL<dim> EL(deg); \
check(EL, #EL #deg); \
}

#define CHECK_ALL(EL, deg) \
{ \
CHECK(EL, deg, 2); \
CHECK(EL, deg, 3); \
}

int
main(int argc, char **argv)
{
Utilities::MPI::MPI_InitFinalize mpi_initialization(argc, argv, 1);
mpi_initlog();
CHECK_ALL(Q, 1);

CHECK_ALL(Q, 2);

return 0;
}
21 changes: 21 additions & 0 deletions tests/dofs/dof_tools_12a.mpirun=3.output
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@

DEAL::Checking Q1 in 2d:
DEAL::111111111111111
DEAL::111111111111111
DEAL::111111111111111
DEAL::111111111111111
DEAL::Checking Q1 in 3d:
DEAL::111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111
DEAL::111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111
DEAL::111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111
DEAL::111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111
DEAL::Checking Q2 in 2d:
DEAL::111111111111111111111111111111111111111111111
DEAL::111111111111111111111111111111111111111111111
DEAL::111111111111111111111111111111111111111111111
DEAL::111111111111111111111111111111111111111111111
DEAL::Checking Q2 in 3d:
DEAL::11111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111
DEAL::11111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111
DEAL::11111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111
DEAL::11111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111
21 changes: 21 additions & 0 deletions tests/dofs/dof_tools_12a.output
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@

DEAL::Checking Q1 in 2d:
DEAL::11111111111111111111111111111111111111111
DEAL::11111111111111111111111111111111111111111
DEAL::11111111111111111111111111111111111111111
DEAL::11111111111111111111111111111111111111111
DEAL::Checking Q1 in 3d:
DEAL::11111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111
DEAL::11111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111
DEAL::11111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111
DEAL::11111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111
DEAL::Checking Q2 in 2d:
DEAL::111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111
DEAL::111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111
DEAL::111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111
DEAL::111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111
DEAL::Checking Q2 in 3d:
DEAL::11111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111
DEAL::11111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111
DEAL::11111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111
DEAL::11111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111