Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] Introduce FERemoteEvaluation #15982

Closed
wants to merge 11 commits into from
Closed

Conversation

jh66637
Copy link
Contributor

@jh66637 jh66637 commented Sep 12, 2023

The new class can be used to access values in quadrature points on remote triangulations similarly to FEEvaluation, FEFaceEvaluation, and FEPointEvaluation. It can be used for volume coupling as well as the coupling of non-matching interfaces.

Current implementations use quadrature points dictated by integration points of given FEEvaluation type objets. However, the structure of the classes is chosen with mortaring (with CGAL and FEPointEvaluation) in mind.

@peterrum @fdrmrc

@jh66637 jh66637 marked this pull request as draft September 12, 2023 10:14
Copy link
Member

@peterrum peterrum left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few comments. Some tests and a change-log entry would be good!

include/deal.II/matrix_free/fe_remote_evaluation.h Outdated Show resolved Hide resolved
include/deal.II/matrix_free/fe_remote_evaluation.h Outdated Show resolved Hide resolved
/**
* Offset to data after last call of `reinit()`.
*/
unsigned int data_offset = numbers::invalid_unsigned_int;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This as well.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should start and cell_start in FEREDataView also be initialized in a constructor to conform with the deal.II style (even though no constructor is present yet) or is it OK?

See: https://github.com/dealii/dealii/pull/15982/files#diff-c360108d85e19ac86c9dd9a82f59988c6611b77a137adeae4b9a0442052e4dbaR307

int n_components_,
int dim_>
void
copy_data(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Documentation is missing.

VectorizedArrayType>,
true>
{
static constexpr bool cell_face_pairs = false;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't this be true!?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No. This is because FEFaceEvaluation::reinit(face_index) takes a unique face batch ID instead of a cell_index and face_number. This is contrary to FEPointEvaluation::reinit(cell_index,face_number) (if setup for faces).

include/deal.II/matrix_free/fe_remote_evaluation.h Outdated Show resolved Hide resolved
* if @p FEEvaluationType works on faces.
*/
template <typename FEEvaluationType_, bool is_face_>
class FERemoteEvaluationCommunicator : public Subscriptor
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this class not internal?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FERemoteEvaluationCommunicator is set up by the user and handed to FEPointEvaluation. I think we should avoid that classes and functions, intended to end up in user-code, are internal. I am open for discussion though :)

@peterrum
Copy link
Member

Let me summarize the purpose of the classes:

  • FERemoteEvaluationData stores the values and gradients for all requested points.
  • FERemoteEvaluationDataView gives a structured access to the values/gradients of a cell/face via get_shiftIO
  • FERemoteEvaluationCommunicator fills the values/gradients in data using RPE
  • FERemoteEvaluation is a class similar to FEEvaluation but accessing precomputed/cached values/gradients in FERemoteEvaluationData. The values/gradients of all points are updated via gather_evaluate(vector, flags). The function reinit(cell) shifts the view to the current cell/face. The functions get_value(q) and get_gradient(q) give access to the value/gradient of the specified cell.

@jh66637 Is the description correct?

I am wondering if FERemoteEvaluation needs gather_evaluate(vector, flags). The data could be also filled manually without FERemoteEvaluationCommunicator`.

@jh66637
Copy link
Contributor Author

jh66637 commented Sep 14, 2023

@peterrum Your description is perfect.

Filling the data without the communicator is tedious. The information on the memory positions of values and gradients are known by DataView which is stored in the Communicator. The gather_evaluate() function passes the data to be filled (stored in FERemoteEval) to the Communicator, which does the work. I like this because

  • it is a similar interface compared to FEEvaluation
  • the user does not have to know how values are stored and accessed in Data
    Correct me if I misunderstood your comment or if you had something different in mind.

Copy link
Member

@peterrum peterrum left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I went through the code and added some further comments. At the moment, the code is hard to read because of the many template arguments.

Filling the data without the communicator is tedious. The information on the memory positions of values and gradients are known by DataView which is stored in the Communicator.

I still think to split up the responsibilities makes sense. One class contains the data. Another one adds the notion of cells/faces (by offsets). Another fills the data by communication. And the last class accesses the data using the offsets.

Let's assume that the data is not filled via communication but contains some precomputed quantities, e.g., linerization points in a non-linear context or some variable coefficients. This can be filled quite easily manually and all classes but the one which communicates would be useful in this context.

include/deal.II/matrix_free/fe_remote_evaluation.h Outdated Show resolved Hide resolved
include/deal.II/matrix_free/fe_remote_evaluation.h Outdated Show resolved Hide resolved
include/deal.II/matrix_free/fe_remote_evaluation.h Outdated Show resolved Hide resolved
include/deal.II/matrix_free/fe_remote_evaluation.h Outdated Show resolved Hide resolved
include/deal.II/matrix_free/fe_remote_evaluation.h Outdated Show resolved Hide resolved
auto rpe =
std::make_shared<Utilities::MPI::RemotePointEvaluation<dim>>(tol,
false,
0);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why 0?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the rtree_level which is defaulted to 0 if not specified.

include/deal.II/matrix_free/fe_remote_evaluation.h Outdated Show resolved Hide resolved
include/deal.II/matrix_free/fe_remote_evaluation.h Outdated Show resolved Hide resolved
jh66637 and others added 10 commits November 20, 2023 18:07
Co-authored-by: Peter Munch <peterrmuench@gmail.com>
Co-authored-by: Peter Munch <peterrmuench@gmail.com>
Co-authored-by: Peter Munch <peterrmuench@gmail.com>
Co-authored-by: Peter Munch <peterrmuench@gmail.com>
Co-authored-by: Peter Munch <peterrmuench@gmail.com>
Co-authored-by: Peter Munch <peterrmuench@gmail.com>
Co-authored-by: Peter Munch <peterrmuench@gmail.com>
Co-authored-by: Peter Munch <peterrmuench@gmail.com>
Co-authored-by: Peter Munch <peterrmuench@gmail.com>
Co-authored-by: Peter Munch <peterrmuench@gmail.com>
@jh66637
Copy link
Contributor Author

jh66637 commented Nov 27, 2023

The implementations are outdated. I changed a lot during the last weeks so I will close this. All of the suggestions from the review have been included in the new implementations.

@jh66637 jh66637 closed this Nov 27, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants