Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

step-66: a nonlinear, parallel matrix-free example [WIP] #8229

Closed
wants to merge 68 commits into from

Conversation

tjhei
Copy link
Member

@tjhei tjhei commented May 17, 2019

I am working with Fabian Castelli on a nonlinear, parallel, matrix-free
tutorial. Because this is the second time my chosen tutorial number was
taken, I would like to reserve it with this PR.

We can either merge this as is or leave it open for now.

I am working with Fabian Castelli on a nonlinear, parallel, matrix-free
tutorial. Because this is the second time my chosen tutorial number was
taken, I would like to reserve it with this PR.
@bangerth
Copy link
Member

What's the status here?

@bangerth
Copy link
Member

bangerth commented May 7, 2020

Ping? Is this still in the works?

@bangerth
Copy link
Member

@tjhei Can I steal step-66? I'd like to put the particle example before step-70, and there may be two of them (which would claim 68 as well). You could then take step-71?

@tjhei
Copy link
Member Author

tjhei commented May 21, 2020

@tjhei Can I steal step-66?

I would like to keep it as step-66 if possible. I dropped the ball on it, but it is more or less ready to be looked at / merged.

@kronbichler
Copy link
Member

Great to see this finalized! Let me know when I should start looking at it.

@bangerth
Copy link
Member

What would be the downside for you to rename it to 71?

I'm just thinking that it would be nice to have the two new particle tutorials numerically before the first "real" application step-70. Do you already have links to step-66 spread everywhere, or have you used that name anywhere before?

@drwells
Copy link
Member

drwells commented May 21, 2020

@bangerth but you already reserved step-68 for particles in #10043 - shouldn't that suffice?

@bangerth
Copy link
Member

We're going to write two tutorial programs that would conceptually sit before step-70, and I was hoping to get 66 and 68 for those...

Copy link
Member

@kronbichler kronbichler left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like this program, it shows a lot of new things. I did some random remarks on language on the parts where you're already reached a good state, and add some bigger comments where I think you could adapt things a bit. I realized too late (after too many comments) that I could have pushed them myself, but I will do so once we have agreed on the bigger topics.

On the unit circle $\Omega = \bigl\{ x \in \mathbb{R}^2 : \|x\| \leq 1 \bigr\}$
we consider the following nonlinear elliptic boundary value problem subject to a
homogeneous Dirichlet boundary condition: Find a function
$u\colon\Omega\to\mathbb{R}$ such that holds:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
$u\colon\Omega\to\mathbb{R}$ such that holds:
$u\colon\Omega\to\mathbb{R}$ such that it holds:



<h3>Discretization with finite elements</h3>
Even it is a nonlinear problem, we first derive the weak formulation for this
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Even it is a nonlinear problem, we first derive the weak formulation for this
As usual, we first derive the weak formulation for this

(I do not think the nonlinearity matters for the weak form.)

<h3>Discretization with finite elements</h3>
Even it is a nonlinear problem, we first derive the weak formulation for this
problem by multiplying with a smooth test function $v\colon\Omega\to\mathbb{R}$
respecting the boundary condition and integrate over the domain $\Omega$.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
respecting the boundary condition and integrate over the domain $\Omega$.
respecting the boundary condition and integrating over the domain $\Omega$.

respecting the boundary condition and integrate over the domain $\Omega$.
Integration by parts and putting the term on the right hand side to the left
yields the weak formulation: Find a function $u\colon\Omega\to\mathbb{R}$
such that for all test functions $v$ holds:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
such that for all test functions $v$ holds:
such that for all test functions $v$ it holds:

Choosing the Lagrangian finite element space $V_h$ we can define a basis
$\{\varphi_i\}_{i=1,\dots,N}$ and it suffices to test only with those basis
functions. So the discrete problem reads as follows: Find $u_h\in V_h$ such that
for all $i=1,\dots,N$ holds:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
for all $i=1,\dots,N$ holds:
for all $i=1,\dots,N$ it holds:

// object handling the loop over all cells, and a local_apply function
// implementing the calculation of the cell contribution.

// TODO: As functor with operator() function?
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree on that - I would remove the boilerplate of having a class and simply write the code inline where you use the residual with a functor - or add a function residual to the JacobianOperator class (well, then the name is not so clear).

{
TimerOutput::Scope t(computing_timer, "assemble right hand side");

residual_operator.apply(system_rhs, solution);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, moving the code with a functor here would be very nice.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just implemented a version, which is now even faster as with the ResidualOperator approach.

// important if we would use an adaptive version of the Newton method. Then
// for example we would compute the residual for different step lengths and
// compare the residuals. However for our problem the full Newton step with
// $\alpha=1$ is the best we can do. An adaptive verison of Newton's method
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// $\alpha=1$ is the best we can do. An adaptive verison of Newton's method
// $\alpha=1$ is the best we can do. An adaptive version of Newton's method

Comment on lines 773 to 774
MGTransferMatrixFree<dim, float> mg_transfer(mg_constrained_dofs);
mg_transfer.build(dof_handler);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you make MGTransferMatrixFree a class member? From an educational point of view, I think it would make sense to show that the transfer only needs to be set up once and can be re-used in subsequent nonlinear iterations. This is different to the mg_smoother object that must indeed be rebuilt because the diagonal changes (even though one could try to find a balance of not recomputing in every iteration - that would make for a good possibility for extensions, though).

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thats a good point! But maybe I didnt get the idea for the 'possibility of extensions'.

Comment on lines 988 to 1010
std::ofstream output(
"solution-" + Utilities::to_string(cycle, 2) + "." +
Utilities::to_string(Utilities::MPI::this_mpi_process(MPI_COMM_WORLD),
4) +
".vtu");
data_out.write_vtu(output);

if (Utilities::MPI::this_mpi_process(MPI_COMM_WORLD) == 0)
{
std::vector<std::string> filenames;
for (unsigned int i = 0;
i < Utilities::MPI::n_mpi_processes(MPI_COMM_WORLD);
++i)
{
filenames.emplace_back("solution-" +
Utilities::to_string(cycle, 2) + "." +
Utilities::to_string(i, 4) + ".vtu");
}
std::ofstream master_output("solution-" +
Utilities::to_string(cycle, 2) + ".pvtu");
data_out.write_pvtu_record(master_output, filenames);
}
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you want to use DataOut::write_vtu_with_pvtu_record?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, thats the function we want to use. Unfortunately, when writing this code, this functions was not yet available.

@ghost
Copy link

ghost commented May 3, 2021

Thanks for the comments on this code and sorry for the long time without any reaction from my side. I was hardly stressed in writing my phd but now I am on a postdoc position.

I just created a pull request to tjhei:step-66 with a small update. Major missing point is the results section (which could consist of a wall clock comaprison as in step-50) and the reorganization of the residual evaluation. So the easiest way would be to write the corresponding code in a fuction, which I could do tomorrow morning?

@tjhei
Copy link
Member Author

tjhei commented May 4, 2021

Hi @gfcas, I am sorry that I have been sitting on it for 12 months. 😢
Thank you for the help. I was going to draft a result section this week. With at least the minor edits done, we can hopefully merge the tutorial.

@ghost
Copy link

ghost commented May 4, 2021

Hi @tjhei dont worry ;-) I am just updating the evaluation of the residual according to @kronbichler's comment, which I can push later.

@ghost
Copy link

ghost commented May 4, 2021

@tjhei OK, I commited the newest version and updated the pr to the step-66 branch of your dealii fork. One minor issue opened: If we build the patches with given mapping and fe_degree the paraview output looks somehow strange for coarse grids.

Copy link
Member

@marcfehling marcfehling left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The results section is still missing, but otherwise the tutorial already looks very good!

Have you checked that doxygen generates the documentation correctly (i.e. parsing MathJax formulas, alignment of pictures)?

I'll leave some general remarks in this PR. @gfcas I'll open pull requests on your branch with other minor suggestions, see https://github.com/gfcas/dealii/pull/3, https://github.com/gfcas/dealii/pull/4, https://github.com/gfcas/dealii/pull/5, https://github.com/gfcas/dealii/pull/6.

Comment on lines +25 to +27
<a href="https://link.springer.com/book/10.1007/978-1-4612-4546-9">
Mathematical Problems from Combustion Theory
by Jerrold Bebernes and David Eberly</a>.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you want to add a proper literature reference in doc/doxygen/references.bib?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Of course could be done, but I suppose this is the only place where this book is cited. What is the general guideline?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have a look at doc/doxygen/references.bib. You'll find sections designated for each step with literature specific for the tutorial. Feel free to create such a section for step-66 and add the literature reference there.

Newton step $A = F'(u^n)$. Hence before we need to tell the function that
computes the system matrix about the solution at the last Newton step. In an
implementation with a classical
assemble_system function we would gather this information from the last Newton
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
assemble_system function we would gather this information from the last Newton
<code>assemble_system()</code> function we would gather this information from the last Newton

Here and later, you can highlight functions like this if you want to.

Comment on lines 102 to 103
step during assembly by the use of the member functions get_function_values and
get_function_gradients of the FEValuesBase class. The assemble_system function
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
step during assembly by the use of the member functions get_function_values and
get_function_gradients of the FEValuesBase class. The assemble_system function
step during assembly by the use of the member functions FEValuesBase::get_function_values() and
FEValuesBase::get_function_gradients(). The assemble_system function

Here and later: If you write it like this, doxygen will be place a link to the member functions in the documentation.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes thanks for this hint.

Copy link
Member

@peterrum peterrum left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks quite good already! I would be in favor of including this tutorial in the 9.3 release. A non-linear matrix-free tutorial was definitely missing (and would have helped one or two persons I know)! Personally, I find the usage of multigrid here a bit of an overkill but in the combination with the transfer of the solution to the levels has its useful purpose 👍

examples/step-66/doc/intro.dox Show resolved Hide resolved
Comment on lines +183 to +187
// the quadrature points with the gather_evaluate function. Remember if we
// only loop over cells this function is just a wrapper around the functions
// read_dof_values and evaluate. Since we store the evaluated values of the
// finite element function in a table we do not have to call integrate in
// combination with distribute_local_to_global or integrate scatter.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// the quadrature points with the gather_evaluate function. Remember if we
// only loop over cells this function is just a wrapper around the functions
// read_dof_values and evaluate. Since we store the evaluated values of the
// finite element function in a table we do not have to call integrate in
// combination with distribute_local_to_global or integrate scatter.
// the quadrature points with the gather_evaluate function. We store the evaluated
// values of the finite element function directly in a table.

Comment on lines +166 to +174
// The clear function resets the table holding the values for the
// nonlinearity and call the clear function of the base class.
template <int dim, int fe_degree, typename number>
void JacobianOperator<dim, fe_degree, number>::clear()
{
nonlinear_values.reinit(0, 0);
MatrixFreeOperators::Base<dim, LinearAlgebra::distributed::Vector<number>>::
clear();
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this really needed?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When writing the first version of this example I copied the MatrixFreeOperator from step-37 an adapted it. So this function exsists in accordance with step-37. If it is totally unnecessary we can of course delete it.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the umber of iterations stay the same, it can be deleted ;)

examples/step-66/step-66.cc Show resolved Hide resolved
Comment on lines +266 to +341
// The remaining two functions of the JacobianOperator calculate the diagonal
// entries of the Jacobian. The only difference compared to step-37 is the
// calculation of the cell contribution in the local_compute_diagonal
// function. Therefore, we only have to extend and change the
// arguments for the submit functions in the loop over all quadrature points
// and this can be done according to the local_apply function. So no further
// comments to these two functions should be necessary.
template <int dim, int fe_degree, typename number>
void JacobianOperator<dim, fe_degree, number>::compute_diagonal()
{
this->inverse_diagonal_entries.reset(
new DiagonalMatrix<LinearAlgebra::distributed::Vector<number>>());
LinearAlgebra::distributed::Vector<number> &inverse_diagonal =
this->inverse_diagonal_entries->get_vector();
this->data->initialize_dof_vector(inverse_diagonal);

unsigned int dummy = 0;

this->data->cell_loop(&JacobianOperator::local_compute_diagonal,
this,
inverse_diagonal,
dummy);

this->set_constrained_entries_to_one(inverse_diagonal);

for (unsigned int i = 0; i < inverse_diagonal.local_size(); ++i)
{
Assert(
inverse_diagonal.local_element(i) > 0.,
ExcMessage(
"No diagonal entry in a positive definite operator should be zero"));
inverse_diagonal.local_element(i) =
1. / inverse_diagonal.local_element(i);
}
}



template <int dim, int fe_degree, typename number>
void JacobianOperator<dim, fe_degree, number>::local_compute_diagonal(
const MatrixFree<dim, number> & data,
LinearAlgebra::distributed::Vector<number> &dst,
const unsigned int &,
const std::pair<unsigned int, unsigned int> &cell_range) const
{
FEEvaluation<dim, fe_degree, fe_degree + 1, 1, number> phi(data);

AlignedVector<VectorizedArray<number>> diagonal(phi.dofs_per_cell);

for (unsigned int cell = cell_range.first; cell < cell_range.second; ++cell)
{
AssertDimension(nonlinear_values.size(0), data.n_cell_batches());
AssertDimension(nonlinear_values.size(1), phi.n_q_points);

phi.reinit(cell);
for (unsigned int i = 0; i < phi.dofs_per_cell; ++i)
{
for (unsigned int j = 0; j < phi.dofs_per_cell; ++j)
phi.submit_dof_value(VectorizedArray<number>(), j);
phi.submit_dof_value(make_vectorized_array<number>(1.), i);

phi.evaluate(true, true);
for (unsigned int q = 0; q < phi.n_q_points; ++q)
{
phi.submit_value(-nonlinear_values(cell, q) * phi.get_value(q),
q);
phi.submit_gradient(phi.get_gradient(q), q);
}
phi.integrate(true, true);
diagonal[i] = phi.get_dof_value(i);
}
for (unsigned int i = 0; i < phi.dofs_per_cell; ++i)
phi.submit_dof_value(diagonal[i], i);
phi.distribute_local_to_global(dst);
}
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could I motivate you to use MatrixFreeTools::compute_diagonal() here?

Copy link

@ghost ghost May 14, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point, this funciton looks good, however, @peterrum would you please take a look on this, whether it is possible in the context here? Yesterday evening it was a bit late and I just tried it straightforewardly following @marcfehling's step-75. I arrived at the problem where his do_cell_integral_local needs in the nonlinear case to know the cell number for getting the value out of the nonlinear_values table.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Of course ... thanks, now it works.

Comment on lines +623 to +660
template <int dim, int fe_degree>
void GelfandProblem<dim, fe_degree>::evaluate_residual(
LinearAlgebra::distributed::Vector<double> & dst,
const LinearAlgebra::distributed::Vector<double> &src) const
{
// First we update the ghost values of the given input vector src and clear
// the output vector dst.
src.update_ghost_values();
dst = 0.0;

// Then we get a reference to the MatrixFree object stored in the
// JacobianOperator and set up the FEEvaluation.
const MatrixFree<dim, double> &data = *system_matrix.get_matrix_free();
FEEvaluation<dim, fe_degree> phi(data);

// At the main part of this function we loop over all cell batches defined
// in the MatrixFree object and compute the residual evaluation, by
// evaluating the input vector and integrate against the test functions
// according to the weak formulation of the Gelfand problem.
for (unsigned int cell = 0; cell < data.n_cell_batches(); ++cell)
{
phi.reinit(cell);
phi.read_dof_values_plain(src);
phi.evaluate(true, true);

for (unsigned int q = 0; q < phi.n_q_points; ++q)
{
phi.submit_value(-std::exp(phi.get_value(q)), q);
phi.submit_gradient(phi.get_gradient(q), q);
}

phi.integrate_scatter(true, true, dst);
}

// Finally, we must not forget to initiate the MPI data exchange via the
// compress function.
dst.compress(VectorOperation::add);
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why are you not using a normal cell-loop? In that case you could skip all the MPI vector handling.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Originally this was done in a ResidualOperator with the cell_loop(), but then shifted to member function. I observed, that with this implementations is slightly faster than with the cell_loop in an extra operator. Is there a nice way to use the cell loop directly in the evaluate_residual function without adding an further function for the local_cell_operation?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can pass a lambda function to MatrixFree::cell_loop() like in step 76. But if you do this, I would like to be consistent in the whole tutorial: either member function or lambda function.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I observed, that with this implementations is slightly faster than with the cell_loop in an extra operator.

Personally, I have doubts that this version is faster than the normal version. You are doing dst = 0.0; which adds an additional read/write operation (which could be hidden by MatrixFree).

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can pass a lambda function to MatrixFree::cell_loop() like in step 76. But if you do this, I would like to be consistent in the whole tutorial: either member function or lambda function.

Now it uses the cell_loop (implemented with an additional member function local_evaluate_residual).

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I observed, that with this implementations is slightly faster than with the cell_loop in an extra operator.

Personally, I have doubts that this version is faster than the normal version. You are doing dst = 0.0; which adds an additional read/write operation (which could be hidden by MatrixFree).

In the past, I did some time measurements with prior dst=0.0 and false flag compared to directly true flag. However, I have never seen a clear result which is faster...

template <int dim, int fe_degree>
double GelfandProblem<dim, fe_degree>::compute_solution_norm() const
{
solution.update_ghost_values();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

May I ask you that you always when you call update_ghost_values() you also call zero_out_ghost_values(). Just regard it as malloc-free or new-delete. If you let MatrixFree do the the update_ghost_values and compression you often are able get a speedup of a couple of percent.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Honestly, maybe I didnt get your comment... I thought, that for the evaluation ghost values must be active. But with zero_ou_ghost_values() I would remove them? How can I make MatrixFree do that?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Honestly, maybe I didnt get your comment... I thought, that for the evaluation ghost values must be active. But with zero_ou_ghost_values() I would remove them? How can I make MatrixFree do that?

I meant to call zero_out_ghost_values() at the end of the function. This shouldn't be a problem, or?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thats of course no problem, zeroing out the ghost in the end of each function updating the ghosts.

examples/step-66/step-66.cc Show resolved Hide resolved
// @sect3{The main function}

// As typical for programs running in parallel with MPI we set up the MPI
// framework and limit the number of threads to one. Finally to run the solver
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

// framework. Finally to run the solver

I would not be surprised that the code also works with threads. Could you try it out. Thx.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Of course I will try it. BUt maybe let us postpone this detail to a later update of the example, as time is running?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure.

Comment on lines +94 to +97

#
Fabian Castelli <fabian.castelli@kit.edu>
Fabian Castelli <fabian.castelli@kit.edu> <50630942+gfcas@users.noreply.github.com>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we still need to do this manully?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@tjhei maybe you know about that? Personally I have no knowledge on these internals.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There shouldn't really be any need for this if you have set up your GitHub account properly (and we don't already have commits with the "wrong" e-mail-address in the repository). I guess this was just a quick and dirty solution.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So nothing to do for me?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just remove these lines.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK

@ghost
Copy link

ghost commented May 14, 2021

@tjhei may I ask: you mentioned, that you will goning for a draft of the results section. Do you have already something? Should I try a draft?

@ghost
Copy link

ghost commented May 14, 2021

@tjhei @peterrum @marcfehling I created a new pr tjhei#12 with some updates according your suggestions. Thanks a lot for your comments so far!

@ghost
Copy link

ghost commented May 14, 2021

Sorry for the circumstances, forget about tjhei#12 I opened #12201 directly. So this pr can be closed?

@peterrum
Copy link
Member

Sorry for the circumstances, forget about tjhei#12 I opened #12201 directly. So this pr can be closed?

I am fine with closing this PR. Except @tjhei gives you the right to his branch, this should make our life easier!

@peterrum
Copy link
Member

@tjhei may I ask: you mentioned, that you will goning for a draft of the results section. Do you have already something? Should I try a draft?

I don't think you don't need to much results. I think it is a good result if MatrixFree users know after reading this tutorial that they can solve non-linear problems with MatrixFree and what the steps are!

@tjhei tjhei closed this May 14, 2021
@ghost ghost mentioned this pull request May 19, 2021
5 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

7 participants