Dof distribution slow

[termi-official]

The current dof distribution algorithm works well, but it is also very slow. This can be a deal-breaker for problems with changing mesh topology of on problems where multiple dof handlers may be used. We should explore different strategies to distribute the dofs faster, (e.g. via fast paths or by using additional information that can be precomputed).

Minimal example is takes from Kim ( #629 )

using Ferrite
using BenchmarkTools

grid = generate_grid(Quadrilateral, (1000, 1000))
ip_v = Lagrange{2,RefCube,2}()
ip_s = Lagrange{2,RefCube,1}()

function setup_close(grid, T)
    dh = T(grid)
    add!(dh, :v, 2, Lagrange{2,RefCube,2}()) 
    add!(dh, :s, 1, Lagrange{2,RefCube,1}()) 
    return dh
end

@btime close!(dh) setup=(dh=setup_close($grid, $DofHandler)) evals=1; # 1.542 s (224 allocations: 565.04 MiB)

[kimauth]

I'm sure there are things that could be optimised in the dof distribution, but is ~0.65 s really that slow for a problem with ~9e6 dofs? Wouldn't the assembly or the linear solve be much more likely candidates for a performance bottleneck at that system size?

[termi-official]

Depends on the problem. I have some traveling wave problem where I was able to optimize assembly and solve quite a lot, but my current implementation is capped by the AMR cost (have a quite customized C++ implementation).

Reference timings here are (not optimized dof distribution here either):

dof distribution time: 0.678458s
Number of finite element unknowns: 11'804'161

Ferrite timings are here

dof distribution time: 1.542 s
Total dofs: 9'010'003

^Sorry, copy pasted a wrong function above, which lead to wrong timings. Here is the updated one.

[KristofferC]

I would also push back a little bit about calling this "very slow" but yeah, I am sure the performance can be improved.

As it is right now the cost is basically completely in dictionary lookups in the part where we check if a dof has already been assigned to a certain entity.

[fredrikekre]

it is also very slow

It is a bit faster now: #643 (comment)

[termi-official]

Will take a look at eliminating the other two dicts.

To elaborate, there is still a chunk of time spend the hash computation for the face and edges which we should be able to eliminate by vectors (as for the vertices now).