-
Notifications
You must be signed in to change notification settings - Fork 228
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
optimize the CPML code (which is currently very slow) and use weights in Scotch decomposition for C-PML elements in the code #18
Comments
already done by Zhinan Xie @xiezhinan I think (?). Dimitri, June 2015: no, not done yet. |
Fixing this would be easy (measuring the CPU time spent in each kind of elements with a CPU timer, and then using that as a weight when calling SCOTCH); but it is not done yet I think, unless Zhinan has done it? |
James Smith @Jas11 and Dmitry Borisov @dborissov are currently doing it. |
See also the comment at the end of #625 |
From Zhinan Xie @xiezhinan : Computational cost with or without memory variables in an elastic element and PML element: We have counted the computational work and memory variable in elastic element and PML element in both specfem2d and specfem3d. 1.SPECFEM2D Elastic element Computational amount involved (NGLLX = 5):1450 Memory variable :0 Elastic PMLelement Computational amount involved (NGLLX = 5):3775 + 1450=5225 Memory variable:8NxNz+4NxNz=300 Viscoelastic element Computational amount involved (NGLLX = 5):1650+3N_SLS25 Memory variable:3*25 Viscoelastic PML element Computational amount involved (NGLLX = 5):5425 +2875*N_SLS Memory variable:(12+6*N_SLS) 2.SPECFEM3D Elastic element Computational amount involved (NGLLX = 5):74NxNyNz+24 NxNxNy*Nz Memory variable: 0 Elastic PML element Computational amount involved (NGLLX = 5):(557+48Nx)NxNy*Nz Memory variable:39NxNy*Nz Viscoelastic element Computational amount involved(NGLLX = 5): 111NxNyNz+24NxNyNzNx+60N_SLSNxNyNz Memory variable:6NxNyNzN_SLS Viscoelastic PML element Computational amount involved(NGLLX = 5): 522NxNyNz+48NxNxNxNy+444N_SLSNxNx*Ny Memory variable:29NxNxNy+9N_SLSNxNxNy Best regards, Zhinan |
Note also that the PML source code itself could probably be optimized a lot (reducing the size of the arrays used, avoiding "if" statements in loops, precomputing things, factorizing things, vectorizing loops etc). This should of course be done before computing and assigning some PML weights to account for the load as explained above, since doing such optimizations will reduce the load and thus change the relative weights. |
use weights in Scotch decomposition for C-PML elements in the code;
because C-PML elements will be more expensive because they will compute more terms and will solve more equations (for the memory
variables / convolution terms) thus we should assign a higher weight to them when we call Scotch
the "elmnts_load" array up in decompose_mesh/part_decompose_mesh.f90 (around line 1432)
currently does not take into account C-PML elements weights. The matching array in the code is
called "CPML_type" and is defined as follows: 1 = face, 2 = edge, 3 = corner.
that is similar to what Daniel does for acoustic elements in the current version of SPECFEM3D: he uses a weight of 1 for acoustic
elements and 3 (or something like that) for elastic elements, which use a 3D vector instead of a scalar.
In principle the weighting factors can be computed analytically by counting the exact number of additional multiplications and
additions performed by C-PML elements
In PML corners this additional factor can be multiplied by 2 (for instance in the slice of PML that has X and Y damping) or by 3
for corners that combine an X, a Y and a Z PML.
easy to do once we have an expression for the weighting factors
The text was updated successfully, but these errors were encountered: