SPECFEM2D crashes with tomographic model input

[pasansherath]

I have the need to include both vertical and horizontal velocity and density gradients into the layers in my models.

I tried using using a tomo.xyz file, with two layers with vertical velocity gradients. I was able to get the simulation running, but the simulation crashed after 1200 time steps.

******************************************************************
Time step number    1200   t =    0.2398 s out of  120000
 ******************************************************************
 We have done   0.999175012     % of the total

Program received signal SIGFPE: Floating-point exception - erroneous arithmetic operation.

Backtrace for this error:
#0  0x7f0805fe7cd1 in ???
#1  0x7f0805fe6ea5 in ???
#2  0x7f0805cb220f in ???
#3  0x558c98a6c451 in check_stability_
    at src/specfem2D/check_stability.F90:95
#4  0x558c98ade3f6 in iterate_time_
    at src/specfem2D/iterate_time.F90:142
#5  0x558c98a4b7e6 in specfem2d
    at src/specfem2D/specfem2D.F90:381
#6  0x558c98a4b7e6 in main
    at src/specfem2D/specfem2D.F90:349
--------------------------------------------------------------------------
Primary job  terminated normally, but 1 process returned
a non-zero exit code. Per user-direction, the job has been aborted.
--------------------------------------------------------------------------
--------------------------------------------------------------------------
mpiexec noticed that process rank 0 with PID 0 on node thinkpad exited on signal 8 (Floating point exception). 

After that I did the following, but the simulation crashed. I had the CFL stability condition <<< 0.5 and also DT < maximum recommended DT in all the following instances.

• I removed the velocity gradients, and the simulation crashed the same number of time steps.
  

• I increased the number of spectral elements in both x and z directions. It crashed at the same number of time steps (1200)
  

• I removed a layer and its velocity gradient (uniform velocity), still crashed at 1200 time steps.
  

• I increased the resolution in the tomo.xyz file by a factor of 2 along each axis. It still crashes at 1200 time steps.
  

• I reduced the DT  by half, and the simulation crashed after 2300 time steps.
  

I tried running this on two different clusters with 36 processors (using mpi) and on my laptop with 4 processors. The result is the same.

I have uploaded my Par_file, interfaces, SOURCE and tomo files.

Can someone let me what the problem here is? Is there any other way I could incorporate velocity gradients in the velocity model?

Thanks very much.

interfaces.txt
Par_file.txt
SOURCE.txt
tomo.txt

[bottero]

Hi Pasan, In SOURCE file changing : factor = 1.d30 # amplification factor to (for example) : factor = 1.d5 # amplification factor solves the issue. 1.d30 was way too big, it generated huge amplitudes and the code thought it had exploded. Hope this helps, Alexis Bottero Marine Physical Laboratory Scripps Institution of Oceanography (+1) 619 677 9887 (+33) 695 170 097 Le mer. 29 avr. 2020 à 16:13, Pasan Herath <notifications@github.com> a écrit :

…

I have the need to include both vertical and horizontal velocity and density gradients into the layers in my models. I tried using using a tomo.xyz file, with two layers with vertical velocity gradients. I was able to get the simulation running, but the simulation crashed after 1200 time steps. _****************************************************************** Time step number 1200 t = 0.2398 s out of 120000 ------------------------------ We have done 0.999175012 % of the total Program received signal SIGFPE: Floating-point exception - erroneous arithmetic operation. Backtrace for this error: #0 0x7f0805fe7cd1 in ??? #1 <#1> 0x7f0805fe6ea5 in ??? #2 <#2> 0x7f0805cb220f in ??? #3 <#3> 0x558c98a6c451 in check_stability_ at src/specfem2D/check_stability.F90:95 #4 <#4> 0x558c98ade3f6 in iterate_time_ at src/specfem2D/iterate_time.F90:142 #5 <#5> 0x558c98a4b7e6 in specfem2d at src/specfem2D/specfem2D.F90:381 #6 <#6> 0x558c98a4b7e6 in main at src/specfem2D/specfem2D.F90:349 Primary job terminated normally, but 1 process returned a non-zero exit code. Per user-direction, the job has been aborted. ------------------------------ mpiexec noticed that process rank 0 with PID 0 on node thinkpad exited on signal 8 (Floating point exception)._ After that I did the following, but the simulation crashed. I had the CFL stability condition <<< 0.5 and also DT < maximum recommended DT in all the following instances. - I removed the velocity gradients, and the simulation crashed the same number of time steps. - I increased the number of spectral elements in both x and z directions. It crashed at the same number of time steps (1200) - I removed a layer and its velocity gradient (uniform velocity), still crashed at 1200 time steps. - I increased the resolution in the tomo.xyz file by a factor of 2 along each axis. It still crashes at 1200 time steps. - I reduced the DT by half, and the simulation crashed after 2300 time steps. I tried running this on two different clusters with 36 processors (using mpi) and on my laptop with 4 processors. The result is the same. I have uploaded my Par_file, interfaces, SOURCE and tomo files. Can someone let me what the problem here is? Is there any other way I could incorporate velocity gradients in the velocity model? Thanks very much. interfaces.txt <https://github.com/geodynamics/specfem2d/files/4555272/interfaces.txt> Par_file.txt <https://github.com/geodynamics/specfem2d/files/4555273/Par_file.txt> SOURCE.txt <https://github.com/geodynamics/specfem2d/files/4555274/SOURCE.txt> tomo.txt <https://github.com/geodynamics/specfem2d/files/4555279/tomo.txt> — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#1061>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABVY5YS72ACDCE3SF6WH34LRPCYAFANCNFSM4MUMTGTQ> .

[Raylhu004]

I used to encounter this problem. The problem is mainly caused by a large time step even though the CFL has been fulfilled. Please reduce the time step.

[bottero]

My answer solved this issue. This can be closed