SIMD on CI runners: occurrence of "illegal" instructions

[FabienPean-Virtonomy]

The PR #176 unfortunately did not solve the "Illegal instructions" error popping up at random on the CI for Linux system

See the logs below for anyone wanting to dive into this problem:

logs_473.zip
logs_458.zip
logs_431.zip
without ccache logs_458.zip

The following tests FAILED for logs 473:
49 - bernoulli_beam_struct_sim (ILLEGAL)
50 - mass_spring_damper_response_struct_sim (ILLEGAL)
52 - test_3d_beam_pulling_pressure_load (ILLEGAL)
55 - test_3d_heart_electromechanics (Failed)
61 - test_3d_particle_relaxation (ILLEGAL)
62 - test_3d_particle_relaxation_single_resolution (ILLEGAL)
65 - test_3d_pkj_lv_electrocontraction (Failed)
67 - test_3d_self_contact (Failed)
68 - test_3d_shell_particle_relaxation (ILLEGAL)
69 - test_3d_taylor_bar (Failed)

The following tests FAILED for logs 458:
49 - bernoulli_beam_struct_sim (ILLEGAL)
50 - mass_spring_damper_response_struct_sim (ILLEGAL)
52 - test_3d_beam_pulling_pressure_load (ILLEGAL)
55 - test_3d_heart_electromechanics (Failed)
61 - test_3d_particle_relaxation (ILLEGAL)
62 - test_3d_particle_relaxation_single_resolution (ILLEGAL)
65 - test_3d_pkj_lv_electrocontraction (Failed)
67 - test_3d_self_contact (Failed)
68 - test_3d_shell_particle_relaxation (ILLEGAL)
69 - test_3d_taylor_bar (Failed)

The following tests FAILED for logs 431:
42 - test_scalar_functions_particle_relaxation (ILLEGAL)
49 - bernoulli_beam_struct_sim (ILLEGAL)
50 - mass_spring_damper_response_struct_sim (ILLEGAL)
52 - test_3d_beam_pulling_pressure_load (ILLEGAL)
55 - test_3d_heart_electromechanics (Failed)
61 - test_3d_particle_relaxation (ILLEGAL)
62 - test_3d_particle_relaxation_single_resolution (ILLEGAL)
65 - test_3d_pkj_lv_electrocontraction (Failed)
67 - test_3d_self_contact (Failed)
68 - test_3d_shell_particle_relaxation (ILLEGAL)
69 - test_3d_taylor_bar (Failed)

The following tests FAILED for logs 458:
49 - bernoulli_beam_struct_sim (ILLEGAL)
50 - mass_spring_damper_response_struct_sim (ILLEGAL)
52 - test_3d_beam_pulling_pressure_load (ILLEGAL)
56 - test_3d_heart_electromechanics_particle_relaxation (ILLEGAL)
57 - test_3d_heart_electromechanics (ILLEGAL)
63 - test_3d_particle_relaxation (ILLEGAL)
64 - test_3d_particle_relaxation_single_resolution (ILLEGAL)
67 - test_3d_pkj_lv_electrocontraction_particle_relaxation (ILLEGAL)
68 - test_3d_pkj_lv_electrocontraction (ILLEGAL)
70 - test_3d_self_contact_particle_relaxation (ILLEGAL)
71 - test_3d_self_contact (ILLEGAL)
72 - test_3d_shell_particle_relaxation (ILLEGAL)
73 - test_3d_taylor_bar_particle_relaxation (ILLEGAL)
74 - test_3d_taylor_bar (ILLEGAL)

There seems to be a pattern in the tests that fails, but what do they have in common ?

[Xiangyu-Hu]

Does this only happen in linux system or in all systems?

[FabienPean-Virtonomy]

On Linux only

[Xiangyu-Hu]

if it is the case, we may first simply deactivate ccache in linux. I have seen the same complain in Ccache github issue last year already, However, it seems that the developer is not interesting find a solution yet,
ccache/ccache#824

[FabienPean-Virtonomy]

As mentioned in a previous discussion, this problem occurred already without ccache, https://github.com/Virtonomy/SPHinXsys/actions/runs/3592043737/jobs/6047298762. Besides, the flags are laid out in the build command which avoids the problem you mention.

[Xiangyu-Hu]

Ok. I remember that never happened before the revamping of the CMAKE. Or, it is a issue after eigen introduced?

[FabienPean-Virtonomy]

Eigen and CMake revamp came one after another. They are not enough samples of CI runs between Eigen and the CMake revamp to rule out an Eigen cause. What we can infer for now is that it is a SIMD vectorization issue problem and it seems to affect the same examples.

[Xiangyu-Hu]

Ok. These samples are 3d solid dynamics which they have higher chance to use simd operations.