plugins and standalones crash with `illegal hardware instruction` #61

dvzrv · 2019-12-22T19:50:06Z

After building 1.1.11 for Arch Linux I installed the package for testing purposes (on another machine than my build machine), as I introduced XDG desktop integration (if you're interested I'll open a pull request for it!).

Unfortunately all standalones crash with a segfault:

[1]    2577855 illegal hardware instruction (core dumped)  lsp-plugins-gate-mono

All plugins crash the hosts (possibly for the same reason).

My build machine's CPU has the following specs:

Architecture:                    x86_64
CPU op-mode(s):                  32-bit, 64-bit
Byte Order:                      Little Endian
Address sizes:                   39 bits physical, 48 bits virtual
CPU(s):                          8
On-line CPU(s) list:             0-7
Thread(s) per core:              2
Core(s) per socket:              4
Socket(s):                       1
NUMA node(s):                    1
Vendor ID:                       GenuineIntel
CPU family:                      6
Model:                           60
Model name:                      Intel(R) Core(TM) i7-4700MQ CPU @ 2.40GHz
Stepping:                        3
CPU MHz:                         2394.365
CPU max MHz:                     3400.0000
CPU min MHz:                     800.0000
BogoMIPS:                        4790.61
Virtualization:                  VT-x
L1d cache:                       128 KiB
L1i cache:                       128 KiB
L2 cache:                        1 MiB
L3 cache:                        6 MiB
NUMA node0 CPU(s):               0-7
Vulnerability Itlb multihit:     KVM: Mitigation: Split huge pages
Vulnerability L1tf:              Mitigation; PTE Inversion; VMX conditional cache flushes, SMT vulnerable
Vulnerability Mds:               Mitigation; Clear CPU buffers; SMT vulnerable
Vulnerability Meltdown:          Mitigation; PTI
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl and seccomp
Vulnerability Spectre v1:        Mitigation; usercopy/swapgs barriers and __user pointer sanitization
Vulnerability Spectre v2:        Mitigation; Full generic retpoline, IBPB conditional, IBRS_FW, STIBP conditional, RSB filling
Vulnerability Tsx async abort:   Not affected
Flags:                           fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc
                                  arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid ss
                                 e4_1 sse4_2 movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm cpuid_fault epb invpcid_single pti ssbd ibrs ibpb stibp tpr_shadow vnmi flexpri
                                 ority ept vpid ept_ad fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid xsaveopt dtherm ida arat pln pts md_clear flush_l1d

My test machine has the following CPU specs:

Architecture:                    x86_64
CPU op-mode(s):                  32-bit, 64-bit
Byte Order:                      Little Endian
Address sizes:                   36 bits physical, 48 bits virtual
CPU(s):                          4
On-line CPU(s) list:             0-3
Thread(s) per core:              2
Core(s) per socket:              2
Socket(s):                       1
NUMA node(s):                    1
Vendor ID:                       GenuineIntel
CPU family:                      6
Model:                           58
Model name:                      Intel(R) Core(TM) i7-3520M CPU @ 2.90GHz
Stepping:                        9
CPU MHz:                         1457.276
CPU max MHz:                     3600.0000
CPU min MHz:                     1200.0000
BogoMIPS:                        5788.48
Virtualization:                  VT-x
L1d cache:                       64 KiB
L1i cache:                       64 KiB
L2 cache:                        512 KiB
L3 cache:                        4 MiB
NUMA node0 CPU(s):               0-3
Vulnerability Itlb multihit:     KVM: Vulnerable
Vulnerability L1tf:              Mitigation; PTE Inversion
Vulnerability Mds:               Mitigation; Clear CPU buffers; SMT vulnerable
Vulnerability Meltdown:          Mitigation; PTI
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl and seccomp
Vulnerability Spectre v1:        Mitigation; usercopy/swapgs barriers and __user pointer sanitization
Vulnerability Spectre v2:        Mitigation; Full generic retpoline, IBPB conditional, IBRS_FW, STIBP conditional, RSB filling
Vulnerability Tsx async abort:   Not affected
Flags:                           fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_pe
                                 rfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid sse4_1 sse4_2 x
                                 2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm cpuid_fault epb pti ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase smep er
                                 ms xsaveopt dtherm ida arat pln pts md_clear flush_l1d

Note, the 2nd doesn't have avx2 support and I assume, that's why I'm seeing that problem.
Is there a way to not build with avx2 support? I think this would break for a lot of people, using this specific ISA and is nothing I can ship as a package this way.

The text was updated successfully, but these errors were encountered:

magnetophon · 2019-12-22T20:14:07Z

Same here, 1.1.11 gives me:
[1] + 8633 illegal hardware instruction (core dumped) ardour5.

CPU:

processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 42
model name	: Intel(R) Core(TM) i5-2520M CPU @ 2.50GHz
stepping	: 7
microcode	: 0x2f
cpu MHz		: 2990.254
cache size	: 3072 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 2
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 13
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx lahf_lm epb pti ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid xsaveopt dtherm ida arat pln pts md_clear flush_l1d
bugs		: cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swapgs itlb_multihit
bogomips	: 4983.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 36 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 42
model name	: Intel(R) Core(TM) i5-2520M CPU @ 2.50GHz
stepping	: 7
microcode	: 0x2f
cpu MHz		: 2990.245
cache size	: 3072 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 2
apicid		: 1
initial apicid	: 1
fpu		: yes
fpu_exception	: yes
cpuid level	: 13
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx lahf_lm epb pti ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid xsaveopt dtherm ida arat pln pts md_clear flush_l1d
bugs		: cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swapgs itlb_multihit
bogomips	: 4983.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 36 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 42
model name	: Intel(R) Core(TM) i5-2520M CPU @ 2.50GHz
stepping	: 7
microcode	: 0x2f
cpu MHz		: 2990.307
cache size	: 3072 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 2
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 13
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx lahf_lm epb pti ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid xsaveopt dtherm ida arat pln pts md_clear flush_l1d
bugs		: cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swapgs itlb_multihit
bogomips	: 4983.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 36 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 42
model name	: Intel(R) Core(TM) i5-2520M CPU @ 2.50GHz
stepping	: 7
microcode	: 0x2f
cpu MHz		: 2990.512
cache size	: 3072 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 2
apicid		: 3
initial apicid	: 3
fpu		: yes
fpu_exception	: yes
cpuid level	: 13
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx lahf_lm epb pti ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid xsaveopt dtherm ida arat pln pts md_clear flush_l1d
bugs		: cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swapgs itlb_multihit
bogomips	: 4983.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 36 bits physical, 48 bits virtual
power management:

sadko4u · 2019-12-22T20:50:55Z

Could you please run Ardour in GDB and when it crashes issue the command:

layout asm

And then take a screenshot of the debugger's screen?

sadko4u · 2019-12-22T20:56:42Z

We need to know what function and what instruction caused a fail. Then I'll do a patch.

sadko4u · 2019-12-22T20:59:10Z

Also I want to ask you to run unit tests and check what unit tests do fail on your systems.
How to run unit test:

make clean
make test
.test/lsp-plugins-test utest

It will execute all available unit tests and report which tests have failed and which are running properly.

dvzrv · 2019-12-22T21:59:03Z

I ran the unit tests during packaging (the package is not released btw!) and have a failing test, that I didn't recognize, as I made the tests not fail due to other failing tests earlier:

Testing sse::calc_normal3d_v2...
Unit test 'dsp.3d.normal' has failed at file utest/dsp/3d/normal.cpp, line 63 with message: 
  result of sse::calc_normal3d_v2 differs
sv[0]: { 1.792737e+00, 7.925274e+00, 8.185709e+00, 0.000000e+00 }
sv[1]: { -4.943272e+00, 3.777019e+00, 5.371266e+00, 0.000000e+00 }
n[0]: { 1.689408e-01, -7.263491e-01, 6.662403e-01, 1.000000e+00 }
n[1]: { 1.689407e-01, -7.263491e-01, 6.662402e-01, 0.000000e+00 }

sadko4u · 2019-12-22T22:01:30Z

This is not that test but interesting... I'll inspect it later.

sadko4u · 2019-12-22T22:02:29Z

You need to check unit tests on the machine that is affected by the 'illegal hardware instruction' error.

sadko4u · 2019-12-22T22:10:36Z

It seems I've found bad instruction. It's vbroadcastss which actually is AVX2 instruction when trying to broadcast value from XMM register to the YMM register.
I'll do a fix.

dvzrv · 2019-12-22T22:12:41Z

This is the unit test run on the test machine (without avx2):
lsp-plugins-1.1.11-check-no_avx2.log

These tests are failing:

  dsp.pmath.fmop4
  dsp.pmath.fmop3
  dsp.pmath.op_k2
  dsp.pmath.fmop_k3
  dsp.pmath.op_k3
  dsp.pmath.fmop_k4
  dsp.fft.fastconv
  dsp.fft.norm
  dsp.fft.pfft
  dsp.fft.fft
  core.util.convolver
  core.sampling.player

sadko4u · 2019-12-22T22:23:10Z

@dvzrv Thanks
I'll fix the problem ASAP

sadko4u · 2019-12-22T22:42:29Z

As a hotfix, you can replace this string:
https://github.com/sadko4u/lsp-plugins/blob/a6a8e9ca03429899e2514da9d56feaccc0a2ec7d/src/dsp/avx.cpp#L88

by:

        if ((f->features & (CPU_OPTION_AVX | CPU_OPTION_AVX2)) != (CPU_OPTION_AVX | CPU_OPTION_AVX2))

sadko4u · 2019-12-22T23:33:13Z

Please try to build and test from the github-issue-61 branch.

magnetophon · 2019-12-23T09:21:07Z

The github-issue-61 branch works perfectly.
Thanks!

sadko4u · 2019-12-23T09:30:51Z

Nice. I'll prepare for the 1.1.13 release then.

sadko4u · 2019-12-23T09:48:21Z

@dvzrv

as I introduced XDG desktop integration (if you're interested I'll open a pull request for it!)

Of course I'm interesed. But please provide a pull request to the devel branch, not master.

dvzrv · 2019-12-23T10:50:04Z

@sadko4u thanks! The branch in question seems to work for me (for the unit tests)

sadko4u · 2019-12-23T11:45:17Z

Please also check whether some random plugins do work at least as standalone versions.

dvzrv · 2019-12-23T12:20:57Z

I've built 1.1.11 with the changes from github-issue-61 applied on top:

Both random standalones, lv2 and vst plugins work

sadko4u · 2019-12-23T12:59:17Z

Good, thanks

sadko4u · 2019-12-23T14:02:13Z

Merged changes into devel branch: 1bccb10

sadko4u · 2019-12-23T15:43:40Z

Removed github-issue-61 branch.

sadko4u · 2019-12-23T19:41:25Z

Available in 1.1.13 release!

sadko4u added the discuss Discussing the issue label Dec 22, 2019

sadko4u added the bug Something isn't working label Dec 22, 2019

dvzrv mentioned this issue Dec 22, 2019

1.1.11 unit test dsp.3d.normal fails #62

Closed

sadko4u added this to the 1.1.13 milestone Dec 22, 2019

sadko4u added PENDING FOR RELEASE This issue will be added to the nearest upcoming release and removed discuss Discussing the issue labels Dec 23, 2019

sadko4u closed this as completed Dec 23, 2019

sadko4u removed the PENDING FOR RELEASE This issue will be added to the nearest upcoming release label Mar 20, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

plugins and standalones crash with `illegal hardware instruction` #61

plugins and standalones crash with `illegal hardware instruction` #61

dvzrv commented Dec 22, 2019

magnetophon commented Dec 22, 2019

sadko4u commented Dec 22, 2019 •

edited

Loading

sadko4u commented Dec 22, 2019

sadko4u commented Dec 22, 2019

dvzrv commented Dec 22, 2019

sadko4u commented Dec 22, 2019

sadko4u commented Dec 22, 2019

sadko4u commented Dec 22, 2019

dvzrv commented Dec 22, 2019 •

edited

Loading

sadko4u commented Dec 22, 2019

sadko4u commented Dec 22, 2019

sadko4u commented Dec 22, 2019

magnetophon commented Dec 23, 2019

sadko4u commented Dec 23, 2019 •

edited

Loading

sadko4u commented Dec 23, 2019

dvzrv commented Dec 23, 2019

sadko4u commented Dec 23, 2019

dvzrv commented Dec 23, 2019

sadko4u commented Dec 23, 2019

sadko4u commented Dec 23, 2019

sadko4u commented Dec 23, 2019

sadko4u commented Dec 23, 2019

plugins and standalones crash with illegal hardware instruction #61

plugins and standalones crash with illegal hardware instruction #61

Comments

dvzrv commented Dec 22, 2019

magnetophon commented Dec 22, 2019

sadko4u commented Dec 22, 2019 • edited Loading

sadko4u commented Dec 22, 2019

sadko4u commented Dec 22, 2019

dvzrv commented Dec 22, 2019

sadko4u commented Dec 22, 2019

sadko4u commented Dec 22, 2019

sadko4u commented Dec 22, 2019

dvzrv commented Dec 22, 2019 • edited Loading

sadko4u commented Dec 22, 2019

sadko4u commented Dec 22, 2019

sadko4u commented Dec 22, 2019

magnetophon commented Dec 23, 2019

sadko4u commented Dec 23, 2019 • edited Loading

sadko4u commented Dec 23, 2019

dvzrv commented Dec 23, 2019

sadko4u commented Dec 23, 2019

dvzrv commented Dec 23, 2019

sadko4u commented Dec 23, 2019

sadko4u commented Dec 23, 2019

sadko4u commented Dec 23, 2019

sadko4u commented Dec 23, 2019

plugins and standalones crash with `illegal hardware instruction` #61

plugins and standalones crash with `illegal hardware instruction` #61

sadko4u commented Dec 22, 2019 •

edited

Loading

dvzrv commented Dec 22, 2019 •

edited

Loading

sadko4u commented Dec 23, 2019 •

edited

Loading