Tabby not starting, no logs are displayed

[Daandeve]

Describe the bug
When I try to start tabby it instantly cashes, command: serve --model TabbyML/StarCoder-3B and serve --model TabbyML/StarCoder-3B --device cuda. It exits with code 132 and no logs are displayed, only:
INFO tabby::serve: crates/tabby/src/serve.rs:114: Starting server, this might take a few minutes...

Information about your version
Tried 0.7.0 and the nightly build of 27-12-2023

Information about your GPU

Additional information

OS: Unraid 6.12.4
CPU: 2x Xeon E5-2670

[wsxiaoys]

Hi - Given that your GPU has 2G vram, could you try a smaller model, e.g TabbyML/StarCoder-1B?

[Daandeve]

This sadly produces the same result :(

[wsxiaoys]

could you also share the content of /proc/cpuinfo on your system?

[Daandeve]

Output of /proc/cpuinfo

processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 45
model name	: Intel(R) Xeon(R) CPU E5-2670 0 @ 2.60GHz
stepping	: 7
microcode	: 0x71a
cpu MHz		: 3295.689
cache size	: 20480 KB
physical id	: 0
siblings	: 16
core id		: 0
cpu cores	: 8
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 13
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx lahf_lm epb pti ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid xsaveopt dtherm ida arat pln pts md_clear flush_l1d
vmx flags	: vnmi preemption_timer invvpid ept_x_only ept_1gb flexpriority tsc_offset vtpr mtf vapic ept vpid unrestricted_guest ple
bugs		: cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swapgs itlb_multihit mmio_unknown
bogomips	: 5200.04
clflush size	: 64
cache_alignment	: 64
address sizes	: 46 bits physical, 48 bits virtual
power management:

[wsxiaoys]

The cause should be that E5-2670 doesn't support avx2

Filing #1142 to track