FHE Examples are failing - Illegal instruction (core dumped)

[vishnudevk]

Describe the bug
Illegal instruction (core dumped) error is coming when I am trying to make and run any included examples.
hePtr = HelibContext::create(HELIB_NOT_SECURE_CKKS_512_FAST); line is failing with that error.

To Reproduce
Steps to reproduce the behaviour:

1. Go to 'CKKS_credit_card_fraud dir'
2. Run cmake . && make
3. Run ./CKKS_credit_card_fraud
4. The following error is displayed in the terminal
   *** Starting inference demo ***
   Initializing HElib . . .
   Illegal instruction (core dumped)

Upon checking
hePtr = HelibContext::create(HELIB_NOT_SECURE_CKKS_512_FAST);
line is failing with that error.

Expected behavior
A clear and concise description of what you expected to happen.

Screenshots

Environment (please complete the following information):

• Host Info (The machine hosting the FHE Toolkit Docker container), please provide the output of running the following commands in a termina window:

  • uname -

  • [vishnudev@arch ~]$ uname -a
    Linux arch 5.11.14-arch1-1 hello! when will the toolkit be released?  #1 SMP PREEMPT Wed, 14 Apr 2021 12:06:34 +0000 x86_64 GNU/Linux

  • ulimit -a
    [vishnudev@arch ~]$ ulimit -a
    real-time non-blocking time (microseconds, -R) unlimited
    core file size (blocks, -c) unlimited
    data seg size (kbytes, -d) unlimited
    scheduling priority (-e) 0
    file size (blocks, -f) unlimited
    pending signals (-i) 31385
    max locked memory (kbytes, -l) 64
    max memory size (kbytes, -m) unlimited
    open files (-n) 1024
    pipe size (512 bytes, -p) 8
    POSIX message queues (bytes, -q) 819200
    real-time priority (-r) 0
    stack size (kbytes, -s) 8192
    cpu time (seconds, -t) unlimited
    max user processes (-u) 31385
    virtual memory (kbytes, -v) unlimited
    file locks (-x) unlimited

• Docker version (run docker -v in Terminal):

• Toolkit platform (Fedora, CentOS, Ubuntu):

Additional context
I ran ./RunToolkit.sh -p ubuntu command with sudo because it was failing with some permission issues when I ran without admin privilege.

[github-actions]

Thank you very much for helping us to improve the quality of our software. The FHE Toolkit develiopment team want to sincerely thank you for submitting your first issue and joining our technical community! We will get started on your issue immediately.

[dubek]

Thank you @vishnudevk for submitting this bug report. It looks like the way we compile HElib to create the fhe-toolkit-linux docker image generates an assembly instruction which is invalid on your machine.

Can you please post the output of lscpu on the host?

[vishnudevk]

`[vishnudev@arch ~]$ lscpu
Architecture:                    x86_64
CPU op-mode(s):                  32-bit, 64-bit
Byte Order:                      Little Endian
Address sizes:                   36 bits physical, 48 bits virtual
CPU(s):                          4
On-line CPU(s) list:             0-3
Thread(s) per core:              2
Core(s) per socket:              2
Socket(s):                       1
NUMA node(s):                    1
Vendor ID:                       GenuineIntel
CPU family:                      6
Model:                           42
Model name:                      Intel(R) Core(TM) i5-2450M CPU @ 2.50GHz
Stepping:                        7
CPU MHz:                         800.000
CPU max MHz:                     3100.0000
CPU min MHz:                     800.0000
BogoMIPS:                        4990.85
Virtualization:                  VT-x
L1d cache:                       64 KiB
L1i cache:                       64 KiB
L2 cache:                        512 KiB
L3 cache:                        3 MiB
NUMA node0 CPU(s):               0-3
Vulnerability Itlb multihit:     KVM: Mitigation: VMX disabled
Vulnerability L1tf:              Mitigation; PTE Inversion; VMX conditional cach
                                 e flushes, SMT vulnerable
Vulnerability Mds:               Vulnerable: Clear CPU buffers attempted, no mic
                                 rocode; SMT vulnerable
Vulnerability Meltdown:          Mitigation; PTI
Vulnerability Spec store bypass: Vulnerable
Vulnerability Spectre v1:        Mitigation; usercopy/swapgs barriers and __user
                                  pointer sanitization
Vulnerability Spectre v2:        Mitigation; Full generic retpoline, STIBP disab
                                 led, RSB filling
Vulnerability Srbds:             Not affected
Vulnerability Tsx async abort:   Not affected
Flags:                           fpu vme de pse tsc msr pae mce cx8 apic sep mtr
                                 r pge mca cmov pat pse36 clflush dts acpi mmx f
                                 xsr sse sse2 ht tm pbe syscall nx rdtscp lm con
                                 stant_tsc arch_perfmon pebs bts rep_good nopl x
                                 topology nonstop_tsc cpuid aperfmperf pni pclmu
                                 lqdq dtes64 monitor ds_cpl vmx est tm2 ssse3 cx
                                 16 xtpr pdcm pcid sse4_1 sse4_2 x2apic popcnt t
                                 sc_deadline_timer aes xsave avx lahf_lm epb pti
                                  tpr_shadow vnmi flexpriority ept vpid xsaveopt
                                  dtherm ida arat pln pts
[vishnudev@arch ~]$ 

`

[dubek]

OK, it seems that your CPU doesn't have avx2 and fma; and I believe we build the toolkit on a machine which does have them, and the build produces assembly which uses these features for improved performance (@boland25 - do you know?).

@vishnudevk I suggest you should try building the FHE toolkit images yourself; see instructions in https://github.com/IBM/fhe-toolkit-linux/blob/master/GettingStarted.Advanced.md .

[boland25]

@dubek that is probably the case, but I have to look into deeper on how its setup. Maybe we can meet internally about this next week after I've had some time to look at it?

[boland25]

also @vishnudevk curious if you tried to build anything yourself from your machine? and if so how did it go?

[vishnudevk]

I am getting the following error when tried to build the ubuntu based image myself. (sudo ./BuildDockerImage.sh ubuntu)

Step 37/42 : RUN adduser --uid ${USER_ID}  --gecos "FHE Toolkit User" --disabled-login fhe
 ---> Running in d397ac934d48
adduser: The UID 0 is already in use.
The command '/bin/sh -c adduser --uid ${USER_ID}  --gecos "FHE Toolkit User" --disabled-login fhe' returned a non-zero code: 1

[boland25]

ooh ok, I think this is related to our recent permissions fix. @deanthomasson do you have time to look into this at all?

[boland25]

@vishnudevk can you share with us what the user id was when you called this? You said that you used sudo but we are curious to see if it was using sudo's user id, or the current user's id. We might have to put a check in the script to look for this , thanks!

[boland25]

@vishnudevk can you try again without using sudo and see if the problem still persists, you shouldn't need to use sudo with our build script. we are able to re-produce it on our side

[vishnudevk]

My user id is 1000.

[vishnudev@arch fhe-toolkit-linux]$ id
uid=1000(vishnudev) gid=1000(vishnudev) groups=1000(vishnudev),998(wheel)

[vishnudev@arch fhe-toolkit-linux]$ sudo id
uid=0(root) gid=0(root) groups=0(root)

[vishnudevk]

@boland25 I could make it work by fixing the issue of the docker needing sudo to run.
I could build and run few examples too. Thank you very much on help me to get through.

[boland25]

@vishnudevk ah interesting, glad you were able to fix it! Thanks for pointing out this issue to us, we're going to add a change to the script, so your feed back is very helpful