Switchless mode and sgx-ssl conflict each other?

[ghost]

I am trying to build an app using switchless mode and sgx-ssl.

On my machines (Ubuntu 18.04, sgxsdk 2.5, sgxssl 2.4.1), standalone switchless example and sgx-ssl test-app (found here) works well, but, when trying to combine these two, I get an illigal intstruction error from libsgx_urts.so.

More specifically, after linking sgx-ssl (i.e. ligsgx_tsgxssl) and libsgx_tsetjmp to an enclave, attempt to initialize and enclave using sgx_create_enclave_ex() raises SIGILL.

p.s. I wanted to provide more detailed error report, but I currently don't know how to dump SE_TRACEs ;)

Where can I find documentation for debugging libsgx_urts.so?

[lzha101]

sgx-ssl has been integrated into libsgx_tcrypto.a. And libsgx_tsetjmp is not an isolated public library. It is part of libsgx_tstdc.a now. Please don't directly link it in your enclave.
If you want to build the switchless sample with sgx-ssl, perhaps you can try it in the following way:
1: Build SGX SDK by yourself with command "make USE_OPT_LIBS=0"
2: Install the built-out SDK
3: Build the switchless sample and run the sample.

You can follow the README.md to build a debug version SDK/PSW for debugging.

[ghost]

Ah, is standalone sgx-ssl no more used? Then what is the most idiomatic way to write enclave with openssl header funcitons?

More specifically, how can I port test-app to switchless version?

How can I deal with sgx-ssl specific header files? e.g. tSgxSSL_apis.h or pthread.h from /opt/intel/sgxssl/include

Thanks!

[akryeem-INTC]

is standalone sgx-ssl no more used?

Not exactly. SgxSSL still can be used as a standalone library, by linking your enclave to its libraries directly.
Could you please try working with the latest SgxSSL version i.e. lin_2.4_1.1.1a tag?

[akryeem-INTC]

Update:
The SIGILL is recieved at the point where SgxSSL tried to call sgx_cpuid(), status returned: SGX_ERROR_INVALID_FUNCTION.
BT:

#0  setup_cpuinfo (cpuinfo_table=0x0) at texceptions.c:133
#1  init_exception_handler (cpuinfo_table=0x0) at texceptions.c:191
#2  0x00007ffff5003415 in const_init_exception_handler ()
#3  0x00007ffff5007c52 in init_global_object ()
#4  0x00007ffff500608a in trts_ecall(unsigned int, void*) ()
#5  0x00007ffff5006409 in do_ecall ()
#6  0x00007ffff502f595 in enter_enclave ()
#7  0x00007ffff502f762 in enclave_entry ()
#8  0x00007ffff7bb85db in __morestack () from /usr/lib/x86_64-linux-gnu/libsgx_urts.so
#9  0x00007ffff7bbb700 in CEnclave::ecall(int, void const*, void*, bool) () from /usr/lib/x86_64-linux-gnu/libsgx_urts.so
#10 0x00007ffff7bbb882 in CEnclave::init_uswitchless(sgx_uswitchless_config_t const*) () from /usr/lib/x86_64-linux-gnu/libsgx_urts.so
#11 0x00007ffff7bbe4e5 in _create_enclave_from_buffer_ex(bool, unsigned char*, unsigned long, _se_file_t&, _le_prd_css_file_t*, unsigned long*, _sgx_misc_attribute_t*, unsigned int, void const**) () from /usr/lib/x86_64-linux-gnu/libsgx_urts.so
#12 0x00007ffff7bbe883 in _create_enclave_ex(bool, int, _se_file_t&, _le_prd_css_file_t*, unsigned char (*) [1024], int*, unsigned long*, _sgx_misc_attribute_t*, unsigned int, void const**) () from /usr/lib/x86_64-linux-gnu/libsgx_urts.so
#13 0x00007ffff7bbe9b9 in __sgx_create_enclave_ex.part.18 () from /usr/lib/x86_64-linux-gnu/libsgx_urts.so
#14 0x0000000000400ee7 in initialize_enclave (us_config=0x7fffffffe080) at App/App.cpp:170
#15 0x000000000040121a in main (argc=1, argv=0x7fffffffe1a8) at App/App.cpp:243

Issue is under investigations.

[ghost]

Thank you for looking further at the issue!

[ghost]

Update:
Simply linking lsgx_tsgxssl and lsgx_tsgxssl_crypto against SampleCode/Switchless results in similar error. In this case SIGILL is received at __morestack#EENTER_EPILOG.

This line was added to the Makefile:

- -Wl,--whole-archive -lsgx_switchless -l$(Trts_Library_Name) -Wl,-no-whold-arvhive
+ -Wl,--whole-archive -lsgx_tswitchless -l$(Trts_Library_Name) -Wl,--no-whole-archive L$(SGX_SSL_LIBRARY_PATH) \                                                                    
+ -Wl,--whole-archive -lsgx_tsgxssl -Wl,--no-whole-archive -lsgx_tsgxssl_crypto \

And here is the backtrace:

#0  0x00007ffff7b9adb8 in __morestack () at enter_enclave.S:133
#1  0x00007ffff7bacbfd in do_ecall (fn=-6, ocall_table=0x0, ms=0x555555774d00, 
    trust_thread=0x55555576be50) at sig_handler.cpp:240
#2  0x00007ffff7ba328c in CEnclave::ecall (this=0x55555576fc40, proc=-6, ocall_table=0x0, 
    ms=0x555555774d00, is_switchless=false)
    at /home/shpark/linux-sgx/psw/urts/enclave.cpp:361
#3  0x00007ffff7ba28f3 in CEnclave::init_uswitchless (this=0x55555576fc40, 
    config=0x7fffffffdc20) at /home/shpark/linux-sgx/psw/urts/enclave.cpp:99
#4  0x00007ffff7ba9bc9 in __create_enclave (parser=..., 
    base_addr=0x7ffff7f48000 "\177ELF\002\001\001", metadata=0x7ffff7f84b91, file=..., 
    debug=true, lc=0x555555770dc0, prd_css_file=0x0, enclave_id=0x555555757190 <global_eid>, 
    misc_attr=0x0, ex_features=2, ex_features_p=0x7fffffffdaf0)
    at /home/shpark/linux-sgx/psw/urts/urts_com.h:469
#5  0x00007ffff7baa16c in _create_enclave_from_buffer_ex (debug=true, 
    base_addr=0x7ffff7f48000 "\177ELF\002\001\001", file_size=564704, file=..., 
    prd_css_file=0x0, enclave_id=0x555555757190 <global_eid>, misc_attr=0x0, ex_features=2, 
    ex_features_p=0x7fffffffdaf0) at /home/shpark/linux-sgx/psw/urts/urts_com.h:592
#6  0x00007ffff7baa317 in _create_enclave_ex (debug=true, pfile=3, file=..., 
    prd_css_file=0x0, launch=0x0, launch_updated=0x0, 
    enclave_id=0x555555757190 <global_eid>, misc_attr=0x0, ex_features=2, 
    ex_features_p=0x7fffffffdaf0) at /home/shpark/linux-sgx/psw/urts/urts_com.h:632
#7  0x00007ffff7baa68a in __sgx_create_enclave_ex (
    file_name=0x55555555587a "enclave.signed.so", debug=1, launch_token=0x0, 
    launch_token_updated=0x0, enclave_id=0x555555757190 <global_eid>, misc_attr=0x0, 
    ex_features=2, ex_features_p=0x7fffffffdaf0) at urts.cpp:99
#8  0x00007ffff7baa7ba in sgx_create_enclave_ex (
    file_name=0x55555555587a "enclave.signed.so", debug=1, launch_token=0x0, 
    launch_token_updated=0x0, enclave_id=0x555555757190 <global_eid>, misc_attr=0x0, 
    ex_features=2, ex_features_p=0x7fffffffdaf0) at urts.cpp:134
#9  0x00005555555551ab in initialize_enclave (us_config=0x7fffffffdc20) at App/App.cpp:170
#10 0x0000555555555502 in main (argc=1, argv=0x7fffffffdd48) at App/App.cpp:243

[ghost]

I accidentally resolved this issue by re-compiling the whole SGX SDK v2.7 with USE_OPT_LIBS=0 option.

I still don't understand the reason why __morestack failed on ERESUME instruction. (0x00007ffff7b9adb8 was enclu and the leaf number was 3. 3 is ERESUME, isn't it?)