Curl errors in samples/TEE_aware

[itssme]

I am trying to run the TEE_aware sample, but I am getting errors when running the program.

Steps to reproduce

1. clone the project
2. cd mystikos/samples/TEE_aware/gencreds
3. export MYSTIKOS_INSTALL_DIR=/opt/mystikos (installed version 0.11 via .deb package)
4. make appdir
5. make run

Generating a signing key
openssl genrsa -out package.pem -3 3072
Generating RSA private key, 3072 bit long modulus (2 primes)
...................................................................................................++++
..++++
e is 3 (0x03)
Building a ext2 file system to run in Mystikos
Dumping roothash merkel tree
Generating a signed package
Created myst/bin/gencreds

Running application outside a TEE.
appdir/bin/gencreds
****I am in unknown environment, returning
Running Mystikos packaged application inside an SGX TEE.
./myst/bin/gencreds
mystikos: info: enter.c(809): myst_enter_kernel(): Entered Mystikos kernel.
mystikos: warn: exec.c(1164): myst_exec(): 
    The thread stack size may be too small for the given program interpreter
    (link loader), which could result in stack overflows. Consider changing
    the thread stack size to at least 1048576 bytes, using the --thread-stack-size
    option or the ThreadStackSize configuration setting.
    [interpreter=/lib64/ld-linux-x86-64.so.2]
    [program=/bin/gencreds]

mystikos: info: exec.c(1259): myst_exec(): Entering CRT.
****I am in an SGX TEE, I will proceed to generate and verify TEE credentials
2023-09-19T14:34:51+0200.998417Z [(H)ERROR] tid(0x7f74b3bb0f40) | dcap_quoteprov: [ERROR]: Encountered CURL error 7 in curl_easy_perform
2023-09-19T14:34:51+0200.998476Z [(H)ERROR] tid(0x7f74b3bb0f40) | dcap_quoteprov: [ERROR]: error thrown, error code: 7: curl_easy_perform
2023-09-19T14:34:55+0200.070439Z [(H)ERROR] tid(0x7f74b3bb0f40) | dcap_quoteprov: [ERROR]: Encountered CURL error 7 in curl_easy_perform
2023-09-19T14:34:55+0200.070469Z [(H)ERROR] tid(0x7f74b3bb0f40) | dcap_quoteprov: [ERROR]: error thrown, error code: 7: curl_easy_perform
2023-09-19T14:34:58+0200.142432Z [(H)ERROR] tid(0x7f74b3bb0f40) | dcap_quoteprov: [ERROR]: Encountered CURL error 7 in curl_easy_perform
2023-09-19T14:34:58+0200.142462Z [(H)ERROR] tid(0x7f74b3bb0f40) | dcap_quoteprov: [ERROR]: error thrown, error code: 7: curl_easy_perform
2023-09-19T14:35:01+0200.214412Z [(H)ERROR] tid(0x7f74b3bb0f40) | dcap_quoteprov: [ERROR]: Encountered CURL error 7 in curl_easy_perform
2023-09-19T14:35:01+0200.214447Z [(H)ERROR] tid(0x7f74b3bb0f40) | dcap_quoteprov: [ERROR]: error thrown, error code: 7: curl_easy_perform
2023-09-19T14:35:04+0200.286422Z [(H)ERROR] tid(0x7f74b3bb0f40) | dcap_quoteprov: [ERROR]: Encountered CURL error 7 in curl_easy_perform
2023-09-19T14:35:04+0200.286456Z [(H)ERROR] tid(0x7f74b3bb0f40) | dcap_quoteprov: [ERROR]: error thrown, error code: 7: curl_easy_perform
2023-09-19T14:35:07+0200.358424Z [(H)ERROR] tid(0x7f74b3bb0f40) | dcap_quoteprov: [ERROR]: Encountered CURL error 7 in curl_easy_perform
2023-09-19T14:35:07+0200.358453Z [(H)ERROR] tid(0x7f74b3bb0f40) | dcap_quoteprov: [ERROR]: error thrown, error code: 7: curl_easy_perform
Generated a self-signed certificate and a private key
2023-09-19T14:35:09+0200.466812Z [(H)ERROR] tid(0x7f74b3bb0f40) | dcap_quoteprov: [ERROR]: HTTP error (404)
2023-09-19T14:35:09+0200.466831Z [(H)ERROR] tid(0x7f74b3bb0f40) | dcap_quoteprov: [ERROR]: Encountered CURL error 22 in curl_easy_perform
2023-09-19T14:35:09+0200.466890Z [(H)ERROR] tid(0x7f74b3bb0f40) | dcap_quoteprov: [ERROR]: curl error thrown, error code: 16: curl_easy_perform
2023-09-19T14:35:09+0200.601694Z [(H)ERROR] tid(0x7f74b3bb0f40) | dcap_quoteprov: [ERROR]: HTTP error (404)
2023-09-19T14:35:09+0200.601712Z [(H)ERROR] tid(0x7f74b3bb0f40) | dcap_quoteprov: [ERROR]: Encountered CURL error 22 in curl_easy_perform
2023-09-19T14:35:09+0200.601770Z [(H)ERROR] tid(0x7f74b3bb0f40) | dcap_quoteprov: [ERROR]: curl error thrown, error code: 16: curl_easy_perform
2023-09-19T14:35:09+0200.601779Z [(H)ERROR] tid(0x7f74b3bb0f40) | dcap_quoteprov: [ERROR]: Error fetching TCB Info: 57371
2023-09-19T14:35:09+0200.601788Z [(H)ERROR] tid(0x7f74b3bb0f40) | :OE_QUOTE_PROVIDER_CALL_ERROR [/__w/1/s/mystikos/third_party/openenclave/openenclave/host/sgx/sgxquoteprovider.c:oe_get_sgx_quote_verification_collateral:135]
2023-09-19T14:35:09+0200.601791Z [(H)ERROR] tid(0x7f74b3bb0f40) | :OE_QUOTE_PROVIDER_CALL_ERROR [/__w/1/s/mystikos/third_party/openenclave/openenclave/host/sgx/ocalls/ocalls.c:oe_get_quote_verification_collateral_with_baseline_ocall:241]
Assertion failed: ret == 0 (gencreds.c: main: 54)
/home/robo/Downloads/mystikos/samples/TEE_aware/gencreds/myst/bin/gencreds: error: Enclave /tmp/mystvVs6rM/lib/openenclave/mystenc.so returned 134

make: *** [Makefile:37: run] Error 134

Am I missing something? As far as I understand, the sample should create a self signed certificate in an enclave and then verify that certificate? But what is curl trying to do, query some certificates/ or revocations lists etc.?

Any help is much appreciated 👍

[vtikoo]

Can you share some details on where you are running this? Is this an Azure VM?

[radhikaj]

AS @vtikoo indicates - we test on Azure VMs currently. You can also try to set up Intel QPL per the instructions here on non Azure VMs https://github.com/openenclave/openenclave/pull/4773/files, but we have not validates these instructions at this time with Mystikos

[itssme]

Thanks for the quick responses!

Can you share some details on where you are running this? Is this an Azure VM?

I am currently running this from a fresh Ubuntu 20.04 installation on my local machine (I setup a bootable USB M.2. SSD as my dev environment for this project). I followed the openenclave install guide and the mystikos install guide (.deb v0.11.0).

I am currently trying out different confidential computing frameworks that support SGX. For example, asylo, openenclave, Ego etc. Eventually my goal is to develop a small application that hosts an HTTPs server that I can send data to (for example a part of a pointcloud) and then processes that data securely in an enclave. So that the data cannot be seen by the cloud provider nor anyone who listens in transit. And I also intend to use remote attestation, to verify that the cloud provider is running the exact code I provided etc.

AS @vtikoo indicates - we test on Azure VMs currently. You can also try to set up Intel QPL per the instructions here on non Azure VMs https://github.com/openenclave/openenclave/pull/4773/files, but we have not validates these instructions at this time with Mystikos

Thank you for the link to the guide to setup Intel QPL, I will definitely take a look at that 👍

I also looked at guides like:
https://sgx101.gitbook.io/sgx101/sgx-bootstrap/attestation
and read a bit about remote attestation etc.

If you have any more links and resources that could help me, I would very much appreciate it 👍

[itssme]

You can also try to set up Intel QPL per the instructions here on non Azure VMs https://github.com/openenclave/openenclave/pull/4773/files, but we have not validates these instructions at this time with Mystikos

I followed the instruction of @radhikaj and installed the PCCS service. When running the TEE_aware example the curl errors are gone but the self signed certificate generated in the enclave cannot be validated.

Generating a signing key
openssl genrsa -out package.pem -3 3072
Generating RSA private key, 3072 bit long modulus (2 primes)
.....................................................................................++++*.....................................++++*.......................................................++++
..................................................................................................................++++*..........................................................................................................................................................++++*........................................................................................................................................................++++*..............................................................................................................................................................................................................................................................................++++*...........................................................................++++
e is 3 (0x03)
Building a ext2 file system to run in Mystikos
Dumping roothash merkel tree
Generating a signed package
Created myst/bin/gencreds

Running application outside a TEE.
appdir/bin/gencreds
****I am in unknown environment, returning
Running Mystikos packaged application inside an SGX TEE.
./myst/bin/gencreds
mystikos: info: enter.c(809): myst_enter_kernel(): Entered Mystikos kernel.
mystikos: warn: exec.c(1164): myst_exec(): 
    The thread stack size may be too small for the given program interpreter
    (link loader), which could result in stack overflows. Consider changing
    the thread stack size to at least 1048576 bytes, using the --thread-stack-size
    option or the ThreadStackSize configuration setting.
    [interpreter=/lib64/ld-linux-x86-64.so.2]
    [program=/bin/gencreds]

mystikos: info: exec.c(1259): myst_exec(): Entering CRT.
****I am in an SGX TEE, I will proceed to generate and verify TEE credentials
Generated a self-signed certificate and a private key
Assertion failed: ret == 0 (gencreds.c: main: 54)
/home/robo/Downloads/mystikos/samples/TEE_aware/gencreds/myst/bin/gencreds: error: Enclave /tmp/mystgWG3zR/lib/openenclave/mystenc.so returned 134

make: *** [Makefile:37: run] Error 134

ret = syscall(SYS_myst_verify_cert, cert, cert_size, _verifier, NULL);
assert(ret == 0);

In the process of installing the sgx-dcap-pccs package, a self signed certificate is generated. Do I need to change any configuration of mystikos so that the certificate is marked as trusted?