Creating enclave failed when running ServerEnclave

[Rf-xi]

Hi, I have a problem when running ServerEnclave. I want to run NARRATOR with the following command:

~/Narrator$ ./ServerEnclave/build/host/attestation_host ./ServerEnclave/build/enclave/enclave_a.signed 8998 127.0.0.1
SeverEnclave Start time 1688280735035768
[+] Enclave1: ***/home/xrf/Narrator/ServerEnclave/common/crypto.cpp(112): OpenSsl RSA step init Successful!
[+] Enclave1: ***/home/xrf/Narrator/ServerEnclave/common/crypto.cpp(119): AES Key is D65EC97B4DC8A64718FCA734A355C80B
[+] Enclave1: ***/home/xrf/Narrator/ServerEnclave/common/crypto.cpp(126): OpenSsl AES step init Successful!
2023-07-02T06:52:15+0000.555723Z [(H)ERROR] tid(0x7ff66ff87100) | Backtrace:
2023-07-02T06:52:15+0000.563856Z [(H)ERROR] tid(0x7ff66ff87100) | _ZN6Crypto12init_opensslEv(): 0x7ff668055a68
2023-07-02T06:52:15+0000.563865Z [(H)ERROR] tid(0x7ff66ff87100) | _ZN6CryptoC1Ev(): 0x7ff6680541cd
2023-07-02T06:52:15+0000.563867Z [(H)ERROR] tid(0x7ff66ff87100) | _ZN16ecall_dispatcher10initializeEPKc(): 0x7ff66805c6df
2023-07-02T06:52:15+0000.563869Z [(H)ERROR] tid(0x7ff66ff87100) | _ZN16ecall_dispatcherC1EPKcP20_enclave_config_data(): 0x7ff66805c127
2023-07-02T06:52:15+0000.563871Z [(H)ERROR] tid(0x7ff66ff87100) | __cxx_global_var_init(): 0x7ff66805203b
2023-07-02T06:52:15+0000.563874Z [(H)ERROR] tid(0x7ff66ff87100) | _GLOBAL__sub_I_ecalls.cpp(): 0x7ff6680520b9
2023-07-02T06:52:15+0000.563877Z [(H)ERROR] tid(0x7ff66ff87100) | oe_call_init_functions(): 0x7ff6683e71ff
2023-07-02T06:52:15+0000.563880Z [(H)ERROR] tid(0x7ff66ff87100) | _handle_ecall(): 0x7ff6683dd7e9
2023-07-02T06:52:15+0000.563882Z [(H)ERROR] tid(0x7ff66ff87100) | oe_enter(): 0x7ff6683ddffe
2023-07-02T06:52:15+0000.563922Z [(H)ERROR] tid(0x7ff66ff87100) | Backtrace:
2023-07-02T06:52:15+0000.566933Z [(H)ERROR] tid(0x7ff66ff87100) | oe_abort_with_td(): 0x7ff6683dcccf
2023-07-02T06:52:15+0000.566940Z [(H)ERROR] tid(0x7ff66ff87100) | oe_abort(): 0x7ff6683dbb32
2023-07-02T06:52:15+0000.566942Z [(H)ERROR] tid(0x7ff66ff87100) | oe_real_exception_dispatcher(): 0x7ff6683dec08
2023-07-02T06:52:15+0000.566944Z [(H)ERROR] tid(0x7ff66ff87100) | _ZN6Crypto12init_opensslEv(): 0x7ff668055a68
2023-07-02T06:52:15+0000.566946Z [(H)ERROR] tid(0x7ff66ff87100) | _ZN6CryptoC1Ev(): 0x7ff6680541cd
2023-07-02T06:52:15+0000.566949Z [(H)ERROR] tid(0x7ff66ff87100) | _ZN16ecall_dispatcher10initializeEPKc(): 0x7ff66805c6df
2023-07-02T06:52:15+0000.566951Z [(H)ERROR] tid(0x7ff66ff87100) | _ZN16ecall_dispatcherC1EPKcP20_enclave_config_data(): 0x7ff66805c127
2023-07-02T06:52:15+0000.566953Z [(H)ERROR] tid(0x7ff66ff87100) | __cxx_global_var_init(): 0x7ff66805203b
2023-07-02T06:52:15+0000.566959Z [(H)ERROR] tid(0x7ff66ff87100) | _GLOBAL__sub_I_ecalls.cpp(): 0x7ff6680520b9
2023-07-02T06:52:15+0000.566961Z [(H)ERROR] tid(0x7ff66ff87100) | oe_call_init_functions(): 0x7ff6683e71ff
2023-07-02T06:52:15+0000.566963Z [(H)ERROR] tid(0x7ff66ff87100) | _handle_ecall(): 0x7ff6683dd7e9
2023-07-02T06:52:15+0000.566966Z [(H)ERROR] tid(0x7ff66ff87100) | oe_enter(): 0x7ff6683ddffe
2023-07-02T06:52:15+0000.566984Z [(H)ERROR] tid(0x7ff66ff87100) | :OE_ENCLAVE_ABORTING [/source/openenclave/host/sgx/create.c:_initialize_enclave:571]
2023-07-02T06:52:15+0000.566988Z [(H)ERROR] tid(0x7ff66ff87100) | :OE_ENCLAVE_ABORTING [/source/openenclave/host/sgx/create.c:oe_create_enclave:1393]
Error: Creating enclave failed. OE_ENCLAVE_ABORTING[+] Set configuration from ../host/_configuration
file_path../host/_configuration
[+] Local IP address is: 10.**.**.**
[+] Adding peers from 127.0.0.1
[+]Here is Peers:
[+]Here is Clients:0
[+] Adding IPs for connecting peers from ../host/network/_peer_ip_allowed
^C

It seems that create enclave failed. . I suspect that inconsistent PCCS service addresses are causing this error. But I'm not familiar with the sgx configuration, so can you help me ?

~/Narrator$ dmesg | grep -i sgx
[   14.613812] intel_sgx: loading out-of-tree module taints kernel.
[   14.614563] intel_sgx: EPC section 0x4000c00000-0x407f7fffff
[   14.628680] intel_sgx: EPC section 0x8000c00000-0x807fffffff
[   14.661896] intel_sgx: Intel SGX DCAP Driver v1.41

~/Narrator$ curl --noproxy "*" -v -k -G "https://127.0.0.1:8081/sgx/certification/v2/rootcacrl"
*   Trying 127.0.0.1:8081...
* TCP_NODELAY set
* Connected to 127.0.0.1 (127.0.0.1) port 8081 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN, server accepted to use http/1.1
* Server certificate:
*  subject: C=CN; ST=FU; L=XM; O=XMU; emailAddress=xrfgooo@gmail.com
*  start date: Jul  2 06:45:01 2023 GMT
*  expire date: Jul  1 06:45:01 2024 GMT
*  issuer: C=CN; ST=FU; L=XM; O=XMU; emailAddress=xrfgooo@gmail.com
*  SSL certificate verify result: self signed certificate (18), continuing anyway.
> GET /sgx/certification/v2/rootcacrl HTTP/1.1
> Host: 127.0.0.1:8081
> User-Agent: curl/7.68.0
> Accept: */*
> 
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* old SSL session ID is stale, removing
* Mark bundle as not supporting multiuse
< HTTP/1.1 404 Not Found
< X-Powered-By: Express
< Request-ID: 7e3fc9b02d334137a4e5b17953c56e3b
< Content-Security-Policy: default-src 'none'
< X-Content-Type-Options: nosniff
< Content-Type: text/html; charset=utf-8
< Content-Length: 169
< Date: Sun, 02 Jul 2023 07:02:36 GMT
< Connection: keep-alive
< Keep-Alive: timeout=5
< 
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<title>Error</title>
</head>
<body>
<pre>Cannot GET /sgx/certification/v2/rootcacrl</pre>
</body>
</html>
* Connection #0 to host 127.0.0.1 left intact

I tried another PCCS address and it seems to be working. “https: //127.0.0.1:8081/sgx/certification/v4/rootcacrl"

~/Narrator$ curl --noproxy "*" -v -k -G "https://127.0.0.1:8081/sgx/certification/v4/rootcacrl"
*   Trying 127.0.0.1:8081...
* TCP_NODELAY set
* Connected to 127.0.0.1 (127.0.0.1) port 8081 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN, server accepted to use http/1.1
* Server certificate:
*  subject: C=CN; ST=FJ; L=XM; O=XMU; emailAddress=xrfgooo@gmail.com
*  start date: Jul  2 06:45:01 2023 GMT
*  expire date: Jul  1 06:45:01 2024 GMT
*  issuer: C=CN; ST=FU; L=XM; O=XMU; emailAddress=xrfgooo@gmail.com
*  SSL certificate verify result: self signed certificate (18), continuing anyway.
> GET /sgx/certification/v4/rootcacrl HTTP/1.1
> Host: 127.0.0.1:8081
> User-Agent: curl/7.68.0
> Accept: */*
> 
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* old SSL session ID is stale, removing
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< X-Powered-By: Express
< Request-ID: c803b15ae27a42f7b16ed65124ff5d7d
< Content-Type: application/pkix-crl; charset=utf-8
< Content-Length: 586
< ETag: W/"24a-/NnkEyrz7GitRu9J3E31+ENl4wQ"
< Date: Sun, 02 Jul 2023 07:04:09 GMT
< Connection: keep-alive
< Keep-Alive: timeout=5
< 
* Connection #0 to host 127.0.0.1 left intact
308201213081c8020101300a06082a8648ce3d0403023068311a301806035504030c11496e74656c2053475820526f6f74204341311a3018060355040a0c11496e74656c20436f72706f726174696f6e3114301206035504070c0b53616e746120436c617261310b300906035504080c024341310b3009060355040613025553170d3233303430333130323235315a170d3234303430323130323235315aa02f302d300a0603551d140403020101301f0603551d2304183016801422650cd65a9d3489f383b49552bf501b392706ac300a06082a8648ce3d0403020348003045022051577d47d9fba157b65f1eb5f4657bbc5e56ccaf735a03f1b963d704805ab118022100939015ec1636e7eafa5f426c1e402647c673132b6850cabd68cef6bad7682a03

I have tried to reinstall the PCCS but I didn't find any config to change its address. So, Is this the problem, and how do I fix it?

[pw0rld]

It appears that there is no issue with the attestation step. Can you confirm if your openenclave sample is running successfully? It is possible that there might be a problem with your SDK environment.

[Rf-xi]

Hello, thanks for your reply.
You are right, I find a problem when running the attestation sample:

~/mysamples/attestation/build$ make run
[ 19%] Built target attestation_host
[ 23%] Built target public_key_a
[ 47%] Built target common
[ 61%] Built target enclave_b
[ 71%] Built target enclave_b_signed
[ 76%] Built target public_key_b
[ 90%] Built target enclave_a
[100%] Built target enclave_a_signed
[100%] Built target sign
Scanning dependencies of target runsgxremote
Host: Creating two enclaves
Host: Enclave library /home/xrf/mysamples/attestation/build/enclave_a/enclave_a.signed
Enclave1: ***/home/xrf/mysamples/attestation/common/crypto.cpp(80): mbedtls initialized.
Host: Enclave successfully created.
Host: Enclave library /home/xrf/mysamples/attestation/build/enclave_b/enclave_b.signed
Enclave2: ***/home/xrf/mysamples/attestation/common/crypto.cpp(80): mbedtls initialized.
Host: Enclave successfully created.
Host: environment variable SGX_AESM_ADDR is not set


Host: ********** Attest enclave_a to enclave_b **********

Host: Requesting enclave_b format settings
Enclave2: ***/home/xrf/mysamples/attestation/common/dispatcher.cpp(79): get_enclave_format_settings
Host: Requesting enclave_a to generate a targeted evidence with an encryption key
Enclave1: ***/home/xrf/mysamples/attestation/common/dispatcher.cpp(133): get_evidence_with_public_key
Enclave1: ***/home/xrf/mysamples/attestation/common/attestation.cpp(94): oe_serialize_custom_claims
Enclave1: ***/home/xrf/mysamples/attestation/common/attestation.cpp(105): serialized custom claims buffer size: 121
Enclave1: ***/home/xrf/mysamples/attestation/common/attestation.cpp(126): generate_attestation_evidence succeeded.
Enclave1: ***/home/xrf/mysamples/attestation/common/dispatcher.cpp(179): get_evidence_with_public_key succeeded
Host: enclave_a's  public key: 
-----BEGIN PUBLIC KEY-----
MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAvCIjYJ/8lf7Vb592iVJW
QdDr6AwcajZspLXSLp0y1psCDZhGo31q4jEyyN89ebDKI1gYSCYhwb+kLYe/+yKX
J/mGNl++oYtpG8Sn3lzpfCAZWsmuu1oFGY8WvVl/vPJGdrNbVYEoLFAqMD/3QBh/
ErpCmNrD58RHjjbk6UsjWOSchry15JBC04rrQ9duSoVH5url/29FDKLNT9jZ+7XN
gxgm24IGrL1qlH4jP9XLLg5e+soC2YIf3v45K62L7k/dE7b5MWgTyT4f7uHvTJOv
BuUd+QM7tVuyr/GSY0JViJGPCU/xNL/jBV+ScjwhJU5pPmuhYfGKPM/YpHvygm5T
2QIDAQAB
-----END PUBLIC KEY-----

Host: verify_evidence_and_set_public_key in enclave_b
Enclave2: ***/home/xrf/mysamples/attestation/common/attestation.cpp(201): oe_verify_evidence failed (OE_TCB_LEVEL_INVALID).

Enclave2: ***/home/xrf/mysamples/attestation/common/dispatcher.cpp(221): verify_evidence_and_set_public_key failed.
Host: verify_evidence_and_set_public_key failed. OE_OK
Host: attestation failed with 1
Host: Terminating enclaves
Enclave1: ***/home/xrf/mysamples/attestation/common/crypto.cpp(94): mbedtls cleaned up.
Host: Enclave successfully terminated.
Enclave2: ***/home/xrf/mysamples/attestation/common/crypto.cpp(94): mbedtls cleaned up.
Host: Enclave successfully terminated.
Host:  failed 
make[3]: *** [CMakeFiles/runsgxremote.dir/build.make:57: CMakeFiles/runsgxremote] Error 1
make[2]: *** [CMakeFiles/Makefile2:107: CMakeFiles/runsgxremote.dir/all] Error 2
make[1]: *** [CMakeFiles/Makefile2:185: CMakeFiles/run.dir/rule] Error 2
make: *** [Makefile:157: run] Error 2

It appears that there is no issue with the attestation step. Can you confirm if your openenclave sample is running successfully? It is possible that there might be a problem with your SDK environment.

[pw0rld]

Maybe you can try to rebuild the openenclave sdk, Narrator use this version openenclave