Is there a way to only build the changes made?

[beejones]

Hi,

I am really enjoying my experiments with edgelessdb, hence my question. The only way I've found to build the image is to run:

DOCKER_BUILDKIT=1 sudo docker build -t edb --no-cache --secret id=signingkey,src=private.pem - < Dockerfile

Clearly it is very time consuming to rebuild everything after a change.
Is there a way to only build the changes made?

Thanks

[thomasten]

Hi @beejones,
if you want to test your source code changes, it might be easier to not build the Docker image, but just the executable: https://github.com/edgelesssys/edgelessdb/blob/main/BUILD.md#build-from-source

[beejones]

Hi @thomasten,

I am working on an SGX machine.
cpuid | grep SGX
SGX: Software Guard Extensions supported = true
SGX_LC: SGX launch config supported = true
Software Guard Extensions (SGX) capability (0x12/0):
SGX1 supported = true
SGX2 supported = false
SGX ENCLV EVIRTCHILD, ESETCONTEXT = false
SGX ENCLS ETRACKC, ERDINFO, ELDBC, ELDUC = false
SGX attributes (0x12/1):
SGX: Software Guard Extensions supported = true
SGX_LC: SGX launch config supported = true
Software Guard Extensions (SGX) capability (0x12/0):
SGX1 supported = true
SGX2 supported = false
SGX ENCLV EVIRTCHILD, ESETCONTEXT = false
SGX ENCLS ETRACKC, ERDINFO, ELDBC, ELDUC = false
SGX attributes (0x12/1):

dmesg | grep -i sgx
[ 2.610967] sgx: EPC section 0x2c0200000-0x2c37fffff
[ 2.611425] sgx: EPC section 0x2c3a00000-0x2c3bfffff
[ 2.625208] sgx: intel_sgx: Intel SGX DCAP Driver v1.33

I installed edgelessrt (https://github.com/edgelesssys/edgelessrt) and openenclave SDK (https://github.com/openenclave/openenclave/blob/master/docs/GettingStartedDocs/install_oe_sdk-Ubuntu_20.04.md)

when I run cmake .. in build I get errors.
Any idea what I am missing?
Thanks

cmake ..
-- Looking for crypto library - found
-- Looking for dl library - found
-- Configuring done
CMake Error at CMakeLists.txt:230 (add_executable):
Target "edb-enclave" links to target "openenclave::ertcalls" but the target
was not found. Perhaps a find_package() call is missing for an IMPORTED
target, or an ALIAS target is missing?

CMake Error at CMakeLists.txt:230 (add_executable):
Target "edb-enclave" links to target "openenclave::ertcalls" but the target
was not found. Perhaps a find_package() call is missing for an IMPORTED
target, or an ALIAS target is missing?

CMake Error at CMakeLists.txt:182 (add_executable):
Target "emariadbd" links to target "openenclave::ertdeventry" but the
target was not found. Perhaps a find_package() call is missing for an
IMPORTED target, or an ALIAS target is missing?

CMake Error at CMakeLists.txt:182 (add_executable):
Target "emariadbd" links to target "openenclave::ertdeventry" but the
target was not found. Perhaps a find_package() call is missing for an
IMPORTED target, or an ALIAS target is missing?

-- Generating done
CMake Generate step failed. Build files cannot be regenerated correctly.

[Nirusu]

You only need to install Edgeless RT, not both. Edgeless RT is a fork from Open Enclave, so if you install the latter it will conflict with the former.

[beejones]

Hi,

Thanks
I had to make some additional changes to run my build.
I needed to install libncurses5-dev.
and
export CMAKE_PREFIX_PATH=/opt/edgelessrt; cmake ..

Now the build runs but it reports 2 errors:
Any suggestions how to fix this?
Thanks

make -jnproc
[ 5%] Performing build step for 'mariadb'
make[3]: warning: -j2 forced in submake: resetting jobserver mode.
[ 10%] Built target genkey
/bin/sh: 1: ertgo: not found
make[2]: *** [CMakeFiles/edb-golib.dir/build.make:57: CMakeFiles/edb-golib] Error 127
make[1]: *** [CMakeFiles/Makefile2:173: CMakeFiles/edb-golib.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
[ 0%] Built target INFO_SRC
[ 0%] Built target INFO_BIN
[ 0%] Built target wsrep_api_v26
[ 0%] Built target abi_check
[ 3%] Built target readline
[ 3%] Built target mytap
[ 3%] Built target gen_lex_token
[ 4%] Built target pcre2
[ 12%] Built target mysys
[ 12%] Built target vio
[ 12%] Built target tpool
[ 12%] Built target edgeless_o
[ 12%] Built target gen_lex_hash
[ 13%] Built target caching_sha2_password
[ 13%] Built target sha256_password
[ 16%] Built target client_ed25519
[ 16%] Built target dialog
[ 16%] Built target mysql_clear_password
[ 18%] Built target mariadb_obj
[ 18%] Built target mariadb_config
[ 18%] Built target cctap
[ 18%] Built target ma_getopt
[ 18%] Built target edgeless
[ 21%] Built target ref10
[ 21%] Built target dbug-unit-tests
[ 22%] Built target hsclient
[ 22%] Built target comp_sql
[ 24%] Built target mysqlservices
[ 24%] Built target my_safe_process
[ 25%] Built target wsrep_check_version
[ 26%] Built target wsrep-lib
[ 29%] Built target strings
[ 29%] Built target simple-t
[ 29%] Built target skip_all-t
[ 30%] Built target todo-t
[ 30%] Built target no_plan-t
[ 30%] Built target skip-t
[ 31%] Built target mariadbclient
[ 31%] Built target libmariadb
[ 31%] Built target SYM_libmysqlclient.a
[ 31%] Built target ps_bugs
[ 31%] Built target ps_new
[ 31%] Built target charset
[ 31%] Built target errors
[ 31%] Built target sp
[ 31%] Built target async
[ 46%] Built target rocksdblib
[ 47%] Built target rpl_api
[ 47%] Built target view
[ 48%] Built target logs
[ 48%] Built target cursor
[ 48%] Built target ps
[ 48%] Built target basic-t
[ 48%] Built target conc336
[ 48%] Built target bulk1
[ 48%] Built target result
[ 50%] Built target connection
[ 51%] Built target misc
[ 51%] Built target thread
[ 51%] Built target fetch
[ 51%] Built target performance
[ 51%] Built target features-10_2
[ 52%] Built target t_aurora
[ 52%] Built target t_conc173
[ 52%] Built target sst_dump
[ 52%] Built target GenFixPrivs
[ 52%] Built target dbug
[ 52%] Built target SYM_libmysqlclient_r.a
[ 52%] Built target SYM_libmysqlclient_r.so
[ 52%] Built target SYM_libmysqlclient.so
[ 53%] Built target factorial
[ 53%] Built target tests
[ 53%] Built target test_hash
[ 53%] Built target thr_timer
[ 53%] Built target thr_lock
[ 54%] Built target mariadb-conv
[ 54%] Built target my_print_defaults
[ 54%] Built target resolveip
[ 54%] Built target resolve_stack_dump
[ 54%] Built target replace
[ 54%] Built target mariadbd-safe-helper
[ 54%] Built target mariadb-waitpid
[ 54%] Built target udf_example
[ 54%] Built target json-t
[ 54%] Built target strings-t
[ 54%] Built target ma_dyncol-t
[ 55%] Built target mysys_ssl
[ 55%] Built target stacktrace-t
[ 56%] Built target base64-t
[ 56%] Built target my_atomic-t
[ 57%] Built target my_rdtsc-t
[ 57%] Built target lf-t
[ 57%] Built target my_malloc-t
[ 57%] Built target my_getopt-t
[ 57%] Built target bitmap-t
[ 57%] Built target dynstring-t
[ 57%] Built target byte_order-t
[ 57%] Built target queues-t
[ 57%] Built target crc32-t
[ 57%] Built target my_vsnprintf-t
[ 57%] Built target my_decimal-t
[ 57%] Built target json_lib-t
[ 57%] Built target comp_err
[ 57%] Built target my_apc-t
[ 57%] Built target ed25519-t
[ 58%] Built target user_t
[ 58%] Built target user_ps
[ 58%] Built target mariadb-tzinfo-to-sql
[ 58%] Built target aes-t
[ 58%] Built target GenError
[ 58%] Built target mf_iocache-t
[ 62%] Built target myisam
[ 64%] Built target wsrep
[ 64%] Built target sql_sequence
[ 64%] Built target rocksdb_aux_lib
[ 64%] Built target csv
[ 64%] Built target myisam_ftdump
[ 66%] Built target heap
[ 66%] Built target mi_test1
[ 66%] Built target rt_test
[ 66%] Built target myisampack
[ 66%] Built target mi_test2
[ 66%] Built target myisamlog
[ 67%] Built target sp_test
[ 67%] Built target myisamchk
[ 67%] Built target mi_test3
[ 69%] Built target myisammrg
[ 69%] Built target type_geom
[ 69%] Built target userstat
[ 70%] Built target type_inet
[ 70%] Built target mariadb-admin
[ 71%] Built target mariadb
[ 71%] Built target mariadb-check
[ 71%] Built target mariadb-dump
[ 71%] Built target mariadb-import
[ 71%] Built target mariadb-upgrade
[ 71%] Built target mariadb-show
[ 71%] Built target mariadb-plugin
[ 72%] Built target mariadb-test
[ 72%] Built target mariadb-binlog
[ 72%] Built target mariadb-slap
[ 72%] Built target perror
[ 72%] Built target async_example
[ 72%] Built target bug25714
[ 72%] Built target mariadb-client-test
[ 77%] Built target aria
[ 78%] Built target rocksdb_se
[ 78%] Built target hp_test1
[ 78%] Built target hp_test2
[ 78%] Built target aria_dump_log
[ 78%] Built target aria_ftdump
[ 78%] Built target ma_test2
[ 79%] Built target aria_chk
[ 79%] Built target aria_read_log
[ 79%] Built target ma_test3
[ 79%] Built target aria_pack
[ 79%] Built target ma_test1
[ 79%] Built target ma_rt_test
[ 79%] Built target test_ma_backup
[ 79%] Built target ma_sp_test
[ 79%] Built target ma_test_loghandler_nologs-t
[ 79%] Built target ma_pagecache_rwconsist_1k-t
[ 80%] Built target ma_test_loghandler_multigroup-t
[ 80%] Built target ma_test_loghandler_purge-t
[ 81%] Built target ma_pagecache_consist_1kRD-t
[ 81%] Built target ma_test_loghandler_readonly-t
[ 81%] Built target ma_test_loghandler_multithread-t
[ 81%] Built target ma_pagecache_consist_1k-t
[ 82%] Built target ma_test_loghandler_pagecache-t
[ 82%] Built target ma_pagecache_consist_64kRD-t
[ 82%] Built target ma_test_loghandler_noflush-t
[ 82%] Built target ma_test_loghandler-t
[ 82%] Built target trnman-t
[ 82%] Built target ma_test_loghandler_max_lsn-t
[ 82%] Built target ma_control_file-t
[ 82%] Built target ma_pagecache_consist_64k-t
[ 82%] Built target ma_test_loghandler_long-t
[ 83%] Built target ma_pagecache_single_8k-t
[ 83%] Built target ma_pagecache_single_64k-t
[ 83%] Built target ma_pagecache_consist_1kHC-t
[ 83%] Built target ma_pagecache_consist_64kHC-t
[ 83%] Built target ma_pagecache_consist_1kWR-t
[ 84%] Built target ma_pagecache_consist_64kWR-t
[ 84%] Built target ma_pagecache_rwconsist2_1k-t
[ 84%] Built target ma_pagecache_single_1k-t
[ 85%] Built target ma_test_loghandler_first_lsn-t
[ 85%] Built target sql_builtins
[100%] Built target sql
[100%] Built target mariadbd
[100%] Built target explain_filename-t
[ 15%] No install step for 'mariadb'
[ 20%] Completed 'mariadb'
[ 45%] Built target mariadb
make: *** [Makefile:95: all] Error 2

[Nirusu]

Did you run this in your current terminal session after installing Edgeless RT?

. /opt/edgelessrt/share/openenclave/openenclaverc

This adds Edgeless RT tools to the path, which also adds ertgo (our modified Go compiler for enclaves) to your PATH.

You can also just add this to your .bashrc if you don't want to execute this command every time.

[beejones]

Almost there I hope.

Build completes but I get a signature error.
[100%] Generating edb-enclave.signed
/opt/edgelessrt/bin/oesign ERROR: /home/ronny/edgelessdb/build/enclave.conf(4): bad value for 'NumTCS': 64
/opt/edgelessrt/bin/oesign ERROR: Failed to load configuration file: /home/ronny/edgelessdb/build/enclave.conf
make[2]: *** [CMakeFiles/sign-edb.dir/build.make:61: edb-enclave.signed] Error 1
make[1]: *** [CMakeFiles/Makefile2:90: CMakeFiles/sign-edb.dir/all] Error 2
make: *** [Makefile:95: all] Error 2

Do I need to run a tool to generate hashes?
Thanks

[Nirusu]

Oh, that's a recent change in Edgeless RT. You cannot use a release version of Edgeless RT to build the current main commit from EdgelessDB, you need to build Edgeless RT from its master branch by source to include recent changes, then you can build EdgelessDB.

Alternatively, you can just turn down NumTCS in the enclave.conf to 32. Should not change that much, we just recently increased the number of allowed threads (which we had to patch in Edgeless RT) to avoid some rare crashes where we run out of threads.