Build Failure

[bows7ring]

Awesome project!

I want to run it on QEMU, but I'm having some build problems.

Build Failure:

Ubuntu 18.04.

Working on master branch.

Result of git submodule status:

1fcdfbfb8a28b81283575efc3faa92ce4a0d4e56 buildroot (2019.08)
4917510626c55c1f199ef7383ae164cf96044aea libsodium (1.0.16-107-g49175106)
4d856f72c10ecb060868ed10ff1b1453943fc6c8 linux (v5.3)
3fe7e2c75df78eef42dcdc352a55757729f451e2 musl (heads/staging)
d4bd55eb41c01482ba6ce3dc0d648f952d75a06c qemu (heads/pmpfix)

Result of git branch -a:

master
remotes/origin/HEAD -> origin/master
remotes/origin/master

Error Log:

Bug 0:

Firstly, I followed the steps in Readme. I ran ./docker.sh with all submodules updated.
But I found some small bugs, as shown in log message below:

make -s -C /keystone/buildroot RISCV=/keystone/riscv/ PATH=/keystone/riscv//bin:/keystone/riscv/bin:/tools/riscv/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin O=/keystone/build/buildroot.build
make[4]: Entering directory '/keystone/buildroot'
>>> toolchain-external-custom Configuring�[27m
Cannot execute cross-compiler '/keystone/riscv//bin/riscv64-unknown-linux-gnu-gcc'
make[4]: ** [package/pkg-generic.mk:231: /keystone/build/buildroot.build/build/toolchain-external-custom/.stamp_configured] Error 1
make[4]: Leaving directory '/keystone/buildroot'
make[3]: ** [CMakeFiles/image.dir/build.make:63: CMakeFiles/image] Error 2
make[3]: Leaving directory '/keystone/build'
make[2]: ** [CMakeFiles/Makefile2:290: CMakeFiles/image.dir/all] Error 2
make[2]: Leaving directory '/keystone/build'
make[1]: ** [CMakeFiles/Makefile2:297: CMakeFiles/image.dir/rule] Error 2
make[1]: Leaving directory '/keystone/build'
make: *** [Makefile:212: image] Error 2

Wrong PATH. I changed the relevant 'Makefile' to get it fixed, and I decided to build it step by step.

---

• sudo ./docker-build.sh
  Succeed.

---

• sudo ./docker-run.sh ./fast-setup.sh

  make: Leaving directory '/keystone/sdk'
  Keystone is fully setup! You can build everything and run the tests with 'make run-tests'

  Looks right. But I got:

  make: *** No rule to make target 'run-tests'. Stop.

---

• sudo ./docker-run.sh ./make-sodium.sh

  Succeed.

---

• sudo ./docker-run.sh ./make-musl.sh

  Succeed.

BUG 1:

• There is something wrong in this step: sudo ./docker-run.sh ./make.sh

  Here is Error Log:

[ 28%] Building buildroot
make -s -C /keystone/buildroot RISCV=/keystone/riscv64 PATH=/keystone/riscv64/bin:/tools/riscv/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin O=/keystone/build/buildroot.build
make[3]: Entering directory '/keystone/buildroot'
make[4]: Entering directory '/keystone/buildroot'
GEN /keystone/build/buildroot.build/Makefile
make[4]: Leaving directory '/keystone/buildroot'

>>> toolchain-external-custom Configuring
Incorrect selection of kernel headers: expected 4.13.x, got 5.0.x
make[3]: *** [package/pkg-generic.mk:231: /keystone/build/buildroot.build/build/toolchain-external-custom/.stamp_configured] Error 1
make[3]: Leaving directory '/keystone/buildroot'
make[2]: *** [CMakeFiles/buildroot.dir/build.make:64: CMakeFiles/buildroot] Error 2
make[2]: Leaving directory '/keystone/build'
make[1]: *** [CMakeFiles/Makefile2:208: CMakeFiles/buildroot.dir/all] Error 2
make[1]: Leaving directory '/keystone/build'
make: *** [Makefile:87: all] Error 2

Well, it seems that my kernel header is too NEW. I found some comments in buildroot/toolchain/toolchain/toolchain.mk:

# Apply a hack that Rick Felker suggested[1] to avoid conflicts between libc
# headers and kernel headers. This is needed for kernel headers older than
# 4.15. Kernel headers 4.15 and newer don't require GLIBC to be defined.
#
# Augment the original suggestion with __USE_MISC since recent kernels
# (older than 4.15) require this glibc internal macro. Also, as musl defines
# IFF_LOWER_UP, IFF_DORMANT and IFF_ECHO, add another macro to suppress
# them in the kernel header, and avoid macro/enum conflict.
#
# Kernel version 3.12 introduced the libc-compat.h header.
#
# [1] http://www.openwall.com/lists/musl/2015/10/08/2

So I changed the kernel header to 4.14 via make menuconfig in /buildroot, but failed again with the same Log.

Incorrect selection of kernel headers: expected 4.13.x, got 5.0.x

Bug 2:

• ./docker-run.sh ./make-sdk.sh

/keystone/riscv64/bin:/tools/riscv/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
make: Entering directory '/keystone/sdk/lib'
make -C app
make[1]: Entering directory '/keystone/sdk/lib/app'
make[1]: Nothing to be done for 'all'.
make[1]: Leaving directory '/keystone/sdk/lib/app'
make -C host
make[1]: Entering directory '/keystone/sdk/lib/host'
make[1]: Nothing to be done for 'all'.
make[1]: Leaving directory '/keystone/sdk/lib/host'
make -C edge
make[1]: Entering directory '/keystone/sdk/lib/edge'
make[1]: Nothing to be done for 'all'.
make[1]: Leaving directory '/keystone/sdk/lib/edge'
make -C verifier
make[1]: Entering directory '/keystone/sdk/lib/verifier'
#riscv64-unknown-linux-gnu-g++ -std=gnu++11 test.cpp ../libkeystone-verifier.a -o test
make[1]: Leaving directory '/keystone/sdk/lib/verifier'
make: Leaving directory '/keystone/sdk/lib'
make: Entering directory '/keystone/sdk/rts/eyrie'
riscv64-unknown-linux-gnu-ld -static -nostdlib --no-dynamic-linker -z text -L/keystone/sdk/lib -o eyrie-rt entry.o fp_asm.o boot.o interrupt.o printf.o syscall.o string.o linux_wrap.o io_wrap.o fast_io_wrap.o rt_util.o mm.o env.o freemem.o paging.o performance.o /keystone/sdk/rts/eyrie/rpc/rpc.o /keystone/sdk/rts/eyrie/rpc/rpc_crypto.o /keystone/sdk/rts/eyrie/rpc/edge_wrapper.o /keystone/sdk/rts/eyrie/rpc/rpc_crypto_enc.o /keystone/sdk/lib/libkeystone-edge.a uaccess.o -T runtime.lds
riscv64-unknown-linux-gnu-ld: interrupt.o:(.bss+0x0): multiple definition of `runtime_va_start'; boot.o:(.bss+0x0): first defined here
riscv64-unknown-linux-gnu-ld: interrupt.o:(.bss+0x8): multiple definition of `kernel_offset'; boot.o:(.bss+0x8): first defined here
riscv64-unknown-linux-gnu-ld: interrupt.o:(.bss+0x10): multiple definition of `load_pa_start'; boot.o:(.bss+0x10): first defined here
riscv64-unknown-linux-gnu-ld: interrupt.o:(.bss+0x1000): multiple definition of `root_page_table'; boot.o:(.bss+0x1000): first defined here
riscv64-unknown-linux-gnu-ld: interrupt.o:(.bss+0x2000): multiple definition of `secondary_page_tables'; boot.o:(.bss+0x2000): first defined here
riscv64-unknown-linux-gnu-ld: interrupt.o:(.bss+0x202000): multiple definition of `page_tables_count'; boot.o:(.bss+0x202000): first defined here
riscv64-unknown-linux-gnu-ld: interrupt.o:(.bss+0x202008): multiple definition of `program_break'; boot.o:(.bss+0x202008): first defined here
riscv64-unknown-linux-gnu-ld: interrupt.o:(.bss+0x202010): multiple definition of `freemem_va_start'; boot.o:(.bss+0x202010): first defined here
riscv64-unknown-linux-gnu-ld: interrupt.o:(.bss+0x202018): multiple definition of `freemem_size'; boot.o:(.bss+0x202018): first defined here
riscv64-unknown-linux-gnu-ld: interrupt.o:(.bss+0x202020): multiple definition of `shared_buffer'; boot.o:(.bss+0x202020): first defined here
riscv64-unknown-linux-gnu-ld: interrupt.o:(.bss+0x202028): multiple definition of `shared_buffer_size'; boot.o:(.bss+0x202028): first defined here
riscv64-unknown-linux-gnu-ld: syscall.o:(.bss+0x0): multiple definition of `runtime_va_start'; boot.o:(.bss+0x0): first defined here
riscv64-unknown-linux-gnu-ld: syscall.o:(.bss+0x8): multiple definition of `kernel_offset'; boot.o:(.bss+0x8): first defined here
riscv64-unknown-linux-gnu-ld: syscall.o:(.bss+0x10): multiple definition of `load_pa_start'; boot.o:(.bss+0x10): first defined here
riscv64-unknown-linux-gnu-ld: syscall.o:(.bss+0x1000): multiple definition of `root_page_table'; boot.o:(.bss+0x1000): first defined here
riscv64-unknown-linux-gnu-ld: syscall.o:(.bss+0x2000): multiple definition of `secondary_page_tables'; boot.o:(.bss+0x2000): first defined here
riscv64-unknown-linux-gnu-ld: syscall.o:(.bss+0x202000): multiple definition of `page_tables_count'; boot.o:(.bss+0x202000): first defined here
riscv64-unknown-linux-gnu-ld: syscall.o:(.bss+0x202008): multiple definition of `program_break'; boot.o:(.bss+0x202008): first defined here
riscv64-unknown-linux-gnu-ld: syscall.o:(.bss+0x202010): multiple definition of `freemem_va_start'; boot.o:(.bss+0x202010): first defined here
riscv64-unknown-linux-gnu-ld: syscall.o:(.bss+0x202018): multiple definition of `freemem_size'; boot.o:(.bss+0x202018): first defined here
riscv64-unknown-linux-gnu-ld: syscall.o:(.bss+0x202020): multiple definition of `shared_buffer'; boot.o:(.bss+0x202020): first defined here
riscv64-unknown-linux-gnu-ld: syscall.o:(.bss+0x202028): multiple definition of `shared_buffer_size'; boot.o:(.bss+0x202028): first defined here
riscv64-unknown-linux-gnu-ld: linux_wrap.o:(.bss+0x0): multiple definition of `runtime_va_start'; boot.o:(.bss+0x0): first defined here
riscv64-unknown-linux-gnu-ld: linux_wrap.o:(.bss+0x8): multiple definition of `kernel_offset'; boot.o:(.bss+0x8): first defined here
riscv64-unknown-linux-gnu-ld: linux_wrap.o:(.bss+0x10): multiple definition of `load_pa_start'; boot.o:(.bss+0x10): first defined here
riscv64-unknown-linux-gnu-ld: linux_wrap.o:(.bss+0x1000): multiple definition of `root_page_table'; boot.o:(.bss+0x1000): first defined here
riscv64-unknown-linux-gnu-ld: linux_wrap.o:(.bss+0x2000): multiple definition of `secondary_page_tables'; boot.o:(.bss+0x2000): first defined here
riscv64-unknown-linux-gnu-ld: linux_wrap.o:(.bss+0x202000): multiple definition of `page_tables_count'; boot.o:(.bss+0x202000): first defined here
riscv64-unknown-linux-gnu-ld: linux_wrap.o:(.bss+0x202008): multiple definition of `program_break'; boot.o:(.bss+0x202008): first defined here
riscv64-unknown-linux-gnu-ld: linux_wrap.o:(.bss+0x202010): multiple definition of `freemem_va_start'; boot.o:(.bss+0x202010): first defined here
riscv64-unknown-linux-gnu-ld: linux_wrap.o:(.bss+0x202018): multiple definition of `freemem_size'; boot.o:(.bss+0x202018): first defined here
riscv64-unknown-linux-gnu-ld: linux_wrap.o:(.bss+0x202020): multiple definition of `shared_buffer'; boot.o:(.bss+0x202020): first defined here
riscv64-unknown-linux-gnu-ld: linux_wrap.o:(.bss+0x202028): multiple definition of `shared_buffer_size'; boot.o:(.bss+0x202028): first defined here
riscv64-unknown-linux-gnu-ld: io_wrap.o:(.bss+0x0): multiple definition of `runtime_va_start'; boot.o:(.bss+0x0): first defined here
riscv64-unknown-linux-gnu-ld: io_wrap.o:(.bss+0x8): multiple definition of `kernel_offset'; boot.o:(.bss+0x8): first defined here
riscv64-unknown-linux-gnu-ld: io_wrap.o:(.bss+0x10): multiple definition of `load_pa_start'; boot.o:(.bss+0x10): first defined here
riscv64-unknown-linux-gnu-ld: io_wrap.o:(.bss+0x1000): multiple definition of `root_page_table'; boot.o:(.bss+0x1000): first defined here
riscv64-unknown-linux-gnu-ld: io_wrap.o:(.bss+0x2000): multiple definition of `secondary_page_tables'; boot.o:(.bss+0x2000): first defined here
riscv64-unknown-linux-gnu-ld: io_wrap.o:(.bss+0x202000): multiple definition of `page_tables_count'; boot.o:(.bss+0x202000): first defined here
riscv64-unknown-linux-gnu-ld: io_wrap.o:(.bss+0x202008): multiple definition of `program_break'; boot.o:(.bss+0x202008): first defined here
riscv64-unknown-linux-gnu-ld: io_wrap.o:(.bss+0x202010): multiple definition of `freemem_va_start'; boot.o:(.bss+0x202010): first defined here
riscv64-unknown-linux-gnu-ld: io_wrap.o:(.bss+0x202018): multiple definition of `freemem_size'; boot.o:(.bss+0x202018): first defined here
riscv64-unknown-linux-gnu-ld: io_wrap.o:(.bss+0x202020): multiple definition of `shared_buffer'; boot.o:(.bss+0x202020): first defined here
riscv64-unknown-linux-gnu-ld: io_wrap.o:(.bss+0x202028): multiple definition of `shared_buffer_size'; boot.o:(.bss+0x202028): first defined here
riscv64-unknown-linux-gnu-ld: fast_io_wrap.o:(.bss+0x0): multiple definition of `runtime_va_start'; boot.o:(.bss+0x0): first defined here
riscv64-unknown-linux-gnu-ld: fast_io_wrap.o:(.bss+0x8): multiple definition of `kernel_offset'; boot.o:(.bss+0x8): first defined here
riscv64-unknown-linux-gnu-ld: fast_io_wrap.o:(.bss+0x10): multiple definition of `load_pa_start'; boot.o:(.bss+0x10): first defined here
riscv64-unknown-linux-gnu-ld: fast_io_wrap.o:(.bss+0x1000): multiple definition of `root_page_table'; boot.o:(.bss+0x1000): first defined here
riscv64-unknown-linux-gnu-ld: fast_io_wrap.o:(.bss+0x2000): multiple definition of `secondary_page_tables'; boot.o:(.bss+0x2000): first defined here
riscv64-unknown-linux-gnu-ld: fast_io_wrap.o:(.bss+0x202000): multiple definition of `page_tables_count'; boot.o:(.bss+0x202000): first defined here
riscv64-unknown-linux-gnu-ld: fast_io_wrap.o:(.bss+0x202008): multiple definition of `program_break'; boot.o:(.bss+0x202008): first defined here
riscv64-unknown-linux-gnu-ld: fast_io_wrap.o:(.bss+0x202010): multiple definition of `freemem_va_start'; boot.o:(.bss+0x202010): first defined here
riscv64-unknown-linux-gnu-ld: fast_io_wrap.o:(.bss+0x202018): multiple definition of `freemem_size'; boot.o:(.bss+0x202018): first defined here
riscv64-unknown-linux-gnu-ld: fast_io_wrap.o:(.bss+0x202020): multiple definition of `shared_buffer'; boot.o:(.bss+0x202020): first defined here
riscv64-unknown-linux-gnu-ld: fast_io_wrap.o:(.bss+0x202028): multiple definition of `shared_buffer_size'; boot.o:(.bss+0x202028): first defined here
riscv64-unknown-linux-gnu-ld: rt_util.o:(.bss+0x0): multiple definition of `runtime_va_start'; boot.o:(.bss+0x0): first defined here
riscv64-unknown-linux-gnu-ld: rt_util.o:(.bss+0x8): multiple definition of `kernel_offset'; boot.o:(.bss+0x8): first defined here
riscv64-unknown-linux-gnu-ld: rt_util.o:(.bss+0x10): multiple definition of `load_pa_start'; boot.o:(.bss+0x10): first defined here
riscv64-unknown-linux-gnu-ld: rt_util.o:(.bss+0x1000): multiple definition of `root_page_table'; boot.o:(.bss+0x1000): first defined here
riscv64-unknown-linux-gnu-ld: rt_util.o:(.bss+0x2000): multiple definition of `secondary_page_tables'; boot.o:(.bss+0x2000): first defined here
riscv64-unknown-linux-gnu-ld: rt_util.o:(.bss+0x202000): multiple definition of `page_tables_count'; boot.o:(.bss+0x202000): first defined here
riscv64-unknown-linux-gnu-ld: rt_util.o:(.bss+0x202008): multiple definition of `program_break'; boot.o:(.bss+0x202008): first defined here
riscv64-unknown-linux-gnu-ld: rt_util.o:(.bss+0x202010): multiple definition of `freemem_va_start'; boot.o:(.bss+0x202010): first defined here
riscv64-unknown-linux-gnu-ld: rt_util.o:(.bss+0x202018): multiple definition of `freemem_size'; boot.o:(.bss+0x202018): first defined here
riscv64-unknown-linux-gnu-ld: rt_util.o:(.bss+0x202020): multiple definition of `shared_buffer'; boot.o:(.bss+0x202020): first defined here
riscv64-unknown-linux-gnu-ld: rt_util.o:(.bss+0x202028): multiple definition of `shared_buffer_size'; boot.o:(.bss+0x202028): first defined here
riscv64-unknown-linux-gnu-ld: mm.o:(.bss+0x0): multiple definition of `runtime_va_start'; boot.o:(.bss+0x0): first defined here
riscv64-unknown-linux-gnu-ld: mm.o:(.bss+0x8): multiple definition of `kernel_offset'; boot.o:(.bss+0x8): first defined here
riscv64-unknown-linux-gnu-ld: mm.o:(.bss+0x10): multiple definition of `load_pa_start'; boot.o:(.bss+0x10): first defined here
riscv64-unknown-linux-gnu-ld: mm.o:(.bss+0x1000): multiple definition of `root_page_table'; boot.o:(.bss+0x1000): first defined here
riscv64-unknown-linux-gnu-ld: mm.o:(.bss+0x2000): multiple definition of `secondary_page_tables'; boot.o:(.bss+0x2000): first defined here
riscv64-unknown-linux-gnu-ld: mm.o:(.bss+0x202000): multiple definition of `page_tables_count'; boot.o:(.bss+0x202000): first defined here
riscv64-unknown-linux-gnu-ld: mm.o:(.bss+0x202008): multiple definition of `program_break'; boot.o:(.bss+0x202008): first defined here
riscv64-unknown-linux-gnu-ld: mm.o:(.bss+0x202010): multiple definition of `freemem_va_start'; boot.o:(.bss+0x202010): first defined here
riscv64-unknown-linux-gnu-ld: mm.o:(.bss+0x202018): multiple definition of `freemem_size'; boot.o:(.bss+0x202018): first defined here
riscv64-unknown-linux-gnu-ld: mm.o:(.bss+0x202020): multiple definition of `shared_buffer'; boot.o:(.bss+0x202020): first defined here
riscv64-unknown-linux-gnu-ld: mm.o:(.bss+0x202028): multiple definition of `shared_buffer_size'; boot.o:(.bss+0x202028): first defined here
riscv64-unknown-linux-gnu-ld: env.o:(.bss+0x0): multiple definition of `runtime_va_start'; boot.o:(.bss+0x0): first defined here
riscv64-unknown-linux-gnu-ld: env.o:(.bss+0x8): multiple definition of `kernel_offset'; boot.o:(.bss+0x8): first defined here
riscv64-unknown-linux-gnu-ld: env.o:(.bss+0x10): multiple definition of `load_pa_start'; boot.o:(.bss+0x10): first defined here
riscv64-unknown-linux-gnu-ld: env.o:(.bss+0x1000): multiple definition of `root_page_table'; boot.o:(.bss+0x1000): first defined here
riscv64-unknown-linux-gnu-ld: env.o:(.bss+0x2000): multiple definition of `secondary_page_tables'; boot.o:(.bss+0x2000): first defined here
riscv64-unknown-linux-gnu-ld: env.o:(.bss+0x202000): multiple definition of `page_tables_count'; boot.o:(.bss+0x202000): first defined here
riscv64-unknown-linux-gnu-ld: env.o:(.bss+0x202008): multiple definition of `program_break'; boot.o:(.bss+0x202008): first defined here
riscv64-unknown-linux-gnu-ld: env.o:(.bss+0x202010): multiple definition of `freemem_va_start'; boot.o:(.bss+0x202010): first defined here
riscv64-unknown-linux-gnu-ld: env.o:(.bss+0x202018): multiple definition of `freemem_size'; boot.o:(.bss+0x202018): first defined here
riscv64-unknown-linux-gnu-ld: env.o:(.bss+0x202020): multiple definition of `shared_buffer'; boot.o:(.bss+0x202020): first defined here
riscv64-unknown-linux-gnu-ld: env.o:(.bss+0x202028): multiple definition of `shared_buffer_size'; boot.o:(.bss+0x202028): first defined here
riscv64-unknown-linux-gnu-ld: freemem.o:(.bss+0x0): multiple definition of `runtime_va_start'; boot.o:(.bss+0x0): first defined here
riscv64-unknown-linux-gnu-ld: freemem.o:(.bss+0x8): multiple definition of `kernel_offset'; boot.o:(.bss+0x8): first defined here
riscv64-unknown-linux-gnu-ld: freemem.o:(.bss+0x10): multiple definition of `load_pa_start'; boot.o:(.bss+0x10): first defined here
riscv64-unknown-linux-gnu-ld: freemem.o:(.bss+0x1000): multiple definition of `root_page_table'; boot.o:(.bss+0x1000): first defined here
riscv64-unknown-linux-gnu-ld: freemem.o:(.bss+0x2000): multiple definition of `secondary_page_tables'; boot.o:(.bss+0x2000): first defined here
riscv64-unknown-linux-gnu-ld: freemem.o:(.bss+0x202000): multiple definition of `page_tables_count'; boot.o:(.bss+0x202000): first defined here
riscv64-unknown-linux-gnu-ld: freemem.o:(.bss+0x202008): multiple definition of `program_break'; boot.o:(.bss+0x202008): first defined here
riscv64-unknown-linux-gnu-ld: freemem.o:(.bss+0x202010): multiple definition of `freemem_va_start'; boot.o:(.bss+0x202010): first defined here
riscv64-unknown-linux-gnu-ld: freemem.o:(.bss+0x202018): multiple definition of `freemem_size'; boot.o:(.bss+0x202018): first defined here
riscv64-unknown-linux-gnu-ld: freemem.o:(.bss+0x202020): multiple definition of `shared_buffer'; boot.o:(.bss+0x202020): first defined here
riscv64-unknown-linux-gnu-ld: freemem.o:(.bss+0x202028): multiple definition of `shared_buffer_size'; boot.o:(.bss+0x202028): first defined here
riscv64-unknown-linux-gnu-ld: /keystone/sdk/rts/eyrie/rpc/edge_wrapper.o:(.bss+0x0): multiple definition of `runtime_va_start'; boot.o:(.bss+0x0): first defined here
riscv64-unknown-linux-gnu-ld: /keystone/sdk/rts/eyrie/rpc/edge_wrapper.o:(.bss+0x8): multiple definition of `kernel_offset'; boot.o:(.bss+0x8): first defined here
riscv64-unknown-linux-gnu-ld: /keystone/sdk/rts/eyrie/rpc/edge_wrapper.o:(.bss+0x10): multiple definition of `load_pa_start'; boot.o:(.bss+0x10): first defined here
riscv64-unknown-linux-gnu-ld: /keystone/sdk/rts/eyrie/rpc/edge_wrapper.o:(.bss+0x1000): multiple definition of `root_page_table'; boot.o:(.bss+0x1000): first defined here
riscv64-unknown-linux-gnu-ld: /keystone/sdk/rts/eyrie/rpc/edge_wrapper.o:(.bss+0x2000): multiple definition of `secondary_page_tables'; boot.o:(.bss+0x2000): first defined here
riscv64-unknown-linux-gnu-ld: /keystone/sdk/rts/eyrie/rpc/edge_wrapper.o:(.bss+0x202000): multiple definition of `page_tables_count'; boot.o:(.bss+0x202000): first defined here
riscv64-unknown-linux-gnu-ld: /keystone/sdk/rts/eyrie/rpc/edge_wrapper.o:(.bss+0x202008): multiple definition of `program_break'; boot.o:(.bss+0x202008): first defined here
riscv64-unknown-linux-gnu-ld: /keystone/sdk/rts/eyrie/rpc/edge_wrapper.o:(.bss+0x202010): multiple definition of `freemem_va_start'; boot.o:(.bss+0x202010): first defined here
riscv64-unknown-linux-gnu-ld: /keystone/sdk/rts/eyrie/rpc/edge_wrapper.o:(.bss+0x202018): multiple definition of `freemem_size'; boot.o:(.bss+0x202018): first defined here
riscv64-unknown-linux-gnu-ld: /keystone/sdk/rts/eyrie/rpc/edge_wrapper.o:(.bss+0x202020): multiple definition of `shared_buffer'; boot.o:(.bss+0x202020): first defined here
riscv64-unknown-linux-gnu-ld: /keystone/sdk/rts/eyrie/rpc/edge_wrapper.o:(.bss+0x202028): multiple definition of `shared_buffer_size'; boot.o:(.bss+0x202028): first defined here
make: *** [Makefile:47: eyrie-rt] Error 1
make: Leaving directory '/keystone/sdk/rts/eyrie'

Looks like there's something wrong with the RISC-V linker rather than eyrie. I noticed that there is a similar issue in Keystone's repo. However, I still didn't figure out the solution... Well, I'm really not an expert on toolchain or docker.

Merry Xmas !

Any help would be appreciated !

[jasonyu1996]

Hi! Thanks for trying! It's not entirely clear to me yet what specifically caused the errors you saw, but I did find that I had messed something up in the build process. Could you retry with the latest version on the master branch?

[bows7ring]

@jasonyu1996 Hi, thanks for reply, I tried again with new master branch.

This time, previous steps are OK.

Bug 0:

sudo ./docker-run.sh ./make.sh

make[4]: Entering directory '/keystone/qemu/slirp'
make[4]: Nothing to be done for 'all'.
make[4]: Leaving directory '/keystone/qemu/slirp'
CHK version_gen.h
GEN trace/generated-tcg-tracers.h
GEN trace/generated-helpers-wrappers.h
GEN trace/generated-helpers.h
GEN trace/generated-helpers.c
GEN module_block.h
GEN trace-root.h
GEN accel/kvm/trace.h
GEN accel/tcg/trace.h
GEN backends/trace.h
GEN crypto/trace.h
GEN monitor/trace.h
GEN linux-user/trace.h
GEN authz/trace.h
GEN block/trace.h
GEN io/trace.h
GEN nbd/trace.h
GEN scsi/trace.h
GEN audio/trace.h
GEN chardev/trace.h
GEN hw/9pfs/trace.h
GEN hw/acpi/trace.h
GEN hw/alpha/trace.h
GEN hw/arm/trace.h
GEN hw/audio/trace.h
GEN hw/block/trace.h
GEN hw/block/dataplane/trace.h
GEN hw/char/trace.h
GEN hw/dma/trace.h
GEN hw/hppa/trace.h
GEN hw/i2c/trace.h
GEN hw/i386/trace.h
GEN hw/i386/xen/trace.h
GEN hw/ide/trace.h
GEN hw/input/trace.h
GEN hw/intc/trace.h
GEN hw/isa/trace.h
GEN hw/mem/trace.h
GEN hw/mips/trace.h
GEN hw/misc/trace.h
GEN hw/misc/macio/trace.h
GEN hw/net/trace.h
GEN hw/nvram/trace.h
GEN hw/pci/trace.h
GEN hw/pci-host/trace.h
GEN hw/ppc/trace.h
GEN hw/rdma/trace.h
GEN hw/rdma/vmw/trace.h
GEN hw/rtc/trace.h
GEN hw/s390x/trace.h
GEN hw/scsi/trace.h
GEN hw/sd/trace.h
GEN hw/sparc/trace.h
GEN hw/sparc64/trace.h
GEN hw/timer/trace.h
GEN hw/tpm/trace.h
GEN hw/usb/trace.h
GEN hw/vfio/trace.h
GEN hw/virtio/trace.h
GEN hw/watchdog/trace.h
GEN hw/xen/trace.h
GEN hw/gpio/trace.h
GEN hw/riscv/trace.h
GEN migration/trace.h
GEN net/trace.h
GEN ui/trace.h
GEN hw/core/trace.h
GEN hw/display/trace.h
GEN qapi/trace.h
GEN qom/trace.h
GEN target/arm/trace.h
GEN target/hppa/trace.h
GEN target/i386/trace.h
GEN target/mips/trace.h
GEN target/ppc/trace.h
GEN target/riscv/trace.h
GEN target/s390x/trace.h
GEN target/sparc/trace.h
GEN util/trace.h
GEN trace-root.c
GEN accel/kvm/trace.c
GEN accel/tcg/trace.c
GEN backends/trace.c
GEN crypto/trace.c
GEN monitor/trace.c
GEN linux-user/trace.c
GEN authz/trace.c
GEN block/trace.c
GEN io/trace.c
GEN nbd/trace.c
GEN scsi/trace.c
GEN audio/trace.c
GEN chardev/trace.c
GEN hw/9pfs/trace.c
GEN hw/acpi/trace.c
GEN hw/alpha/trace.c
GEN hw/arm/trace.c
GEN hw/audio/trace.c
GEN hw/block/trace.c
GEN hw/block/dataplane/trace.c
GEN hw/char/trace.c
GEN hw/dma/trace.c
GEN hw/hppa/trace.c
GEN hw/i2c/trace.c
GEN hw/i386/trace.c
GEN hw/i386/xen/trace.c
GEN hw/ide/trace.c
GEN hw/input/trace.c
GEN hw/intc/trace.c
GEN hw/isa/trace.c
GEN hw/mem/trace.c
GEN hw/mips/trace.c
GEN hw/misc/trace.c
GEN hw/misc/macio/trace.c
GEN hw/net/trace.c
GEN hw/nvram/trace.c
GEN hw/pci/trace.c
GEN hw/pci-host/trace.c
GEN hw/ppc/trace.c
GEN hw/rdma/trace.c
GEN hw/rdma/vmw/trace.c
GEN hw/rtc/trace.c
GEN hw/s390x/trace.c
GEN hw/scsi/trace.c
GEN hw/sd/trace.c
GEN hw/sparc/trace.c
GEN hw/sparc64/trace.c
GEN hw/timer/trace.c
GEN hw/tpm/trace.c
GEN hw/usb/trace.c
GEN hw/vfio/trace.c
GEN hw/virtio/trace.c
GEN hw/watchdog/trace.c
GEN hw/xen/trace.c
GEN hw/gpio/trace.c
GEN hw/riscv/trace.c
GEN migration/trace.c
GEN net/trace.c
GEN ui/trace.c
GEN hw/core/trace.c
GEN hw/display/trace.c
GEN qapi/trace.c
GEN qom/trace.c
GEN target/arm/trace.c
GEN target/hppa/trace.c
GEN target/i386/trace.c
GEN target/mips/trace.c
GEN target/ppc/trace.c
GEN target/riscv/trace.c
GEN target/s390x/trace.c
GEN target/sparc/trace.c
GEN util/trace.c
make[4]: Entering directory '/keystone/qemu/slirp'
make[4]: Nothing to be done for 'all'.
make[4]: Leaving directory '/keystone/qemu/slirp'
CHK version_gen.h
CC block.o
LINK qemu-nbd
LINK qemu-img
LINK qemu-io
LINK scsi/qemu-pr-helper
scsi/qemu-pr-helper.o: In function `do_pr_out':
/keystone/qemu/scsi/qemu-pr-helper.c:556: undefined reference to `fcntl64'
collect2: error: ld returned 1 exit status
/keystone/qemu/rules.mak:124: recipe for target 'scsi/qemu-pr-helper' failed
make[3]: *** [scsi/qemu-pr-helper] Error 1
make[3]: Leaving directory '/keystone/qemu'
CMakeFiles/qemu.dir/build.make:63: recipe for target '../qemu/riscv64-softmmu/qemu-system-riscv64' failed
make[2]: *** [../qemu/riscv64-softmmu/qemu-system-riscv64] Error 2
make[2]: Leaving directory '/keystone/build'
CMakeFiles/Makefile2:414: recipe for target 'CMakeFiles/qemu.dir/all' failed
make[1]: *** [CMakeFiles/qemu.dir/all] Error 2
make[1]: Leaving directory '/keystone/build'
Makefile:86: recipe for target 'all' failed
make: *** [all] Error 2

Bug 1:

The multiple definition in eyrie still exists.

sudo ./docker-run.sh ./make-sdk.sh

/keystone/riscv64/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
make: Entering directory '/keystone/sdk/lib'
make -C app
make[1]: Entering directory '/keystone/sdk/lib/app'
make[1]: Nothing to be done for 'all'.
make[1]: Leaving directory '/keystone/sdk/lib/app'
make -C host
make[1]: Entering directory '/keystone/sdk/lib/host'
make[1]: Nothing to be done for 'all'.
make[1]: Leaving directory '/keystone/sdk/lib/host'
make -C edge
make[1]: Entering directory '/keystone/sdk/lib/edge'
make[1]: Nothing to be done for 'all'.
make[1]: Leaving directory '/keystone/sdk/lib/edge'
make -C verifier
make[1]: Entering directory '/keystone/sdk/lib/verifier'
#riscv64-unknown-linux-gnu-g++ -std=gnu++11 test.cpp ../libkeystone-verifier.a -o test
make[1]: Leaving directory '/keystone/sdk/lib/verifier'
make: Leaving directory '/keystone/sdk/lib'
make: Entering directory '/keystone/sdk/rts/eyrie'
riscv64-unknown-linux-gnu-ld -static -nostdlib --no-dynamic-linker -z text -L/keystone/sdk/lib -o eyrie-rt entry.o fp_asm.o boot.o interrupt.o printf.o syscall.o string.o linux_wrap.o io_wrap.o fast_io_wrap.o rt_util.o mm.o env.o freemem.o paging.o performance.o /keystone/sdk/rts/eyrie/rpc/rpc.o /keystone/sdk/rts/eyrie/rpc/rpc_crypto.o /keystone/sdk/rts/eyrie/rpc/edge_wrapper.o /keystone/sdk/rts/eyrie/rpc/rpc_crypto_enc.o /keystone/sdk/lib/libkeystone-edge.a uaccess.o -T runtime.lds
riscv64-unknown-linux-gnu-ld: interrupt.o:(.bss+0x0): multiple definition of `runtime_va_start'; boot.o:(.bss+0x0): first defined here
riscv64-unknown-linux-gnu-ld: interrupt.o:(.bss+0x8): multiple definition of `kernel_offset'; boot.o:(.bss+0x8): first defined here
riscv64-unknown-linux-gnu-ld: interrupt.o:(.bss+0x10): multiple definition of `load_pa_start'; boot.o:(.bss+0x10): first defined here
riscv64-unknown-linux-gnu-ld: interrupt.o:(.bss+0x1000): multiple definition of `root_page_table'; boot.o:(.bss+0x1000): first defined here
riscv64-unknown-linux-gnu-ld: interrupt.o:(.bss+0x2000): multiple definition of `secondary_page_tables'; boot.o:(.bss+0x2000): first defined here
riscv64-unknown-linux-gnu-ld: interrupt.o:(.bss+0x202000): multiple definition of `page_tables_count'; boot.o:(.bss+0x202000): first defined here
riscv64-unknown-linux-gnu-ld: interrupt.o:(.bss+0x202008): multiple definition of `program_break'; boot.o:(.bss+0x202008): first defined here
riscv64-unknown-linux-gnu-ld: interrupt.o:(.bss+0x202010): multiple definition of `freemem_va_start'; boot.o:(.bss+0x202010): first defined here
riscv64-unknown-linux-gnu-ld: interrupt.o:(.bss+0x202018): multiple definition of `freemem_size'; boot.o:(.bss+0x202018): first defined here
riscv64-unknown-linux-gnu-ld: interrupt.o:(.bss+0x202020): multiple definition of `shared_buffer'; boot.o:(.bss+0x202020): first defined here
riscv64-unknown-linux-gnu-ld: interrupt.o:(.bss+0x202028): multiple definition of `shared_buffer_size'; boot.o:(.bss+0x202028): first defined here
riscv64-unknown-linux-gnu-ld: syscall.o:(.bss+0x0): multiple definition of `runtime_va_start'; boot.o:(.bss+0x0): first defined here
riscv64-unknown-linux-gnu-ld: syscall.o:(.bss+0x8): multiple definition of `kernel_offset'; boot.o:(.bss+0x8): first defined here
riscv64-unknown-linux-gnu-ld: syscall.o:(.bss+0x10): multiple definition of `load_pa_start'; boot.o:(.bss+0x10): first defined here
riscv64-unknown-linux-gnu-ld: syscall.o:(.bss+0x1000): multiple definition of `root_page_table'; boot.o:(.bss+0x1000): first defined here
riscv64-unknown-linux-gnu-ld: syscall.o:(.bss+0x2000): multiple definition of `secondary_page_tables'; boot.o:(.bss+0x2000): first defined here
riscv64-unknown-linux-gnu-ld: syscall.o:(.bss+0x202000): multiple definition of `page_tables_count'; boot.o:(.bss+0x202000): first defined here
riscv64-unknown-linux-gnu-ld: syscall.o:(.bss+0x202008): multiple definition of `program_break'; boot.o:(.bss+0x202008): first defined here
riscv64-unknown-linux-gnu-ld: syscall.o:(.bss+0x202010): multiple definition of `freemem_va_start'; boot.o:(.bss+0x202010): first defined here
riscv64-unknown-linux-gnu-ld: syscall.o:(.bss+0x202018): multiple definition of `freemem_size'; boot.o:(.bss+0x202018): first defined here
riscv64-unknown-linux-gnu-ld: syscall.o:(.bss+0x202020): multiple definition of `shared_buffer'; boot.o:(.bss+0x202020): first defined here
riscv64-unknown-linux-gnu-ld: syscall.o:(.bss+0x202028): multiple definition of `shared_buffer_size'; boot.o:(.bss+0x202028): first defined here
riscv64-unknown-linux-gnu-ld: linux_wrap.o:(.bss+0x0): multiple definition of `runtime_va_start'; boot.o:(.bss+0x0): first defined here
riscv64-unknown-linux-gnu-ld: linux_wrap.o:(.bss+0x8): multiple definition of `kernel_offset'; boot.o:(.bss+0x8): first defined here
riscv64-unknown-linux-gnu-ld: linux_wrap.o:(.bss+0x10): multiple definition of `load_pa_start'; boot.o:(.bss+0x10): first defined here
riscv64-unknown-linux-gnu-ld: linux_wrap.o:(.bss+0x1000): multiple definition of `root_page_table'; boot.o:(.bss+0x1000): first defined here
riscv64-unknown-linux-gnu-ld: linux_wrap.o:(.bss+0x2000): multiple definition of `secondary_page_tables'; boot.o:(.bss+0x2000): first defined here
riscv64-unknown-linux-gnu-ld: linux_wrap.o:(.bss+0x202000): multiple definition of `page_tables_count'; boot.o:(.bss+0x202000): first defined here
riscv64-unknown-linux-gnu-ld: linux_wrap.o:(.bss+0x202008): multiple definition of `program_break'; boot.o:(.bss+0x202008): first defined here
riscv64-unknown-linux-gnu-ld: linux_wrap.o:(.bss+0x202010): multiple definition of `freemem_va_start'; boot.o:(.bss+0x202010): first defined here
riscv64-unknown-linux-gnu-ld: linux_wrap.o:(.bss+0x202018): multiple definition of `freemem_size'; boot.o:(.bss+0x202018): first defined here
riscv64-unknown-linux-gnu-ld: linux_wrap.o:(.bss+0x202020): multiple definition of `shared_buffer'; boot.o:(.bss+0x202020): first defined here
riscv64-unknown-linux-gnu-ld: linux_wrap.o:(.bss+0x202028): multiple definition of `shared_buffer_size'; boot.o:(.bss+0x202028): first defined here
riscv64-unknown-linux-gnu-ld: io_wrap.o:(.bss+0x0): multiple definition of `runtime_va_start'; boot.o:(.bss+0x0): first defined here
riscv64-unknown-linux-gnu-ld: io_wrap.o:(.bss+0x8): multiple definition of `kernel_offset'; boot.o:(.bss+0x8): first defined here
riscv64-unknown-linux-gnu-ld: io_wrap.o:(.bss+0x10): multiple definition of `load_pa_start'; boot.o:(.bss+0x10): first defined here
riscv64-unknown-linux-gnu-ld: io_wrap.o:(.bss+0x1000): multiple definition of `root_page_table'; boot.o:(.bss+0x1000): first defined here
riscv64-unknown-linux-gnu-ld: io_wrap.o:(.bss+0x2000): multiple definition of `secondary_page_tables'; boot.o:(.bss+0x2000): first defined here
riscv64-unknown-linux-gnu-ld: io_wrap.o:(.bss+0x202000): multiple definition of `page_tables_count'; boot.o:(.bss+0x202000): first defined here
riscv64-unknown-linux-gnu-ld: io_wrap.o:(.bss+0x202008): multiple definition of `program_break'; boot.o:(.bss+0x202008): first defined here
riscv64-unknown-linux-gnu-ld: io_wrap.o:(.bss+0x202010): multiple definition of `freemem_va_start'; boot.o:(.bss+0x202010): first defined here
riscv64-unknown-linux-gnu-ld: io_wrap.o:(.bss+0x202018): multiple definition of `freemem_size'; boot.o:(.bss+0x202018): first defined here
riscv64-unknown-linux-gnu-ld: io_wrap.o:(.bss+0x202020): multiple definition of `shared_buffer'; boot.o:(.bss+0x202020): first defined here
riscv64-unknown-linux-gnu-ld: io_wrap.o:(.bss+0x202028): multiple definition of `shared_buffer_size'; boot.o:(.bss+0x202028): first defined here
riscv64-unknown-linux-gnu-ld: fast_io_wrap.o:(.bss+0x0): multiple definition of `runtime_va_start'; boot.o:(.bss+0x0): first defined here
riscv64-unknown-linux-gnu-ld: fast_io_wrap.o:(.bss+0x8): multiple definition of `kernel_offset'; boot.o:(.bss+0x8): first defined here
riscv64-unknown-linux-gnu-ld: fast_io_wrap.o:(.bss+0x10): multiple definition of `load_pa_start'; boot.o:(.bss+0x10): first defined here
riscv64-unknown-linux-gnu-ld: fast_io_wrap.o:(.bss+0x1000): multiple definition of `root_page_table'; boot.o:(.bss+0x1000): first defined here
riscv64-unknown-linux-gnu-ld: fast_io_wrap.o:(.bss+0x2000): multiple definition of `secondary_page_tables'; boot.o:(.bss+0x2000): first defined here
riscv64-unknown-linux-gnu-ld: fast_io_wrap.o:(.bss+0x202000): multiple definition of `page_tables_count'; boot.o:(.bss+0x202000): first defined here
riscv64-unknown-linux-gnu-ld: fast_io_wrap.o:(.bss+0x202008): multiple definition of `program_break'; boot.o:(.bss+0x202008): first defined here
riscv64-unknown-linux-gnu-ld: fast_io_wrap.o:(.bss+0x202010): multiple definition of `freemem_va_start'; boot.o:(.bss+0x202010): first defined here
riscv64-unknown-linux-gnu-ld: fast_io_wrap.o:(.bss+0x202018): multiple definition of `freemem_size'; boot.o:(.bss+0x202018): first defined here
riscv64-unknown-linux-gnu-ld: fast_io_wrap.o:(.bss+0x202020): multiple definition of `shared_buffer'; boot.o:(.bss+0x202020): first defined here
riscv64-unknown-linux-gnu-ld: fast_io_wrap.o:(.bss+0x202028): multiple definition of `shared_buffer_size'; boot.o:(.bss+0x202028): first defined here
riscv64-unknown-linux-gnu-ld: rt_util.o:(.bss+0x0): multiple definition of `runtime_va_start'; boot.o:(.bss+0x0): first defined here
riscv64-unknown-linux-gnu-ld: rt_util.o:(.bss+0x8): multiple definition of `kernel_offset'; boot.o:(.bss+0x8): first defined here
riscv64-unknown-linux-gnu-ld: rt_util.o:(.bss+0x10): multiple definition of `load_pa_start'; boot.o:(.bss+0x10): first defined here
riscv64-unknown-linux-gnu-ld: rt_util.o:(.bss+0x1000): multiple definition of `root_page_table'; boot.o:(.bss+0x1000): first defined here
riscv64-unknown-linux-gnu-ld: rt_util.o:(.bss+0x2000): multiple definition of `secondary_page_tables'; boot.o:(.bss+0x2000): first defined here
riscv64-unknown-linux-gnu-ld: rt_util.o:(.bss+0x202000): multiple definition of `page_tables_count'; boot.o:(.bss+0x202000): first defined here
riscv64-unknown-linux-gnu-ld: rt_util.o:(.bss+0x202008): multiple definition of `program_break'; boot.o:(.bss+0x202008): first defined here
riscv64-unknown-linux-gnu-ld: rt_util.o:(.bss+0x202010): multiple definition of `freemem_va_start'; boot.o:(.bss+0x202010): first defined here
riscv64-unknown-linux-gnu-ld: rt_util.o:(.bss+0x202018): multiple definition of `freemem_size'; boot.o:(.bss+0x202018): first defined here
riscv64-unknown-linux-gnu-ld: rt_util.o:(.bss+0x202020): multiple definition of `shared_buffer'; boot.o:(.bss+0x202020): first defined here
riscv64-unknown-linux-gnu-ld: rt_util.o:(.bss+0x202028): multiple definition of `shared_buffer_size'; boot.o:(.bss+0x202028): first defined here
riscv64-unknown-linux-gnu-ld: mm.o:(.bss+0x0): multiple definition of `runtime_va_start'; boot.o:(.bss+0x0): first defined here
riscv64-unknown-linux-gnu-ld: mm.o:(.bss+0x8): multiple definition of `kernel_offset'; boot.o:(.bss+0x8): first defined here
riscv64-unknown-linux-gnu-ld: mm.o:(.bss+0x10): multiple definition of `load_pa_start'; boot.o:(.bss+0x10): first defined here
riscv64-unknown-linux-gnu-ld: mm.o:(.bss+0x1000): multiple definition of `root_page_table'; boot.o:(.bss+0x1000): first defined here
riscv64-unknown-linux-gnu-ld: mm.o:(.bss+0x2000): multiple definition of `secondary_page_tables'; boot.o:(.bss+0x2000): first defined here
riscv64-unknown-linux-gnu-ld: mm.o:(.bss+0x202000): multiple definition of `page_tables_count'; boot.o:(.bss+0x202000): first defined here
riscv64-unknown-linux-gnu-ld: mm.o:(.bss+0x202008): multiple definition of `program_break'; boot.o:(.bss+0x202008): first defined here
riscv64-unknown-linux-gnu-ld: mm.o:(.bss+0x202010): multiple definition of `freemem_va_start'; boot.o:(.bss+0x202010): first defined here
riscv64-unknown-linux-gnu-ld: mm.o:(.bss+0x202018): multiple definition of `freemem_size'; boot.o:(.bss+0x202018): first defined here
riscv64-unknown-linux-gnu-ld: mm.o:(.bss+0x202020): multiple definition of `shared_buffer'; boot.o:(.bss+0x202020): first defined here
riscv64-unknown-linux-gnu-ld: mm.o:(.bss+0x202028): multiple definition of `shared_buffer_size'; boot.o:(.bss+0x202028): first defined here
riscv64-unknown-linux-gnu-ld: env.o:(.bss+0x0): multiple definition of `runtime_va_start'; boot.o:(.bss+0x0): first defined here
riscv64-unknown-linux-gnu-ld: env.o:(.bss+0x8): multiple definition of `kernel_offset'; boot.o:(.bss+0x8): first defined here
riscv64-unknown-linux-gnu-ld: env.o:(.bss+0x10): multiple definition of `load_pa_start'; boot.o:(.bss+0x10): first defined here
riscv64-unknown-linux-gnu-ld: env.o:(.bss+0x1000): multiple definition of `root_page_table'; boot.o:(.bss+0x1000): first defined here
riscv64-unknown-linux-gnu-ld: env.o:(.bss+0x2000): multiple definition of `secondary_page_tables'; boot.o:(.bss+0x2000): first defined here
riscv64-unknown-linux-gnu-ld: env.o:(.bss+0x202000): multiple definition of `page_tables_count'; boot.o:(.bss+0x202000): first defined here
riscv64-unknown-linux-gnu-ld: env.o:(.bss+0x202008): multiple definition of `program_break'; boot.o:(.bss+0x202008): first defined here
riscv64-unknown-linux-gnu-ld: env.o:(.bss+0x202010): multiple definition of `freemem_va_start'; boot.o:(.bss+0x202010): first defined here
riscv64-unknown-linux-gnu-ld: env.o:(.bss+0x202018): multiple definition of `freemem_size'; boot.o:(.bss+0x202018): first defined here
riscv64-unknown-linux-gnu-ld: env.o:(.bss+0x202020): multiple definition of `shared_buffer'; boot.o:(.bss+0x202020): first defined here
riscv64-unknown-linux-gnu-ld: env.o:(.bss+0x202028): multiple definition of `shared_buffer_size'; boot.o:(.bss+0x202028): first defined here
riscv64-unknown-linux-gnu-ld: freemem.o:(.bss+0x0): multiple definition of `runtime_va_start'; boot.o:(.bss+0x0): first defined here
riscv64-unknown-linux-gnu-ld: freemem.o:(.bss+0x8): multiple definition of `kernel_offset'; boot.o:(.bss+0x8): first defined here
riscv64-unknown-linux-gnu-ld: freemem.o:(.bss+0x10): multiple definition of `load_pa_start'; boot.o:(.bss+0x10): first defined here
riscv64-unknown-linux-gnu-ld: freemem.o:(.bss+0x1000): multiple definition of `root_page_table'; boot.o:(.bss+0x1000): first defined here
riscv64-unknown-linux-gnu-ld: freemem.o:(.bss+0x2000): multiple definition of `secondary_page_tables'; boot.o:(.bss+0x2000): first defined here
riscv64-unknown-linux-gnu-ld: freemem.o:(.bss+0x202000): multiple definition of `page_tables_count'; boot.o:(.bss+0x202000): first defined here
riscv64-unknown-linux-gnu-ld: freemem.o:(.bss+0x202008): multiple definition of `program_break'; boot.o:(.bss+0x202008): first defined here
riscv64-unknown-linux-gnu-ld: freemem.o:(.bss+0x202010): multiple definition of `freemem_va_start'; boot.o:(.bss+0x202010): first defined here
riscv64-unknown-linux-gnu-ld: freemem.o:(.bss+0x202018): multiple definition of `freemem_size'; boot.o:(.bss+0x202018): first defined here
riscv64-unknown-linux-gnu-ld: freemem.o:(.bss+0x202020): multiple definition of `shared_buffer'; boot.o:(.bss+0x202020): first defined here
riscv64-unknown-linux-gnu-ld: freemem.o:(.bss+0x202028): multiple definition of `shared_buffer_size'; boot.o:(.bss+0x202028): first defined here
riscv64-unknown-linux-gnu-ld: /keystone/sdk/rts/eyrie/rpc/edge_wrapper.o:(.bss+0x0): multiple definition of `runtime_va_start'; boot.o:(.bss+0x0): first defined here
riscv64-unknown-linux-gnu-ld: /keystone/sdk/rts/eyrie/rpc/edge_wrapper.o:(.bss+0x8): multiple definition of `kernel_offset'; boot.o:(.bss+0x8): first defined here
riscv64-unknown-linux-gnu-ld: /keystone/sdk/rts/eyrie/rpc/edge_wrapper.o:(.bss+0x10): multiple definition of `load_pa_start'; boot.o:(.bss+0x10): first defined here
riscv64-unknown-linux-gnu-ld: /keystone/sdk/rts/eyrie/rpc/edge_wrapper.o:(.bss+0x1000): multiple definition of `root_page_table'; boot.o:(.bss+0x1000): first defined here
riscv64-unknown-linux-gnu-ld: /keystone/sdk/rts/eyrie/rpc/edge_wrapper.o:(.bss+0x2000): multiple definition of `secondary_page_tables'; boot.o:(.bss+0x2000): first defined here
riscv64-unknown-linux-gnu-ld: /keystone/sdk/rts/eyrie/rpc/edge_wrapper.o:(.bss+0x202000): multiple definition of `page_tables_count'; boot.o:(.bss+0x202000): first defined here
riscv64-unknown-linux-gnu-ld: /keystone/sdk/rts/eyrie/rpc/edge_wrapper.o:(.bss+0x202008): multiple definition of `program_break'; boot.o:(.bss+0x202008): first defined here
riscv64-unknown-linux-gnu-ld: /keystone/sdk/rts/eyrie/rpc/edge_wrapper.o:(.bss+0x202010): multiple definition of `freemem_va_start'; boot.o:(.bss+0x202010): first defined here
riscv64-unknown-linux-gnu-ld: /keystone/sdk/rts/eyrie/rpc/edge_wrapper.o:(.bss+0x202018): multiple definition of `freemem_size'; boot.o:(.bss+0x202018): first defined here
riscv64-unknown-linux-gnu-ld: /keystone/sdk/rts/eyrie/rpc/edge_wrapper.o:(.bss+0x202020): multiple definition of `shared_buffer'; boot.o:(.bss+0x202020): first defined here
riscv64-unknown-linux-gnu-ld: /keystone/sdk/rts/eyrie/rpc/edge_wrapper.o:(.bss+0x202028): multiple definition of `shared_buffer_size'; boot.o:(.bss+0x202028): first defined here
Makefile:47: recipe for target 'eyrie-rt' failed
make: *** [eyrie-rt] Error 1
make: Leaving directory '/keystone/sdk/rts/eyrie'

Here is my toolchain info:

$ riscv64-unknown-linux-gnu-ld -v

GNU ld (GNU Binutils) 2.35

$ riscv64-unknown-linux-gnu-gcc -v

Using built-in specs.
COLLECT_GCC=riscv64-unknown-linux-gnu-gcc
COLLECT_LTO_WRAPPER=/home/**/keystone/elasticlave/riscv64/bin/../libexec/gcc/riscv64-unknown-linux-gnu/10.2.0/lto-wrapper
Target: riscv64-unknown-linux-gnu
Configured with: /home/ubuntu/keystone/riscv-gnu-toolchain/riscv-gcc/configure --target=riscv64-unknown-linux-gnu --prefix=/home/ubuntu/keystone/riscv64 --with-sysroot=/home/ubuntu/keystone/riscv64/sysroot --with-system-zlib --enable-shared --enable-tls --enable-languages=c,c++,fortran --disable-libmudflap --disable-libssp --disable-libquadmath --disable-libsanitizer --disable-nls --disable-bootstrap --src=.././riscv-gcc --disable-multilib --with-abi=lp64d --with-arch=rv64gc --with-tune=rocket 'CFLAGS_FOR_TARGET=-O2 -mcmodel=medlow' 'CXXFLAGS_FOR_TARGET=-O2 -mcmodel=medlow'
Thread model: posix
Supported LTO compression algorithms: zlib
gcc version 10.2.0 (GCC)

[jasonyu1996]

The problem should be the same as reported in that issue of Keystone. You are using a toolchain that is too new. I will probably update the code so it can be built with newer toolchains later, but for now, you should already have a pre-built toolchain available under keystone/riscv (which was downloaded by fast-setup.sh). Could you change the paths in source.sh back (I supposed you've changed them to your own toolchain) and retry?

[bows7ring]

PROBLEM SOLVED

Conclusion：

• 0. Ubuntu 20.04 is recommended.
• 1. Add make install in the end of make-sodium.sh.

I wrote a long bug report about my recent building problems, but fortunately, I succeed in the end. I decide to paste it here all the same, maybe it will help others.

---

Hi, Jason. Thanks for pointing it out. I did change the path in source.sh, and forgot about it.

Then I ran into some bugs which are probably caused by Ubuntu 18.04 ( supports up to Glibc-2.27). However, I had some trouble installing and re-linking Glibc-2.28 on Ubuntu 18.04 ( run into segmental fault every time). So I rebuilt my riscv environment on Ubuntu 20.04 ( I guess this is the OS you are using? ).

• Bug 0:

  building QEMU on 18.04:

make[4]: Entering directory '/keystone/qemu/slirp'
make[4]: Nothing to be done for 'all'.
make[4]: Leaving directory '/keystone/qemu/slirp'
CHK version_gen.h
CC block.o
LINK qemu-nbd
LINK qemu-img
LINK qemu-io
LINK scsi/qemu-pr-helper
scsi/qemu-pr-helper.o: In function do_pr_out': /keystone/qemu/scsi/qemu-pr-helper.c:556: undefined reference to fcntl64'
collect2: error: ld returned 1 exit status
/keystone/qemu/rules.mak:124: recipe for target 'scsi/qemu-pr-helper' failed
make[3]: *** [scsi/qemu-pr-helper] Error 1
make[3]: Leaving directory '/keystone/qemu'
CMakeFiles/qemu.dir/build.make:63: recipe for target '../qemu/riscv64-softmmu/qemu-system-riscv64' failed
make[2]: *** [../qemu/riscv64-softmmu/qemu-system-riscv64] Error 2
make[2]: Leaving directory '/keystone/build'
CMakeFiles/Makefile2:414: recipe for target 'CMakeFiles/qemu.dir/all' failed
make[1]: *** [CMakeFiles/qemu.dir/all] Error 2
make[1]: Leaving directory '/keystone/build'
Makefile:86: recipe for target 'all' failed
make: *** [all] Error 2

• Bug 1:

  Building image on 18.04:

make[3]: Entering directory '/keystone/build'
[ 81%] Generating image
make -s -C /keystone/buildroot RISCV=/keystone/riscv/ PATH=/keystone/riscv//bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin O=/keystone/build/buildroot.build
make[4]: Entering directory '/keystone/buildroot'
>>> Finalizing target directory
Warning: host file "./share/man/man8/fsck.8" is touched by more than one package: [u'host-util-linux', u'host-e2fsprogs']
Warning: host file "./sbin/fsck" is touched by more than one package: [u'host-util-linux', u'host-e2fsprogs']
grep: ./.config: No such file or directory
grep: ./.config: No such file or directory
find: '/keystone/build/buildroot.build/target/usr/libexec/': No such file or directory
>>> Sanitizing RPATH in target tree
>>> Copying overlay /keystone/build/overlay
>>> Generating root filesystems common tables
>>> Generating filesystem image rootfs.ext2
rootdir=/keystone/build/buildroot.build/build/buildroot-fs/ext2/target
table='/keystone/build/buildroot.build/build/buildroot-fs/full_devices_table.txt'
/keystone/build/buildroot.build/host/sbin/mkfs.ext2: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.28' not found (required by /keystone/build/buildroot.build/host/lib/libext2fs.so.2)
*** Maybe you need to increase the filesystem size (BR2_TARGET_ROOTFS_EXT2_SIZE)
fs/ext2/ext2.mk:46: recipe for target '/keystone/build/buildroot.build/images/rootfs.ext2' failed
make[4]: *** [/keystone/build/buildroot.build/images/rootfs.ext2] Error 1
make[4]: Leaving directory '/keystone/buildroot'
CMakeFiles/image.dir/build.make:62: recipe for target 'CMakeFiles/image' failed
make[3]: *** [CMakeFiles/image] Error 2
make[3]: Leaving directory '/keystone/build'
CMakeFiles/Makefile2:104: recipe for target 'CMakeFiles/image.dir/all' failed
make[2]: *** [CMakeFiles/image.dir/all] Error 2
make[2]: Leaving directory '/keystone/build'
CMakeFiles/Makefile2:111: recipe for target 'CMakeFiles/image.dir/rule' failed
make[1]: *** [CMakeFiles/image.dir/rule] Error 2
make[1]: Leaving directory '/keystone/build'
Makefile:134: recipe for target 'image' failed
make: *** [image] Error 2

I restarted building on Ubuntu 20.04. This time, previous steps are bug-free.

Almost done. Finally, it failed during tests building. I tried other EXTRA_TEST items in mkconfig/mk, they failed with same log.

• Bug 2:

Looks like there is something wrong with the path of Libsodium.a...

Well, I got it..

:-) You forgot to make install in make-sodium.sh.

Here is mine:

#!/bin/bash

CURDIR=$(dirname $(readlink -f $0))

cd $CURDIR

source source.sh
cd libsodium
./autogen.sh
./configure --host=riscv64-unknown-linux-gnu --disable-ssp --disable-asm --without-pthreads
make clean
make -j8
make install

[jasonyu1996]

@bows7ring Thank you so much for the detailed report! I will try reproducing the problems you encountered and see what I can do to improve README and the build scripts