Building for simulation on Ubuntu 20.04

[mefistotelis]

(This is not really an issue per se, more of experience sharing.)

Based on the instruction:
https://optee.readthedocs.io/en/latest/building/devices/qemu.html#qemu-v7

I was able to prepare the simulation with no major issues. I was actually impressed how smooth it went.

The packages which were needed:

sudo apt install build-essential git bison flex libssl-dev libglib2.0-dev libpixman-1-dev
sudo apt install python3-pip python3-pycryptodome python3-pyelftools
sudo pip3 install ninja

The issues I faced - let me know if you think I should create separate tickets for them:

1. U-boot does not get configured

The make run command does not seem to select board config for u-boot. I had to do it manually; not sure what the intended config was, but I used:

cd u-boot
make qemu_arm_defconfig ARCH=arm CROSS_COMPILE=arm-none-eabi-

.. and the build worked after that.

2. All python3 except one python

During build, whenever python is needed, python3 binary is used. Except that one place which for some reason calls python.
Some distros no longer have python symlink at all, as python3 and the obsolete python2 are now used everywhere.

With this patch, everything builds with python3 access only:

diff --git a/common.mk b/common.mk
index dad0539..4e67de0 100644
--- a/common.mk
+++ b/common.mk
@@ -292,3 +292,3 @@ buildroot: optee-os
        @$(call append-br2-vars,../out-br/extra.conf)
-       @(cd .. && python build/br-ext/scripts/make_def_config.py \
+       @(cd .. && python3 build/br-ext/scripts/make_def_config.py \
                --br buildroot --out out-br --br-ext build/br-ext \

(I can make a PR if it's suitable)

3. Assuming my architecture

If the system is i386, the make toolchains command will still download 64-bit prebuilds.
(for this I already created a separate issue)

4. Successful build confusion

The last messages when make all ended with success were:

  GIT     ui/keycodemapdb tests/fp/berkeley-testfloat-3 tests/fp/berkeley-softfloat-3 meson dtc capstone slirp
[1/49] Generating qemu-version.h with a meson_exe.py custom command
fatal: No names found, cannot describe anything.
[2/33] Generating QAPI test (include) with a custom command
make[2]: Leaving directory '/op-tee-simulation/qemu/build'
make[1]: Leaving directory '/op-tee-simulation/qemu'
make -C /op-tee-simulation/build/../soc_term
make[1]: Entering directory '/op-tee-simulation/soc_term'
cc     soc_term.c   -o soc_term
make[1]: Leaving directory '/op-tee-simulation/soc_term'
$

As you can imagine, fatal: No names found, cannot describe anything. does not cause optimism in someone trying to look into logs to figure out whether everything is ok.

[jbech-linaro]

The make run command does not seem to select board config for u-boot. I had to do it manually; not sure what the intended config was, but I used:

You got this on a brand new setup? I'm a bit surprised, because in our CI we're running the commands like described in the readme (CI starts from scratch).

[mefistotelis]

You got this on a brand new setup?

Yes. The Ubuntu installation I used was fresh, at least in development area.

I'm a bit surprised, because in our CI we're running the commands like described in the readme (CI starts from scratch).

Looking into the code, I see 'scripts/kconfig/merge_config.sh' is supposed to take care of the configuration. It is used in place of executing make NAME_defconfig. This evidently didn't worked for me.

I also see you use hyphen in variable names for make, ie U-BOOT_DEFCONFIG_FILES. This is allowed, but with remarks. From the manual:

A variable name may be any sequence of characters not containing ‘:’, ‘#’, ‘=’, or whitespace. However, variable names containing characters other than letters, numbers, and underscores should be considered carefully, as in some shells they cannot be passed through the environment to a sub-make (see Communicating Variables to a Sub-make).

At the moment, I can't tell why merge_config.sh didn't worked. I didn't stored logs, unfortunately.

[jbech-linaro]

At the moment, I can't tell why merge_config.sh didn't worked. I didn't stored logs, unfortunately.

I need help to understand whether this is an issue or not? If there is anything not working, then it should be fixed. But since we're running this in CI and haven't seen the problem, we cannot replicate it. So if you're facing an issue, I'd kindly ask you to provide a fix for it if possible.

[mefistotelis]

I need help to understand whether this is an issue or not?

No, there is no issue. I wasn't able to replicate the problem on a second op-tee tree on that machine. So the issue I had must have been caused by not having some of the necessary tools, which I installed later as the compilation progressed further.

[jbech-linaro]

Ok, in that case, then I'll close this ticket. Thanks for raising the issue, sometimes there are issues that needs a fix even though it cannot be replicated on our end (lots of bits and pieces needs to play nicely together).