prepare_host: Installing packages with the install_pkg_deb function

[The-going]

Description

Installing packages using the install_pkg_deb function for the operating system of the build machine.

• Test build image

[rpardini]

Looks good to me!

[rpardini]

Nitpick: I'd use bash arrays (declare -a) for hostdeps.
array syntax allows for multiline without needing \ so easier to reformat later.

[The-going]

Installing packages using the install_pkg_deb function, which requires a list separated by a space, tab, or a newline character.

Verification code:

#!/bin/bash

	      hostdeps="acl aptly aria2 bc binfmt-support bison btrfs-progs       \
	build-essential  ca-certificates ccache cpio cryptsetup curl              \
	debian-archive-keyring debian-keyring debootstrap device-tree-compiler    \
	dialog dirmngr dosfstools dwarves f2fs-tools fakeroot flex gawk           \
	gcc-arm-linux-gnueabihf gdisk gnupg1 gpg imagemagick jq kmod libbison-dev \
	libc6-dev-armhf-cross libelf-dev libfdt-dev libfile-fcntllock-perl        \
	libfl-dev liblz4-tool libncurses-dev libpython2.7-dev libssl-dev          \
	libusb-1.0-0-dev linux-base locales lzop ncurses-base ncurses-term        \
	nfs-kernel-server ntpdate p7zip-full parted patchutils pigz pixz          \
	pkg-config pv python3-dev python3-distutils qemu-user-static rsync swig   \
	systemd-container u-boot-tools udev unzip uuid-dev wget whiptail zip      \
	zlib1g-dev lib32ncurses-dev"

	list_deps="
	acl aptly aria2 bc binfmt-support bison btrfs-progs
	build-essential  ca-certificates ccache cpio cryptsetup curl
	debian-archive-keyring debian-keyring debootstrap device-tree-compiler
	dialog dirmngr dosfstools dwarves f2fs-tools fakeroot flex gawk
	gcc-arm-linux-gnueabihf gdisk gnupg1 gpg imagemagick jq kmod libbison-dev
	libc6-dev-armhf-cross libelf-dev libfdt-dev libfile-fcntllock-perl
	libfl-dev liblz4-tool libncurses-dev libpython2.7-dev libssl-dev
	libusb-1.0-0-dev linux-base locales lzop ncurses-base ncurses-term
	nfs-kernel-server ntpdate p7zip-full parted patchutils pigz pixz
	pkg-config pv python3-dev python3-distutils qemu-user-static rsync swig
	systemd-container u-boot-tools udev unzip uuid-dev wget whiptail zip
	zlib1g-dev lib32ncurses-dev
	"


test_print ()
{
	echo "$*"
	echo ""
	echo -e "$*"
	echo -e "\n=======\n"
}

test_print $hostdeps

test_print $list_deps

In this case, I want to show that if you want to write a long line,
then it should be separated by the "\" character at the 79th position of the line.
I.e., the line in script should not be longer than 80.

It is very difficult to read and correct the text of the code when you look at it through
a small screen panel in a console text editor. For example, directly on a working board.

[rpardini]

Sorry. That's definitely not what I meant. Sorry.
I meant:

declare -a list_of_things=("one-thing" "another-thing"
                           "this-does-not-need-backslashes")
[[ a == a ]] && list_of_things+=("add_one_thing_later")
echo "${list_of_things[@]}" # expand the array correctly into a string, each thing a parameter
echo "${list_of_things[*]}" # expand into a single parameter

[The-going]

Do I understand correctly that you need a list of listed packages designed as an array but not as a string?
Is it easier for you to work with an array?

[The-going]

It is most important to find out the opinion of users about the log that the install_pkg_deb function creates.

[igorpecovnik]

opinion of users about the log that the install_pkg_deb function creates.

Do we need to have in and out list? Perhaps a list and exposed problematic one?

Bad package name: lib32ncurses-dev (on aarch64)

[The-going]

Do we need to have in and out list? Perhaps a list and exposed problematic one?

Initially, I made this function for debugging build dependencies in projects that support older versions of debian. These packages cannot be built in the new ubuntu OS unless the build dependencies are fixed. It is for this reason that it is important for me to see this:

Incoming list:
pkg-config                     u-boot-tools                   binutils-arm-none-eabi         ruby                          
libusb-1.0-0-dev               libfdt-dev                     zlib1g-dev                                                   


status after installation:
binutils-arm-none-eabi      2.34-4ubuntu1+13ubuntu1 [ install ok installed ]
libfdt-dev                  1.5.1-1                 [ install ok installed ]
libusb-1.0-0-dev:arm64      2:1.0.23-2build1        [ install ok installed ]
pkg-config                  0.29.1-0ubuntu4         [ install ok installed ]
ruby                        1:2.7+1                 [ install ok installed ]
u-boot-tools                2021.01+dfsg-3ubuntu0~2 [ install ok installed ]
zlib1g-dev:arm64            1:1.2.11.dfsg-2ubuntu1. [ install ok installed ]

But we can make an additional parameter for debugging -V=1 or --verbose
and output by default only erroneous package names.
And with the parameter to make a complete output.

[The-going]

Bad package name: lib32ncurses-dev (on aarch64)

Thanks. I'll fix it. But could this mean that not everything can be assembled for armhf on aarch64?

[igorpecovnik]

Thanks. I'll fix it. But could this mean that not everything can be assembled for armhf on aarch64?

Don't know for sure.

Probably unrelated - exotic legacy (u-)boot(s) might come with x86 binary tools for assembly or won't compile with distro compiler. So yes, not everything can be assembled on aarch64, but its not critical since those are going towards EOL.

[igorpecovnik]

Looks good. If there is nothing more to change / add we are ready to merge.

[The-going]

This stage, yes, can be merged. I'm still testing the next stage.

[rpardini]

Tag for manual merge into logging branch

[igorpecovnik]

@The-going host apt update should not run each time script is called. This has to be solved somehow or thrown under switch.

[ .... ] Installing build dependencies
Hit:1 http://si.archive.ubuntu.com/ubuntu focal InRelease
Hit:2 http://si.archive.ubuntu.com/ubuntu focal-updates InRelease
Hit:3 http://si.archive.ubuntu.com/ubuntu focal-backports InRelease
Hit:4 https://download.docker.com/linux/ubuntu focal InRelease
Hit:5 http://dl.google.com/linux/chrome/deb stable InRelease
Hit:6 http://packages.microsoft.com/repos/code stable InRelease
Hit:7 https://repo.skype.com/deb stable InRelease
Get:8 http://security.ubuntu.com/ubuntu focal-security InRelease [114 kB]
Hit:9 https://imola.armbian.com/beta focal InRelease
Fetched 114 kB in 1s (178 kB/s)
Reading package lists...

[The-going]

host apt update should not run each time script is called. This has to be solved somehow or thrown under switch.

@igorpecovnik Before that it was:

	if [[ ${#deps[@]} -gt 0 ]]; then
		display_alert "Installing build dependencies"
		# don't prompt for apt cacher selection
		sudo echo "apt-cacher-ng    apt-cacher-ng/tunnelenable      boolean false" | sudo debconf-set-selections
		apt-get -q update
		apt-get -y upgrade
		apt-get -q -y --no-install-recommends install -o Dpkg::Options::='--force-confold' "${deps[@]}" | tee -a "${DEST}"/${LOG_SUBPATH}/hostdeps.log
		update-ccache-symlinks
	fi

If there is no target to install, then nothing happens.
I'll make this an exception.