Add APK package building capabilities to OpenWrt

[aparcar]

APK is the main package manager of Alpine Linux. This is just a proof of concept to double create package for OPKG and APK in parrallel.

(Personal) motivation is a long time replacement of OPKG with APK and join forces with Alpine Linux to create a great package manager for embedded devices rather than maintaining OPKG.

This is a follow up of the accidentally (mystically) closed #3811

Open Issues:

• Implement something like CONFIG_CLEAN_IPKG @dangowrt
• add ABIVersion to both package metadata and installdb @dangowrt
• Ability to distinguish between actively and passively (dependency) installed packages @dangowrt
  APK got /etc/apk/world which unfortunately is not quite like gentoo emerge's @world, but rather just stores a list of explicitely installed packages (while gentoo's emerge @world store a record of the manually selected dependency atoms, ie. it would remember if the user asked for an explicit version or if a package had been selected as a provider for something the user requested). However, using that gets us the same data we previously had with opkg: if a package is user-selected or not.
• rpcd-sys @dangowrt
  • package list (add / remove / update remote list can be added later)
• fix apk version of itself output on cmdline (always 3.0.0pre0, should be 3.0.0pre20240205)
  • fixed apk-tools 3.0.0_pre20240324, compiled for aarch64.
• version definition handling opkg vs apk @aparcar
• lua: add HOST_FPIC for host builds #4907
• Alternatives are not handled (upstream problem).
  • via a post-install and pre-remove script.
• Don't compress help text
  • Allows to drop lua-zlib dependencies
  • via https://gitlab.alpinelinux.org/alpine/apk-tools/-/merge_requests/91
• install to ./staging_dir/host/bin no ./staging_dir/hostpkg/bin
• handle multiple lines of maintainers
• signing keys are not automatically generated
• Fix ImageBuilder @aparcar

Optional:

• Replace libfetch with uclient/curl/wget

[ptpt52]

cool to add this util

[ptpt52]

on my case, I have to use apk command to install packages like openjdk11-jdk, and it depends on busybox, so busybox and musl pkgs would be installed via apk and overwrite the system busybox/musl, and it cause system soft broken.

[ptpt52]

so how to mark busybox.apk and/or musl.apk installed and avoid apk auto install them?

[aparcar]

@ptpt52 these are issues for packages.git since there apk is added as a on device package. This PR is for build integration.

Sure, if Alpine is up for it I'm happy to merge package building but I don't see this to happen any time soon.

[KOLANICH]

I am a bit sorry to ask here, but I wonder if it makes sense to improve some aspect of opkg.

Opkg stores information about installed packages as plain text in files in /usr/lib/opkg/info (dpkg does the similar thing). Space analyser has showed that this dir is one of the largest in an image. The paths are duplicated, i.e. if 2 files have the same prefix in a path (the same parent dir), then it is duplicated. The search is also inefficient, since to search a package manager has to crawl all the files.

IDK if apk is immune to this problem, it seems for search it stores the info in form of a hash of a path, but for efficient proper uninstallation it must store original paths somehow.

I have encountered a similar problem in ubuntu (I needed a fast search of a package by a path). I have solved it by creating a library for storing such information in an own format and a framework for converting the info from various distros into its format and a backend for dpkg.

The format consists of an SQLite database, which has 2 tables. One stores strings, and another one (in fact, 2) stores a prefix tree (using datrie library) of primary keys of these strings. The prefix tree is then compressed.

I wonder if it makes sense for OpenWRT to utilise a similar format in own package managers.

[dangowrt]

--help requires libluaapk.so AFAIK

No. '--help' requires Lua on the build-host to generate the compressed help output. APK doesn't need to be compiled with Lua just for that.

[aparcar]

Okay great, I will fix that.

[aparcar]

I am a bit sorry to ask here, but I wonder if it makes sense to improve some aspect of opkg.

Opkg stores information about installed packages as plain text in files in /usr/lib/opkg/info (dpkg does the similar thing). Space analyser has showed that this dir is one of the largest in an image. The paths are duplicated, i.e. if 2 files have the same prefix in a path (the same parent dir), then it is duplicated. The search is also inefficient, since to search a package manager has to crawl all the files.

IDK if apk is immune to this problem, it seems for search it stores the info in form of a hash of a path, but for efficient proper uninstallation it must store original paths somehow.

I have encountered a similar problem in ubuntu (I needed a fast search of a package by a path). I have solved it by creating a library for storing such information in an own format and a framework for converting the info from various distros into its format and a backend for dpkg.

The format consists of an SQLite database, which has 2 tables. One stores strings, and another one (in fact, 2) stores a prefix tree (using datrie library) of primary keys of these strings. The prefix tree is then compressed.

I wonder if it makes sense for OpenWRT to utilise a similar format in own package managers.

I think this is indeed the wrong place to suggest changes to opkg.

APK uses a flat buffer like database to store for packages, I'll ask the dev on compression is a possibility.

Proper file handling of (un)installed packages is part of both opkg and apk, and so is dependency tracking.

Motivation for apk is that we no longer maintain our own fork of a package manager but share tooling with similar minded projects (alpine).

[kaniini]

I'm not Timo, but compression of the database should be possible in apk3, as it would use the same access methods as the packaging side. So the appropriate compression flags would just need to be added to the file header, basically.