Add APK package building capabilities to OpenWrt

[aparcar]

APK is the main package manager of Alpine Linux. This is just a proof of concept to double create package for OPKG and APK in parrallel.

(Personal) motivation is a long time replacement of OPKG with APK and join forces with Alpine Linux to create a great package manager for embedded devices rather than maintaining OPKG.

This is a follow up of the accidentally (mystically) closed #3811

Open Issues:

• handle multiple lines of maintainers
• signing keys are not automatically generated
• Fix ImageBuilder
• Implement something like CONFIG_CLEAN_IPKG
• Add --strip-abi feature to upstream APK
  • store ABI version in metadata: https://gitlab.alpinelinux.org/alpine/apk-tools/-/merge_requests/92
• Replace libfetch with uclient
• lua: add HOST_FPIC for host builds #4907
• Alternatives are not handled (upstream problem).
  • via a post-install and pre-remove script.
• Don't compress help text
  • Allows to drop lua-zlib dependencies
  • via https://gitlab.alpinelinux.org/alpine/apk-tools/-/merge_requests/91
• install to ./staging_dir/host/bin no ./staging_dir/hostpkg/bin
• more

[ptpt52]

cool to add this util

[ptpt52]

on my case, I have to use apk command to install packages like openjdk11-jdk, and it depends on busybox, so busybox and musl pkgs would be installed via apk and overwrite the system busybox/musl, and it cause system soft broken.

[ptpt52]

so how to mark busybox.apk and/or musl.apk installed and avoid apk auto install them?

[aparcar]

@ptpt52 these are issues for packages.git since there apk is added as a on device package. This PR is for build integration.

Sure, if Alpine is up for it I'm happy to merge package building but I don't see this to happen any time soon.

[KOLANICH]

I am a bit sorry to ask here, but I wonder if it makes sense to improve some aspect of opkg.

Opkg stores information about installed packages as plain text in files in /usr/lib/opkg/info (dpkg does the similar thing). Space analyser has showed that this dir is one of the largest in an image. The paths are duplicated, i.e. if 2 files have the same prefix in a path (the same parent dir), then it is duplicated. The search is also inefficient, since to search a package manager has to crawl all the files.

IDK if apk is immune to this problem, it seems for search it stores the info in form of a hash of a path, but for efficient proper uninstallation it must store original paths somehow.

I have encountered a similar problem in ubuntu (I needed a fast search of a package by a path). I have solved it by creating a library for storing such information in an own format and a framework for converting the info from various distros into its format and a backend for dpkg.

The format consists of an SQLite database, which has 2 tables. One stores strings, and another one (in fact, 2) stores a prefix tree (using datrie library) of primary keys of these strings. The prefix tree is then compressed.

I wonder if it makes sense for OpenWRT to utilise a similar format in own package managers.

[neheb]

why get rid of -Dlua=disabled ?

[aparcar]

To have --help

[neheb]

--help requires libluaapk.so AFAIK

[kaniini]

Lua is only required to build the help database. It can also be pre-built.

[aparcar]

Thak you, updated and it works.

[dangowrt]

--help requires libluaapk.so AFAIK

No. '--help' requires Lua on the build-host to generate the compressed help output. APK doesn't need to be compiled with Lua just for that.

[aparcar]

Okay great, I will fix that.

[aparcar]

I am a bit sorry to ask here, but I wonder if it makes sense to improve some aspect of opkg.

Opkg stores information about installed packages as plain text in files in /usr/lib/opkg/info (dpkg does the similar thing). Space analyser has showed that this dir is one of the largest in an image. The paths are duplicated, i.e. if 2 files have the same prefix in a path (the same parent dir), then it is duplicated. The search is also inefficient, since to search a package manager has to crawl all the files.

IDK if apk is immune to this problem, it seems for search it stores the info in form of a hash of a path, but for efficient proper uninstallation it must store original paths somehow.

I have encountered a similar problem in ubuntu (I needed a fast search of a package by a path). I have solved it by creating a library for storing such information in an own format and a framework for converting the info from various distros into its format and a backend for dpkg.

The format consists of an SQLite database, which has 2 tables. One stores strings, and another one (in fact, 2) stores a prefix tree (using datrie library) of primary keys of these strings. The prefix tree is then compressed.

I wonder if it makes sense for OpenWRT to utilise a similar format in own package managers.

I think this is indeed the wrong place to suggest changes to opkg.

APK uses a flat buffer like database to store for packages, I'll ask the dev on compression is a possibility.

Proper file handling of (un)installed packages is part of both opkg and apk, and so is dependency tracking.

Motivation for apk is that we no longer maintain our own fork of a package manager but share tooling with similar minded projects (alpine).

[kaniini]

I'm not Timo, but compression of the database should be possible in apk3, as it would use the same access methods as the packaging side. So the appropriate compression flags would just need to be added to the file header, basically.

[aparcar]

Just a heads up, this is still alive. Next major tasks are to get rid of libfetch an the hard dependency on OpenSSL. Once that's done I'll rework this massive PR to work side by side to OPKG so people can set a config option like CONFIG_APK compiling everything for APK.