dpkg-ng

dpkg-ng is a re-implementation of Debian dpkg completely in Python. It has two primary goals:

Modularize the dpkg workflow
Make a version of dpkg aware of multiple version installations
- For multi-arch installation (i686 on x86-64); and
- For multi-version (Nix-like) installation on Debian systems

Multi-version Installation

Primarily, dpkg-ng wants to function as a drop-in replacement for dpkg in a traditional Debian system. It should handle all of the same use as dpkg itself.

As a further extension, dpkg-ng wants to be multi-version aware. Nix-style installation is the primary goal for this: we want the ability to install multiple versions of libraries and softwares and isolate dependency problems.

This is a secondary function, and should not break traditional Debian systems.

In the most basic sense, dpkg-ng will offload installation to an API hook that allows for a normal or not-so-normal install. This hook will, among other things, allow an external plug-in to replace the default installation path and to make assumptions about what is/is not installed.

`dpkg-ng` Design

An additional plug-in will change the behavior of dpkg-ng to make different assumptions about what is or is not installed for individual packages. Notably:

When dependency graph generation recognizes another version of an already-installed package, or a package with a conflict, the plug-in instructs dpkg-ng to ignore any relationship between the two packages if the conflict is unresolvable without uninstalling desired software. Conflicts that can be resolved peacefully-- replacing a support package with another support package, such that all desired software remains and continues to function--are not prevented.
When instructed, the plug-in may instruct dpkg-ng to not recognize the availability of an upgrade or replacement package for a specific package. Faced with (1), this means dpkg-ng may keep libfoo-1.1 because foobar doesn't see libfoo-1.2 as an available package to satisfy dependencies.
By default, dpkg-ng will not upgrade orphaned libraries not either manually installed or ultimately depended on by something manually installed. Faced with (1) and (2), this means that while libfoo-1.1 is kept for foobar, because barbaz also uses libfoo it will upgrade to libfoo-1.2 and have both installed.

FIXME: How do we do this? Class in Python and call .conflictResolve()?

Notably, conflicting packages can be resolved in one of two ways:

Eliminating the conflict by removing unnecessary, replaceable support packages.
Altering the dependency graph to ignore the conflict, in the case where dpkg-ng would remove or fail to install user-desired software by resolving the conflict.

dpkg-ng will then adjust as necessary, for example by symlinking a required library for a particular software program into its RUNPATH, so as to override the default system library. Resolving this is done by either majority vote (more stuff conflicts with one library than the other) or bulk work analysis (more symlinks have to be made for one than the other, so the one with more symlinks becomes the system library).

Notably, we don't discuss what to do with conflicting programs here, for example PostgreSQL-7.2 and PostgreSQL-9.3. This framework allows for development in that direction.

Interpreter and R-Path

NixOS sets the interpreter and runpath to directly point at specific versions of libraries. For example, the NixOS bash installation from the LiveCD current as of this writing:

INTERP         0x0000000000000200 0x0000000000400200 0x0000000000400200
               0x0000000000000050 0x0000000000000050  R      1
    [Requesting program interpreter: \
    /nix/store/cj7a81wsm1ijwwpkks3725661h3263p5-glibc-2.13/\
    lib/ld-linux-x86-64.so.2]

(Some line breaks added)

And the R-path:

Dynamic section at offset 0xb3550 contains 26 entries:
  Tag        Type                         Name/Value
 0x0000000000000001 (NEEDED)             Shared library: [libreadline.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [libhistory.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [libncursesw.so.5]
 0x0000000000000001 (NEEDED)             Shared library: [libdl.so.2]
 0x0000000000000001 (NEEDED)             Shared library: [libc.so.6]
 0x000000000000000f (RPATH)              Library rpath: [/nix/store/\
 27116bbxhi8dd7l2fjjfbwj6p35ql7n7-readline-6.2/lib:/nix/store/\
 ahg5mlj2mlp7yfl3x875pq95ar763vgj-ncurses-5.9/lib:/nix/store/\
 cj7a81wsm1ijwwpkks3725661h3263p5-glibc-2.13/lib]
 0x000000000000001d (RUNPATH)            Library runpath: [/nix/store/\
 27116bbxhi8dd7l2fjjfbwj6p35ql7n7-readline-6.2/lib:/nix/store/\
 ahg5mlj2mlp7yfl3x875pq95ar763vgj-ncurses-5.9/lib:/nix/store/\
 cj7a81wsm1ijwwpkks3725661h3263p5-glibc-2.13/lib]

(Line breaks added again)

As we can see, Nix adds a run path for each library, to the specific build version. Notably, because DT_RUNPATH exists, DT_RPATH is ignored by the GNU dynamic linker. The normal search order is:

DT_RPATH if DT_RUNPATH does not exist
LD_LIBRARY_PATH environment if not SUID/SGID
DT_RUNPATH
ld.so.cache file from ldconfig
/lib and then /usr/lib

For our purposes we dislike this. It's useless until it's not; and when it's not useless a lot of patchelf has to happen to move things around when upgrading. This does not appeal.

Instead, dpkg-ng sets executables, only on install in multi-install mode, to look for the Interpreter in $INSTALLPATH/lib/, which contains a symlink to the interpreter. The RUNPATH, also set via patchelf, would follow the sequence:

$INSTALLPATH/lib/
/var/sys/current-system/lib/

Thus our resolution would follow:

LD_LIBRARY_PATH environment
$INSTALLPATH/lib/
/var/sys/current-system/lib/
ld.so.cache from ldconfig
/lib/ and then /usr/lib/

Since by standard convention (2) and (3) never happen and have no meaning, we could theoretically build with these to no harm; however by using patchelf we can ensure proper operation of foreign .deb packages.

Multi-Lib Install

Multi-lib support provides a completely different problem than multi- version support. While it's possible to use the same facilities, a lot of work would go into resolving dependencies: the entire install base needs replacement with the alternate library.