Running a minimal ubuntu rootfs as regular user

sometimes it's convenient to run a specific piece of software that's either only available for ubuntu (or glibc), or building it would require to build a huge clusterfuck of dependencies. here's how you can use ubuntu inside your sabotage install (or any other distro) as regular user and without having to pre-allocate gigabytes of harddisk space for a VM.

step 1: prepare rootfs

• prepate a directory in your home:

    $ mkdir ~/ubuntu
    $ cd ~/ubuntu

• get a minimal ubuntu rootfs

    $ wget http://cdimage.ubuntu.com/ubuntu-base/releases/20.04.1/release/ubuntu-base-20.04.1-base-amd64.tar.gz
    $ mkdir root
    $ cd root
    $ tar xf ubuntu-base-20.04.1-base-amd64.tar.gz
    $ cd ..

if your tar is from GNU coreutils, use instead:

    $ tar --no-same-owner -xf ubuntu-base-20.04.1-base-amd64.tar.gz

this prevents issues with file ownership when you extract it as root.

• make sure you have correct permissions for all files inside root/:

    $ chown -R username:username root/

where username is your username.

• fill inside root/ the file /etc/resolv.conf with a nameserver ip

    $ echo "nameserver 8.8.8.8" > root/etc/resolv.conf

• create apt config that prevents it from doing anal checking of whether all chown-related syscalls REALLY worked

    $ cat << EOF > root/etc/apt/apt.conf.d/99-disable-sandbox
    APT::Sandbox::User "root";
    APT::Sandbox::Verify "0";
    APT::Sandbox::Verify::IDs "0";
    APT::Sandbox::Verify::Groups "0";
    APT::Sandbox::Verify::Regain "0";
    EOF

• likewise for dpkg:

    $ echo force-not-root >> root/etc/dpkg/dpkg.cfg

unfortunately, there's currently a bug in dpkg preventing this configuration from working in all cases, so you still need the idfake tool mentioned later in this article.

step 2: prepare tools to enter the chroot

variant 1 (most generic): using proot

proot allows to chroot as regular user almost everywhere. it achieves this by using ptrace(). this works even on kernels as ancient as 2.6.32. i've uploaded a statically linked binary.

    $ proot -0 -k "3.8.0" -b /dev:/dev -b /proc -r root/ /bin/bash

the only alternatives i'm aware of to achieve the same result as proot (on kernels without USER_NS) is fakeroot and fakechroot, but i really don't recommend them after wasting several hours trying to get them to work. especially fakeroot is a huge turd of code full of cruft and glibc-specific assumptions which doesn't even compile on ubuntu. the rest is fragile bash scripts which reference each other so it's really hard to set them up in a way they find the required files when you can't do a system-wide install. fakeroot and fakechroot are also full of hardcoded paths to glibc dynlinker and libc files, ldconfig and so on, they're really just a huge mess and shouldn't be used.

variant 2 (bubblewrap):

bubblewrap requires USER_NS support built into the kernel. this is the default on most distro kernels > 3.8.0. i uploaded a statically linked binary. it's got lots of options. following stanza allows us to successfully use the rootfs.

    $ bwrap --proc /proc --dev /dev --ro-bind /etc/resolv.conf /etc/resolv.conf --tmpfs /dev/shm --bind ./root / --uid 0 --gid 0 --unshare-user --chdir / /bin/bash

bubblewrap is refreshingly lightweight and generic. certainly the recommended option if your distro ships it and your kernel supports USER_NS.

variant 3: sabotage's super_chroot.c

like bubblewrap, it uses USER_NS, but it's much lighter. basically it's a 20 line C file.

• copy from sabotage git dir the file enter-chroot to your ~/ubuntu dir:

    $ wget https://raw.githubusercontent.com/sabotage-linux/sabotage/563d56edea4077ff30425716eb624e562bdfedb9/enter-chroot
    $ chmod +x enter-chroot

• replace the line /bin/sh --login with /bin/bash --login in enter-chroot

• get super-chroot (it's like a 20 line version of bubblewrap)

    $ mkdir KEEP ; cd KEEP
    $ wget https://raw.githubusercontent.com/sabotage-linux/sabotage/4d436870d2173ebc05c1ab1f7e8e4ec7693bffb0/KEEP/super_chroot.c
    $ cd ..

• remove the 3 lines containing the string "tarballs" in KEEP/super_chroot.c

• put the following lines inside a file called config:

export SABOTAGE_BUILDDIR="$HOME/ubuntu/root"
export SUPER=1
[ -z "$H" ] && H="$PWD"
export K="$H"/KEEP
export R="$SABOTAGE_BUILDDIR"

now you can use ./enter-chroot to chroot into the ubuntu rootfs as regular user.

inside the rootfs

the first thing you want to do is run apt update and then install some tools you need inside the chroot.

good luck!

update: working around several dpkg-related chown attempts

while using apt for smaller things works just fine with the above mentioned tweaks, there are cases when some dpkg triggers try to run additional programs that run chown-related functionality and fail with EPERM. for this purpose i created a small program, called idfake (here's a static linked x86_64 binary for your convenience). there's also a build for i386 which may be helpful if you try this guide for debian i386. the program uses ptrace to hook syscalls to chown-related functionality and makes them appear to succeed. if running plain apt fails, just run idfake apt install whatever.

addendum: making gnu tar less annoying

gnu tar has the idiotic property to try to chown extracted files to the user id that's stored in them if run as root. that might be helpful if you make local backups, however in the internet age 99% of your tarballs come from the net, and you certainly don't want to create files with random UIDs. additionally, this fails as we can't chown files. the default should be other way round, pass a specific option if you do want to extract user ids as root.

anyway, here's the step-by-step guide how we can fix it without modifying every call to tar in existing scripts: (if you just want the solution, skip to the end)

• ubuntu uses GNU tar 1.30 as can be seen from the verbose output with --version.
• we download GNU tar 1.30 tarball, and extract it... with --no-same-owner, since without it even that fails...
• tracing what the --no-same-owner does in the code leads us to src/extract.c, specifically this snippet:

void
extr_init (void)
{
  we_are_root = geteuid () == ROOT_UID; 
  same_permissions_option += we_are_root;
  same_owner_option += we_are_root;

the command line agrument parser in tar.c earlier sets same_owner_option to -1 if the option --no-same-owner is passed. so we want to patch this code that we_are_root is 0 rather than 1. we could do this in this very same file and compile gnu tar 1.30 from source, then replace the binary in /usr/bin.

the patch for that would look like:

 void
 extr_init (void)
 {
-  we_are_root = geteuid () == ROOT_UID; 
+  we_are_root = 0;
   same_permissions_option += we_are_root;
   same_owner_option += we_are_root;

however recompiling it from source is imo too much effort for this scenario, we don't know whether ubuntu has other stuff added to their tar version that might be needed somewhere, and we would need to install packages for a compiler toolchain inside the rootfs, and therefore making it much less minimal.

thus, we just patch the existing binary.

• let's disassemble the binary and look for the geteuid() call:

objdump -dr /usr/bin/tar | less leads us to the only spot in the binary where it is called:

   192e5:       e8 d6 0e ff ff          callq  a1c0 <geteuid@plt>
   192ea:       85 c0                   test   %eax,%eax
   192ec:       0f 94 c0                sete   %al
   192ef:       31 ff                   xor    %edi,%edi
   192f1:       88 05 29 4e 05 00       mov    %al,0x54e29(%rip)        # 6e120 <stderr@@GLIBC_2.2.5+0x340>

the stderr stuff is misleading, this is just an offset into the .data segment. what we can do to patch this is to replace the test is eax 0 ? if so, put 1 into al, else 0 with put zero into al. (eax is the return value register which contains the result of geteuid()). so we need to find the hexadecimal opcode byte sequence to do this. xor eax, eax is known to put 0 into eax and using very few opcodes to do so.

• let's write a tiny asm file to get the hexadecimal values for the opcode.

xor.s:

.global foo
.type foo @function
foo:
        xor %eax, %eax

• compile it with gcc: gcc xor.s -c

• and look at the result with objdump: objdump -d xor.o:

0000000000000000 <foo>:
   0:   31 c0                   xor    %eax,%eax

ok, so xor eax, eax is 31c0. means we need to look for e8d60effff85c00f94c0 in the binary and replace it with e8d60effff31c0909090. for the search we include the e8d60effff prefix of the seteuid() call so we find the right spot in the binary. 85c0 is replaced with 31c0 and the remaining three bytes overwritten with 0x90 which means "nop".

• open /usr/bin/tar with hexedit, search for e8d60effff85c00f94c0 and then overwrite the 85c00f94c0 sequence with 31c0909090, and save (F2).

done, finally GNU tar behaves as it should.