Create minimal template for lightweight application isolation

[haraldrudell]

This would make lxc unprivileged containers easy.

This is the right thing to do.

Below is a bash script for Ubuntu that should be turned into a generic template

lxc unprivileged containers are awesome because they create a no-performance-penalty machine clone featuring hardware acceleration in about 80 KiB. The benefit to share the Linux system software is that the container does not need to be maintained or updated.

Isolation can be network, users, processes and file system.

Mysteriously, for unprivileged use as-installed, lxc does not actually offer this function but only the download template for heavier full Linux distributions. The lxc template support for unprivileged containers is very limited. It is unique for lxc to have no-proprietary-garbage, full machine capability and no root vulnerability. This should be supported well.

The below template provides the function at the expense of 80 KiB disk space. If one looks at the x11docker project, it could be extended with Wayland and secure per-application X server for safe accelerated isolation of graphical user interface applications.

Basically take the below script, rework it by looking at the download template, test it for other than Ubuntu like Gentoo, macOS and Windows, add X/Wayland, publish it as lxc-minimal or lxc-localhost

Ubuntu should use this isolation template for its app store just like Apple does with xhyve. It’s what is privacy and security on the desktop in 2017,

#!/bin/bash -eu
# © 2017 Harald Rudell <harald.rudell@gmail.com> (http://haraldrudell.com) ISC License.
while [ $# -ge 1 ]; do case "$1" in
  --help) echo "lxc-unpriv [--name=name]"; exit;;
  --name=*) NAME="${1#*=}";;
  *) echo >&2 "unknown options: '$*' usage: lxc-unpriv [--name=name]"; exit 2;;
esac; shift; done
if [ ! "${NAME-}" ]; then NAME=`hostname --short`-`date --utc +%y%m%d-%H%M%S`; fi
echo "Creating unprivileged container $NAME …"
config=~/".local/share/lxc/$NAME/config"
lxc-create --template=none --name="$NAME"
if [ ! -s "$config" ]; then echo >2 "Container configuration file missing: '$config'"; exit 1; fi
echo "Container configuration file: $config"
sed --in-place "1i# Container created on: `date --utc +%y%m%d-%H%M%S` by lxc-unpriv © 2017 Harald Rudell <harald.rudell@gmail.com> (http://haraldrudell.com) ISC License.\n# lxc-unpriv: `stat --format=%y "$0"` sha256: `sha256sum "$0" | cut --delimiter=" " --field=1`" "$config"
root_uid="`sed --silent 's/^\s*lxc.id_map\s*=\s*u\s*0\s*\([0-9]\+\).*/\1/p' "$config"`"
root_gid="`sed --silent 's/^\s*lxc.id_map\s*=\s*g\s*0\s*\([0-9]\+\).*/\1/p' "$config"`"
[ "$root_uid" -gt 0 ] && [ "$root_gid" -gt 0 ] || { echo >&2 "Failed to parse lxc.id_map in config file '$config'"; exit 1; }echo "Container root group and user mapped to host: $root_uid:$root_gid"
rootfs=~/".local/share/lxc/$NAME/rootfs"
mkdir --parents \
"$rootfs/bin" \
"$rootfs/etc/init.d" \
"$rootfs/etc/rc.d" \
"$rootfs/etc/sysconfig/network-scripts" \
"$rootfs/home" \
"$rootfs/lib" \
"$rootfs/lib64" \
"$rootfs/proc" \
"$rootfs/root" \
"$rootfs/sbin" \
"$rootfs/sys" \
"$rootfs/tmp" \
"$rootfs/usr" \
"$rootfs/var"
touch "$rootfs/etc/fstab"
grep --quiet "^lxc.network.ipv4" "$config" || echo "send host-name = gethostname();" >"$rootfs/dhclient.conf"
ln --symbolic "$rootfs/run" "$rootfs/var/run"
cat <<EOF >"$rootfs/etc/passwd"
root:x:0:0:root:/root:/bin/bash
EOF
cat <<EOF >"$rootfs/etc/group"
root:x:0:root
EOF
cat <<EOF >>"$config"
lxc.include = /usr/share/lxc/config/ubuntu.common.conf
lxc.include = /usr/share/lxc/config/ubuntu.userns.conf
lxc.rootfs = $rootfs
lxc.utsname = $NAME
lxc.pts = 1024
lxc.cap.drop = sys_module mac_admin mac_override sys_time
lxc.aa_profile = unconfined
lxc.mount.entry = /bin bin none ro,bind 0 0
lxc.mount.entry = /etc/init.d etc/init.d none ro,bind 0 0
lxc.mount.entry = /lib lib none ro,bind 0 0
lxc.mount.entry = /sbin sbin none ro,bind 0 0
lxc.mount.entry = /usr usr none ro,bind 0 0
lxc.mount.auto = cgroup:mixed proc:mixed sys:mixed
EOF
if [ -d /etc/sysconfig/network-scripts ]; then cat <<EOF >>"$config"
lxc.mount.entry = /etc/sysconfig/network-scripts etc/sysconfig/network-scripts none ro,bind 0 0
EOF
fi
if [ -d /etc/rc.d ]; then cat <<EOF >>"$config"
lxc.mount.entry = /etc/rc.d etc/rc.d none ro,bind 0 0
EOF
fi
if [ "$(uname --machine)" = x86_64 ]; then cat <<EOF >>"$config"
lxc.mount.entry = /lib64 lib64 none ro,bind 0 0
lxc.arch = x86_64
EOF
fi
CMD=(sudo chown --recursive $root_uid:$root_gid "$rootfs")
echo -e "\nMust change ownership of the root file system to container’s root user: $root_uid:$root_gid"
echo "Elevated privileges are required to execute: ${CMD[*]}"
"${CMD[@]}"
echo -e "Unprivileged container was created successfully.\n\nUseful commands:"
echo "Start the container: lxc-start --name=$NAME"
echo "Stop the container: lxc-stop --name=$NAME --timeout=1"
echo "Get a command prompt in running container: lxc-attach --name=$NAME"
echo "Follow the container’s log: tail --retry --follow=~/.local/share/lxc/$NAME/$NAME.log"
echo "request ip address for running container: lxc-attach --name=$NAME -- dhclient eth0"
echo "Get container ip addresses: lxc-info --name=$NAME --ips"
echo "Run a command inside the container (container should be stopped): lxc-execute --name=$NAME -- command args…"

[haraldrudell]

Now imports by distro, sudo no longer required

These containers will always run:

• without writing to the hosts's root file system
• with user separation
• with separate network
• with separate processes

It can run with no network lxc.network.type = empty
Somehow it doesn’t presently run with shared network lxc.network.type = none

Neither will it run with shared processes lxc-start --share-ipc 1… but unprivileged containers probably can’t do that

#!/bin/bash -eu
# © 2017 Harald Rudell <harald.rudell@gmail.com> (http://haraldrudell.com) ISC License.
while [ $# -ge 1 ]; do case "$1" in
  --help) echo "lxc-unpriv [--name=name]"; exit;;
  --name=*) NAME="${1#*=}";;
  *) echo >&2 "unknown options: '$*' usage: lxc-unpriv [--name=name]"; exit 2;;
esac; shift; done
if [ ! "${NAME-}" ]; then NAME=`hostname --short`-`date --utc +%y%m%d-%H%M%S`; fi
echo "Creating unprivileged container $NAME …"
config=~/".local/share/lxc/$NAME/config"
lxc-create --template=none --name="$NAME"
if [ ! -s "$config" ]; then echo >&2 "Container configuration file missing: '$config'"; exit 1; fi
echo "Container configuration file: $config"
rootfs=~/".local/share/lxc/$NAME/rootfs"
sed --in-place "1i# Container created on: `date --utc +%y%m%d-%H%M%S` by lxc-unpriv © 2017 Harald Rudell <harald.rudell@gmail.com> (http://haraldrudell.com) ISC License.\n# lxc-unpriv: `stat --format=%y "$0"` sha256: `sha256sum "$0" | cut --delimiter=" " --field=1`" "$config"
root_uid="`sed --silent 's/^\s*lxc.id_map\s*=\s*u\s*0\s*\([0-9]\+\).*/\1/p' "$config"`"
root_gid="`sed --silent 's/^\s*lxc.id_map\s*=\s*g\s*0\s*\([0-9]\+\).*/\1/p' "$config"`"
[ "$root_uid" -gt 0 ] && [ "$root_gid" -gt 0 ] || { echo >&2 "Failed to parse lxc.id_map in config file '$config'"; exit 1; }
echo "Container root group and user mapped to host: $root_uid:$root_gid"
cat <<EOF >>"$config"
lxc.rootfs = $rootfs
lxc.utsname = $NAME
lxc.pts = 1024
lxc.cap.drop = sys_module mac_admin mac_override sys_time
lxc.aa_profile = unconfined
lxc.mount.entry = /bin bin none ro,bind 0 0
lxc.mount.entry = /etc/init.d etc/init.d none ro,bind 0 0
lxc.mount.entry = /lib lib none ro,bind 0 0
lxc.mount.entry = /sbin sbin none ro,bind 0 0
lxc.mount.entry = /usr usr none ro,bind 0 0
lxc.mount.auto = cgroup:mixed proc:mixed sys:mixed
EOF
DISTRO=`lsb_release --id --short` && DISTRO="${DISTRO,,}" # ubuntu
COMMON="/usr/share/lxc/config/$DISTRO.common.conf" && if [ -f "$COMMON" ]; then cat <<EOF >>"$config"
lxc.include = $COMMON
EOF
fi
USERNS="/usr/share/lxc/config/$DISTRO.userns.conf" && if [ -f "$USERNS" ]; then cat <<EOF >>"$config"
lxc.include = $USERNS
EOF
fi
if [ -d /etc/sysconfig/network-scripts ]; then cat <<EOF >>"$config"
lxc.mount.entry = /etc/sysconfig/network-scripts etc/sysconfig/network-scripts none ro,bind 0 0
EOF
fi
if [ -d /etc/rc.d ]; then cat <<EOF >>"$config"
lxc.mount.entry = /etc/rc.d etc/rc.d none ro,bind 0 0
EOF
fi
if [ "$(uname --machine)" = x86_64 ]; then cat <<EOF >>"$config"
lxc.mount.entry = /lib64 lib64 none ro,bind 0 0
lxc.arch = x86_64
EOF
fi
DIRS=(); for D in bin etc/init.d etc/rc.d etc/sysconfig/network-scripts home lib lib64 proc root sbin sys tmp usr var; do DIRS+=("$rootfs/$D"); done
mkdir --parents "${DIRS[@]}"
touch "$rootfs/etc/fstab"
ln --symbolic "$rootfs/run" "$rootfs/var/run"
cat <<EOF >"$rootfs/etc/passwd"
root:x:0:0:root:/root:/bin/bash
EOF
cat <<EOF >"$rootfs/etc/group"
root:x:0:root
EOF
HOST_UID=`id --user` && HOST_GID=`id --group`
lxc-usernsexec -m u:0:$HOST_UID:1 -m g:0:$HOST_GID:1 -m u:1:$root_uid:1 -m g:1:$root_gid:1 -- chown --recursive 1:1 "$rootfs"
echo -e "Unprivileged container was created successfully.\n\nUseful commands:"
echo "Start the container: lxc-start --name=$NAME"
echo "Stop the container: lxc-stop --name=$NAME --timeout=1"
echo "Get a command prompt in running container: lxc-attach --name=$NAME"
echo "Follow the container’s log: tail --retry --follow=~/.local/share/lxc/$NAME/$NAME.log"
echo "request ip address for running container: lxc-attach --name=$NAME -- dhclient eth0"
echo "Get container ip addresses: lxc-info --name=$NAME --ips"
echo "Run a command inside the container (container should be stopped): lxc-execute --name=$NAME -- command args…"

[haraldrudell]

Rewrote to a template at /usr/share/lxc/templates/lxc-localhost

Use like lxc-create --template=localhost --name=thematrix
or sudo lxc-create --template=localhost --name=thematrix

I no test privileged containers or os other than ubuntu. Theoretically, anything works.

#!/bin/bash -eu
# © 2017 Harald Rudell <harald.rudell@gmail.com> (http://haraldrudell.com) ISC License.
while [ $# -ge 1 ]; do case "$1" in
  --help) echo "lxc-localhost: no options"; exit;;
  --name=*) NAME="${1#*=}";;
  --path=*) config="${1#*=}/config";; # …/name
  --rootfs=*) rootfs="${1#*=}";; # …/rootfs
  --mapped-uid) if [ $# -ge 2 ]; then LXCUID="$2"; shift; fi;;
  --mapped-gid) if [ $# -ge 2 ]; then LXCGRP="$2"; shift; fi;;
  *) echo >&2 "unknown options: '$*'"; exit 2;;
esac; shift; done
echo "Container configuration file: $config"
baseconfig=/etc/lxc/default.conf; userconfig=~/.config/lxc/default.conf; if [ "${LXCUID-}" -a -f "$userconfig" ]; then baseconfig="$userconfig"; fi
cat <<EOF >"$config"
# Container created `date --utc +%y%m%d-%H%M%S`Z by lxc-localhost © 2017 Harald Rudell <harald.rudell@gmail.com> (http://haraldrudell.com) ISC License.
# $baseconfig
EOF
cat "$baseconfig" >>"$config"
cat <<EOF >>"$config"
# lxc-localhost
lxc.rootfs = $rootfs
lxc.utsname = $NAME
lxc.pts = 1024
lxc.cap.drop = sys_module mac_admin mac_override sys_time
lxc.aa_profile = unconfined
lxc.mount.entry = /bin bin none ro,bind 0 0
lxc.mount.entry = /etc/init.d etc/init.d none ro,bind 0 0
lxc.mount.entry = /lib lib none ro,bind 0 0
lxc.mount.entry = /sbin sbin none ro,bind 0 0
lxc.mount.entry = /usr usr none ro,bind 0 0
lxc.mount.auto = cgroup:mixed proc:mixed sys:mixed
EOF
DISTRO=`lsb_release --id --short` && DISTRO="${DISTRO,,}" # ubuntu
COMMON="/usr/share/lxc/config/$DISTRO.common.conf" && if [ -f "$COMMON" ]; then cat <<EOF >>"$config"
lxc.include = $COMMON
EOF
fi
if [ -d /etc/sysconfig/network-scripts ]; then cat <<EOF >>"$config"
lxc.mount.entry = /etc/sysconfig/network-scripts etc/sysconfig/network-scripts none ro,bind 0 0
EOF
fi
if [ -d /etc/rc.d ]; then cat <<EOF >>"$config"
lxc.mount.entry = /etc/rc.d etc/rc.d none ro,bind 0 0
EOF
fi
if [ "$(uname --machine)" = x86_64 ]; then cat <<EOF >>"$config"
lxc.mount.entry = /lib64 lib64 none ro,bind 0 0
lxc.arch = x86_64
EOF
fi
DIRS=(); for D in bin etc/init.d etc/rc.d etc/sysconfig/network-scripts home lib lib64 proc root sbin sys tmp usr var; do DIRS+=("$rootfs/$D"); done
mkdir --parents "${DIRS[@]}"
touch "$rootfs/etc/fstab"
ln --symbolic "$rootfs/run" "$rootfs/var/run"
cat <<EOF >"$rootfs/etc/passwd"
root:x:0:0:root:/root:/bin/bash
EOF
cat <<EOF >"$rootfs/etc/group"
root:x:0:root
EOF
if [ "${LXCUID-}" ]; then chown --recursive "$LXCUID" "$rootfs"
  USERNS="/usr/share/lxc/config/$DISTRO.userns.conf" && if [ -f "$USERNS" ]; then cat <<EOF >>"$config"
lxc.include = $USERNS
EOF
fi; fi
if [ "${LXCGID-}" ]; then chgrp --recursive "$LXCUID" "$rootfs"; fi
echo -e "Container was created successfully.\n\nUseful commands:"
echo "Start the container: lxc-start --name=$NAME"
echo "Stop the container: lxc-stop --name=$NAME --timeout=1"
echo "Get a command prompt in running container: lxc-attach --name=$NAME"
echo "Follow the container’s log: tail --retry --follow=~/.local/share/lxc/$NAME/$NAME.log"
echo "request ip address for running container: lxc-attach --name=$NAME -- dhclient eth0"
echo "Get container ip addresses: lxc-info --name=$NAME --ips"
echo "Run a command inside the container (container should be stopped): lxc-execute --name=$NAME -- command args…"

[brauner]

If you to start a proper discussion around this template and it isn't too much work for you and you can deal with a potential "no" you could send a pr that properly integrates this. But @stgraber might already have an opinion on this. :)

[stgraber]

I'm not necessarily against the idea, though the coding style would definitely need cleaning up and we'd like something closer in style to lxc-local and lxc-download.

[bijwaard]

Dear all,

Very nice to be able to use a host-linked template to reduce storage space, especially for unprivileged containers on embedded devices.

I used the current (lxc-3.0.3) lxc-download template to redo some of the steps from Harald's script, fixed a number of issues to make /var, /run and /etc writable, cgroup conflicts, and renamed it to lxc-hostlink. Both lxc-start and lxc-execute now work with networking on a nanopi neo+2 with armbian/debian buster (with confirmed lxc-3.1.0) and on ubuntu bionic (confirmed with lxc-3.0.3). Ubuntu has some poweroff issues when using lxc-start (have to stop it with: lxc-stop -n mycontainer --kill) and the virtual machine may need 8.8.8.8 as extra nameserver in /etc/resolv.conf.

You may need to dpkg-reconfigure on the used packages within the container, to get a vanilla configuration (e.g. for openssh-server).

lxc-hostlink.txt

Kind regards,
Dennis

[bijwaard]

I found a permission problem with the new template, it needs to make /tmp and /var/tmp read/writeable for others, especially for running user programs in the container:
chmod a+rwxt $LXC_ROOTFS/tmp $LXC_ROOTFS/var/tmp

Furthermore, I found that for more stable networking the following packages can better be disabled in the container if they are running on the host: isc-dhcp-server bind9 ntp hostapd lxc lxc-net ufw