swineherd.texi

\input texinfo
@c -*-texinfo-*-
@c SPDX-FileCopyrightText: 2023 Ricardo Wurmus <rekado@elephly.net>
@c
@c SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later

@c %**start of header
@setfilename swineherd.info
@documentencoding UTF-8
@settitle Swineherd Manual
@c %**end of header

@include version.texi

@copying
Copyright @copyright{} 2023 Ricardo Wurmus

Permission is granted to copy, distribute and/or modify this document
under the terms of the GNU Free Documentation License, Version 1.3 or
any later version published by the Free Software Foundation; with no
Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts.  A
copy of the license is included in the section entitled ``GNU Free
Documentation License''.
@end copying

@dircategory Admin
@direntry
* Swineherd: (swineherd).  Guix System container management.
@end direntry

@titlepage
@title Swineherd Manual
@author Ricardo Wurmus

@page
@vskip 0pt plus 1filll
Edition @value{EDITION} @*
@value{UPDATED} @*

@insertcopying
@end titlepage

@contents

@c *********************************************************************
@node Top
@top Swineherd

This document describes the Swineherd @value{VERSION}.

@menu
* Introduction::                What is this all about?
* Building::                    Make it.
* Getting Started::             Preparations and your first container.
* Command Line Interface::      The simplest way to to use it.
* HTTP API::                    Control it via HTTP.
* Acknowledgments::             Thanks!
* GNU Free Documentation License::  The license of this manual.
* Concept Index::               Concepts.
@end menu

@c *********************************************************************
@node Introduction
@chapter Introduction

The @url{https://www.gnu.org/software/shepherd/,GNU Shepherd} is an
elegant service manager looking after a herd of daemons.  It can be
extended with the @url{https://gnu.org/software/guile,Guile} programming
language.

This project aims to provide an extension to the Shepherd, retraining it
as a swineherd, a manager of crude @emph{system containers}.  It does
this by providing a Shepherd service @code{swineherd} that talks to the
Shepherd process to create Guix System containers as Shepherd services.

@c *********************************************************************
@node Building
@chapter Building

This project uses the GNU build system.  To install it from a source
repository, bootstrap the build system first.

@example
autoreconf -vif
@end example

Then run the usual commands to build and test:

@cindex building from source
@example
./configure
make
make check
@end example

@c *********************************************************************
@node Getting Started
@chapter Getting Started

Swineherd uses a btrfs volume to create a subvolume for each container.
Here’s how you can prepare a btrfs root volume that lives in a file.

@example
truncate --size=1024M disk.img
mkfs.btrfs -L container-root -R quota disk.img
mkdir container-root
sudo mount -o loop,compress=zstd disk.img container-root/
@end example

Upon launching the swineherd service, a bridge interface is created,
which is used for all container networking purposes.

You can either launch a separate Shepherd process as root with the
swineherd configuration file or evaluate the swineherd configuration
file in an existing Shepherd process.  Shepherd needs root permissions
because it needs to be able to touch /proc and manipulate network
interfaces.

@example
sudo -E ./pre-inst-env shepherd \
    -c etc/shepherd-config.scm \
    -s /run/swineherd.sock \
    --insecure \
    --logfile=swineherd.log &
@end example

Head to the next section to see how to launch a new container!

@c *********************************************************************
@node Command Line Interface
@chapter Command Line Interface

The swineherd service only has four commands: @command{new} (to add and
run a new container service), @command{kill} (to destroy the container
and remove its service), @command{up} (to create the network bridge),
and @command{down} (to delete the network bridge).

Upon launch of the @code{swineherd} service the network bridge is
created with the name @code{swineherd0}.  While container services are
running it is probably a very bad idea to destroy the bridge interface
with @command{down}.

Here is how you would use the @command{new} command to create a new
container:

@example
SCRIPT=$(guix system container etc/os.scm)
sudo -E herd -s /run/swineherd.sock \
     new swineherd test $SCRIPT
@end example

This registers the new service @code{swine:test}.  Each container
service implements a set of commands.

@deffn Command pid @var{container}
Return the process identifier (PID) of the container process.
@example
sudo -E herd -s /run/swineherd.sock \
     pid swine:test
@end example
@end deffn

@deffn Command exec @var{container} @var{command}
Run the given @var{COMMAND} inside the container's namespaces.
@example
sudo -E herd -s /run/swineherd.sock \
     exec swine:test /run/current-system/profile/bin/touch /home/touched
@end example
@end deffn

@deffn Command fs @var{container} @var{command}
Run the given @var{COMMAND} on the container's root directory.  The root
directory is appended to @var{COMMAND}.  Only commands in the directory
configured with @var{--permitted-host-command-prefix=PREFIX} will be
executed.  This is useful for running a backup command from the host
context.

@example
sudo -E herd -s /run/swineherd.sock \
     fs swine:test /opt/swineherd/bin/backup
@end example
@end deffn

@deffn Command peek @var{container} @var{file} @var{pattern}
Return the last line in the given @var{file} that matches the regular
expression in @var{pattern}.

@example
sudo -E herd -s /run/swineherd.sock \
     peek swine:test /var/log/messages '.*'
@end example
@end deffn

@deffn Command ip @var{container}
Print the IP address assigned to the client network interface inside the
container.  Print nothing at all when no IP address has been assigned.
@end deffn

@deffn Command up @var{container}
Request an IP address for the client network interface inside the
container.  The client network interface has a virtual interface on the
host side that is connected to the @code{swineherd0} bridge interface.
@end deffn

@deffn Command down @var{container}
Request that the client network interface inside the container be
disconnected.
@end deffn

@deffn Command stats @var{container}
Print three container usage statistics on a single line: network bytes
received, network bytes transmitted, and CPU time.  CPU time is computed
as the sum of @code{utime}, @code{stime}, @code{cutime}, and
@code{cstime} for all processes in the process tree spawned by the
container.

@example
receive:888 transmit:1234 cputime:4567
@end example
@end deffn

To destroy the container and its root file system, and remove the
container-specific service use the @command{kill} command:

@example
sudo -E herd -s /run/swineherd.sock \
     kill swineherd test
@end example


@c *********************************************************************
@node HTTP API
@chapter HTTP API

Swineherd comes with an optional HTTP API to launch, manage, and destroy
Guix System containers.  To enable the API server you need to first
start the @code{swineherd-http-api} service.

@example
sudo -E herd -s /run/swineherd.sock \
    start swineherd-http-api
@end example

You can pass configuration options as additional arguments after the
service name, but note that they should exactly match those given to the
@code{swineherd} service.  By default the API server listens on
localhost port 1111.  Below we describe the available API endpoints.

@section API endpoints
@deftypefn {HTTP endpoint} POST /api/container/@var{id}/launch
Register a new container service, run the container by executing the
launch script, and map the specified directories in the container's
@code{mount} namespace.

@table @asis
@item @code{prefix}, string
prefix to add before the given container ID
@item @code{directory-map}, array
array of mapping objects containing three fields: ``source'' (a
string), ``target'' (a string), and ``read-only'' (a boolean).
@item @code{derivations}, array
array of derivation file names (strings).  These derivations will be
built with Guix before launching the container.  Since Guix can
substitute derivations and their outputs this ensures that the machine
running the API server has a complete copy of all requested store items.
You can ignore this if the API server runs on the same machine as the
one making the API requests.
@item @code{script}, string
file name of the run-container script in /gnu/store
@end table

Here is how you would launch a new container named @samp{rekado:test}
(i.e. a container with id @samp{test} and prefix @samp{rekado}) without
any directory mapping:

@example
curl http://localhost:1111/api/container/test/launch \
    -X POST \
    --header "accept: application/json" \
    --header "Content-Type: application/json" \
    --data-binary @@- <<EOF
@{
  "prefix": "rekado",
  "directory-map": [],
  "derivations": [],
  "script": "/gnu/store/@dots{}-run-container"
@}
EOF
@end example

@dots{}and here is an example of the JSON data with two directory
mappings:

@example
@{
  "prefix": "rekado",
  "directory-map": [
    @{
      "source": "/home/me/tmp",
      "target": "/wherever/you/want",
      "read-only": true
    @},
    @{
      "source": "/data/projects/important",
      "target": "/somewhere/else",
      "read-only": false
    @}
  ],
  "derivations": [],
  "script": "/gnu/store/@dots{}-run-container"
@}
@end example
@end deftypefn

@deftypefn {HTTP endpoint} DELETE /api/container/@var{id}/destroy
Same as the @command{kill} command implemented by the @code{swineherd}
service.  It destroys the indicated container and removes its service.

@table @asis
@item @code{prefix}, string
prefix to add before the given container ID
@end table

@example
curl http://localhost:1111/api/container/third/destroy \
    -X DELETE \
    --header "accept: application/json" \
    --header "Content-Type: application/json" \
    --data-binary @@- <<EOF
@{
  "prefix": "rekado"
@}
EOF
@end example
@end deftypefn

@deftypefn {HTTP endpoint} POST /api/container/@var{id}/exec
Same as the @command{exec} command above, except that any command to be
executed must be explicitly permitted in advance.  The only way to
permit commands at the moment is to launch the API server with
@option{--permitted-command-prefix=PREFIX}, where @var{PREFIX} is a
string that the command must be prefixed with.  One way to do this is to
place a collection of authorized scripts in a well-known sub-directory,
such as @file{/swine-scripts/bin/} and use that as the value for the
command prefix.

@table @asis
@item @code{prefix}, string
prefix to add before the given container ID
@item @code{command}, string
command to execute in the container.
@item @code{arguments}, array
JSON array of strings, arguments to pass to COMMAND.
@end table
@end deftypefn

@deftypefn {HTTP endpoint} POST /api/container/@var{id}/fs
Same as the @command{fs} command above.  Swineherd will only permit
those commands that are located in the prefix configured with
@option{--permitted-host-command-prefix=PREFIX}, where @var{PREFIX} is a
string that the command must be prefixed with.

@table @asis
@item @code{prefix}, string
prefix to add before the given container ID
@item @code{command}, string
command to execute in the container.
@item @code{arguments}, array
JSON array of strings, arguments to pass to COMMAND.
@end table
@end deftypefn

@deftypefn {HTTP endpoint} PUT /api/container/@var{id}/connect
Same as the @command{up} command above.  Connects the container's
network interface.

@table @asis
@item @code{prefix}, string
prefix to add before the given container ID
@end table
@end deftypefn

@deftypefn {HTTP endpoint} PUT /api/container/@var{id}/disconnect
Same as the @command{down} command above.  Disconnects the container's
network interface.

@table @asis
@item @code{prefix}, string
prefix to add before the given container ID
@end table
@end deftypefn

@deftypefn {HTTP endpoint} GET /api/container/@var{id}/pid
Same as the @command{pid} command above.

@table @asis
@item @code{prefix}
string, prefix to add before the given container ID
@end table
@end deftypefn

@deftypefn {HTTP endpoint} GET /api/container/@var{id}/info
Same as the @command{stats} command above.

@table @asis
@item @code{prefix}
string, prefix to add before the given container ID
@end table
@end deftypefn

@deftypefn {HTTP endpoint} GET /api/container/@var{id}/peek
Same as the @command{peek} command above.

@table @asis
@item @code{prefix}, string
prefix to add before the given container ID

@item @code{file}, string
absolute name of the file in the container

@item @code{pattern}, string
regular expression to match against lines in FILE.
@end table
@end deftypefn

@deftypefn {HTTP endpoint} GET /api/containers
Return a list of all managed containers.
@end deftypefn


@c *********************************************************************
@node Acknowledgments
@chapter Acknowledgments

Thanks to the following people who contributed to the Swineherd through
testing, patches, or through insightful discussions:

@itemize @bullet
@item Jiminy Cricket
@end itemize

Thank you.

@c *********************************************************************
@node GNU Free Documentation License
@appendix GNU Free Documentation License

@include LICENSES/GFDL-1.3-no-invariants-or-later.texi

@page

@c *********************************************************************
@node Concept Index
@unnumbered Concept Index

@printindex cp
@bye