Tutorial: Building Rumprun Unikernels

huikang edited this page Nov 27, 2016 · 20 revisions
Clone this wiki locally

The Rumprun unikernel enables running POSIX applications as unikernels on top of embedded systems and cloud hypervisors such as Xen and KVM. This tutorial will go over the nuts and bolts of building and running Rumprun unikernels. The target audience is people wanting to create packages on top of Rumprun. It is not necessary to study this tutorial if you want to run existing packages for Rumprun, though doing so will give deep knowledge which may help in potential troubleshooting situations.

While the main purpose of Rumprun is to support existing, unmodified applications, we will use a "hello world" class hand-crafted application here for simplicity reasons. The skills you learn will be applicable to building real-world programs as Rumprun unikernels. We will use commands which produce binaries for QEMU/KVM, but will out the necessary replacements if you want to follow the tutorial on Xen instead.

What we will be covering

  1. Building the Rumprun Platform
  2. Building applications
  3. Running applications
  4. Build frameworks
    4.1. GNU autotools
    4.2. CMake

If something does not work, and you cannot figure out how to move forward, please contact the community for assistance.

  1. Building the Rumprun Platform ================================

The first thing to note is that Rumprun unikernels are always cross-compiled. In practical terms, that statement means that the compiler will never run on a Rumprun unikernel. Instead, the compiler always runs on a build host (e.g. a regular Linux/BSD system). In case a software package supports cross-compiling, odds are that you can just compile it as a Rumprun unikernel. If the software package uses a custom build system which does not support cross-compilation, you will first have to add cross-compile support to the build system in question. Since we are optimistic, and since modifying an arbitrary build system to support cross-compilation is beyond the scope of this tutorial, we assume cross-compilable software.

Before we get to building application software, we need to build the Rumprun framework itself. The first thing to know is that we do not ship our own compiler with Rumprun. Instead, we try to make use of whatever compiler you specify. The specified compiler determines the machine architecture which the produced Rumprun unikernels are capable of running on. By default, you host compiler will be used, so if you are on x86 and want the unikernel to run on x86, you don't need to specify anything.

The second thing to know is that you specify the platform you want the unikernels to run on at this stage. As a rule of thumb, use xen if you want to run on Xen[1], and hw if you want to run on anything else, including bare metal and KVM.

Putting those two bits of information together, let's fetch the Rumprun repo and run the build script[2]. We specify CC explicitly just to show how it can be set.

$ git clone http://repo.rumpkernel.org/rumprun
$ cd rumprun
$ git submodule update --init
$ CC=cc ./build-rr.sh hw
>> toolchain tuple: x86_64-rumprun-netbsd
>> cc wrapper: x86_64-rumprun-netbsd-gcc
>> installed to "/root/rumprun/./rumprun"
>> Set tooldir to front of $PATH (bourne-style shells)
. "/root/rumprun/./obj-amd64-hw/config-PATH.sh"
>> ./build-rr.sh ran successfully

Make note of the toolchain tuple (x86_64-rumprun-netbsd) and compiler (x86_64-rumprun-netbsd-gcc) names. They may be different on your system. Before we proceed to the next stage, let's put the toolchain into our path so that we can execute the compiler later:

$ export PATH="${PATH}:$(pwd)/rumprun/bin"


To build the Rumprun toolchain for the xen platform, you need to have the Xen headers installed. On Debian-based systems you can install these with apt-get install libxen-dev.


If you are on OS X, you need to obtain an ELF cross compiler which you specify to the process. The host compiler is not usable because it generates the Mach-O object format instead of ELF. See Howto: Set up a Rumprun toolchain in Mac OS X

  1. Building applications ========================

So we have the platform built. How does that translate into a running program? Usually, you have to build and link the application. That's precisely what you need to do here too. Let's go into a test directory and try to cross-compile a simple application as a Rumprun unikernel. Of course, it is not a realistic assumption that you will be building large projects manually with cc. Later in this tutorial we will cover build frameworks, but for now it's the sheer, raging basics.

$ mkdir -p test
$ cd test
$ cat > helloer.c << EOF
#include <stdio.h>
#include <unistd.h>

	printf("Hello, Rumprun ... I'm feeling tired\n");
	printf("much better!\n");
	return 0;

You'll notice that the programming environment looks like a normal C/POSIX environment. That compatibility is a given, otherwise existing software would not run. So, let's try to run the above code on the host. Checking that stuff runs on the host is not required when building Rumprun unikernel -- sometimes the host may not even support the same features as Rumprun -- but it's nevertheless a good way to get an idea of what should happen. Therefore, we use run-on-the-host as an educational tool:

$ cc -o helloer helloer.c
$ ./helloer
Hello, Rumprun ... I'm feeling tired
much better!

Ok, seemed to work. Let's try the same for Rumprun. We need to use the cross-compiler wrapper produced by the build-rr.sh script. Good for us, we remembered to make note of the wrapper name earlier. Use below whatever build-rr printed for you.

$ x86_64-rumprun-netbsd-gcc -o helloer-rumprun helloer.c

So, um, how do we run that? The tool for running Rumprun unikernels is eponymously called rumprun. Let's try that the simple way:

$ rumprun qemu helloer-rumprun

!!! NOTE: rumprun is experimental. syntax may change in the future

[some error message]

We observe that no, we cannot run it. We need to bake the binary.

(the remainder of this tutorial will omit the verbose "X is experimental and may change" warnings from the quoted output)


Normally you just compile and are good to go. What gives with this baking step? Consider the regular operating system case. Your kernel is implicitly present on the system where you run the application. With Rumprun there is no implicit kernel since everything is a single package running on top of a hypervisor or bare metal. Therefore, we must explicitly specify the bits which would normally be determined by booting a system with a given kernel. This specifying is done using the rumprun-bake command. There are a number of reasons why rumprun-bake is a separate command instead of being part of the compiler, but we will not go into those in this tutorial.

To use rumprun-bake, we need to specify the kernel bits we want, the output binary and the input binary. The kernel bits are a set of rump kernel components, and the "correct" set depends on what precisely you want to do with your application and where you want to run it. For example, if you're running a web server, you most likely do not need audio drivers. If you don't want to think too hard, there is a "generic" configuration which includes more or less everything. There is also a "virtio" configuration which is more appropriate for cloud deployment, but lacks some components such as audio. We will simply use "generic" here:

$ rumprun-bake hw_generic helloer-rumprun.bin helloer-rumprun

That's it. We can now run the binary.

  1. Running applications =======================

As mentioned earlier, we provide the rumprun tool for running Rumprun unikernels. Let's try the same snippet as earlier, which one exception: we use the -i option to specify that we want an interactive session (otherwise we couldn't see our printf output):

$ rumprun qemu -i helloer-rumprun.bin 

You should see the results. Press ctrl-C to exit QEMU. If your system supports KVM (kernel virtual machine), you can use kvm instead of qemu on the above command line.

The rumprun tool provides a large number of options. Going over them is beyond the scope of this tutorial, but please try rumprun -h to see the usage.

TIP: if you want to see what rumprun actually does, run it with the -D parameter, e.g.

$ rumprun -D qemu -i helloer-rumprun.bin 
  1. Build Frameworks ===================

4.1 GNU autotools

We'll add GNU autotools support to our example program and show how to build it. This section assumes you have the relevant GNU autotools installed on your host (autoconf, automake, etc.)

$ cat > configure.ac << EOF
AC_INIT([1], [2], [3])



AM_INIT_AUTOMAKE([1.11 foreign subdir-objects -Wall -Werror])


$ cat > Makefile.am << EOF
bin_PROGRAMS= helloer
helloer_SOURCES= helloer.c
$ autoreconf -i

Ok, we should have a simple GNU autotools skeleton in place. Let's test it on the host again.

$ ./configure
$ make
$ ./helloer

Seems to work. Let's clean the result, and cross-compile it for Rumprun. Assuming your tools are in the path, the only thing that you need to specify to a GNU configure script is the correct --host parameter, and everything else should, barring some potential intricacies, just work. The correct value for --host, as you recall, was printed by build-rr:

$ make distclean
$ ./configure --host=x86_64-rumprun-netbsd
$ make

Now all that is left is baking and running:

$ rumprun-bake hw_generic helloer-autoconf.bin helloer-rumprun
$ rumprun qemu -i helloer-autoconf.bin

4.2 CMake

We'll add CMake support to our example program and show how to build with it. This section assumes you have a recent version of cmake installed on your host.

$ cat > CMakeLists.txt << EOF
cmake_minimum_required(VERSION 2.8)
add_executable(helloer helloer.c)

Like before, let's test it on the host first:

$ mkdir host_bld
$ cd host_bld
$ cmake -G "Unix Makefiles" ..
$ make
$ ./helloer

Seems to work. We can now cross-compile it for Rumprun. Assuming the tools are in PATH, the only thing you need to specify to cmake is the toolchain file which is done through the CMAKE_TOOLCHAIN_FILE define. Like so:

$ cd ..
$ mkdir rumprun_bld
$ cd rumprun_bld
$ cmake -G "Unix Makefiles" -DCMAKE_TOOLCHAIN_FILE=../../rumprun/rumprun-x86_64/share/x86_64-rumprun-netbsd-toolchain.cmake ..
$ make

Now, bake and run:

$ rumprun-bake hw_generic helloer.bin helloer
$ rumprun qemu -i helloer.bin

If your Rumprun platform is fixed then you can bake as part of the build by passing in the platform. Change CMakeLists.txt to read like this:

cmake_minimum_required(VERSION 2.8)
set(PROG helloer)
add_executable(${PROG} ${PROG}.c)
    add_custom_command(TARGET ${PROG}
                COMMAND rumprun-bake ${RUMPRUN_PLATFORM} ${PROG}.bin ${PROG}
                COMMENT "baking ${PROG}.bin for ${RUMPRUN_PLATFORM}")

On the host, you can build and run as before. For Rumprun cross-compilation, we need to pass RUMPRUN_PLATFORM set to one of the targets supported by rumprun-bake (rumprun-bake list should give a list of supported targets for baking):

$ mkdir rumprun_bld
$ cd rumprun_bld
$ cmake -G "Unix Makefiles" -DCMAKE_TOOLCHAIN_FILE=../../rumprun/rumprun/share/x86_64-rumprun-netbsd-toolchain.cmake -DRUMPRUN_PLATFORM=hw_generic ..
$ make

and you should see output like:

Scanning dependencies of target helloer
[100%] Building C object CMakeFiles/helloer.dir/helloer.c.obj
Linking C executable helloer
baking helloer.bin for hw_generic

!!! NOTE: rumprun-bake is experimental. syntax may change in the future

[100%] Built target helloer

Run it as before:

$ rumprun qemu -i helloer.bin


Other cross-compile compatible build frameworks will work roughly the same way as the ones we covered, usually by setting the toolchain in the environment. Look at the Rumprun-packages repo for hints and clues on how to handle other build systems. Also, you are invited to add coverage to this tutorial.


Please try the above techniques for building some of your favorite software as Rumprun unikernels, and report success or failure to the rumpkernel community.