dead or alive: Linux LibOS project in 2016 #1
Comments
This was referenced Jan 18, 2016
|
Hi Hajime, |
We've (locally) started the DCE and LKL integration: not sure how log it'll be but will publish soon once we got a shape. |
|
Hi, |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Hello folks,
happy new year 2016 !
I've been collecting a couple of news on the Linux libos project since last year but I didn't express much information after the last patch proposal. here is a summary of what the project is going on and I hope you will find a good direction on your year 2016.
I'd be happy for any of your comments and patches are always welcome as we're an open source project !
rump kernel integration
While I was working on cleaning up my libos patches (lkml post) to address feedback from developers, I've been playing with rump kernel and Linux libos for the further opportunities which will be definitely useful with this particular library (i.e., liblinux). I have discussed with the rump kernel developers (Antti Kantee and Justin Cormack especially) to seek how linux libos could be like NetBSD rump kernel.
The main ideas of using the feature in Linux are:
system call proxy
I started to use rump kernel code in Linux libos from this system call library. This is a natural integration for NUSE since a process hijacked with userspace network stack doesn't have any ways to configure the network stack with standard tools like
ipcommand (more detail figure below). The proxy library enables us to use host programs such as iproute2 to interact with a NUSE process as an inter process communication.librumpuser
The next and the biggest step is to re-implement Linux libos on top of rump hypercall. This will make Linux (libos) kernel be a real rump kernel which may run on various underlying platforms such as Xen, bare metal, Genode OS, or POSIXy platforms as a baby of anykernel architecture. My attempt was in libos-nuse/net-next-nuse:rump-hypercall repository, which now is obsolete due to the newer codebase (see more detail below).
rumprun
Once I've implemented a rump hypercall interface, it is relatively easy to integrated as a rumprun unikernel with Linux libos. Although the integration was only with _nolibc_ mode of rumprun, it proved the benefit of running various platforms. The integration required an implementation of a PCI bus I/O operation to work with the virtio_net driver of Linux kernel (drivers/net/virtio_net.c).
The view of this software stack looks like the below, where
#4 rumpkernelpart can be Linux or NetBSD (at the present moment).2 PRs (rumpkernel/buildrump.sh#78, rumpkernel/rumprun#65) are under review but may need to work on later.
Some screencast is available on youtube.
frankenlibc
My next direction was a trial of another rump hypercall implementation for POSIXy platform (and additionally arm platform), with frankenlibc. The hypercall is implemented as the original one but gives a special procedure in a link process so that both the hypercall implementation and upper userspace applications can use standard libc in different namespaces, which is useful to run on various POSIXy operating system (e.g., Linux, xyzBSD). Instead of namespacing by the
objcopytool, the frankenlibc toolkit provides a namespace for rump hypercall with#defineand the other (for upper application) as-is.As an output of this frankenlibc toolkit, a single libc.a and required build environment (compiler with configuration files) will be provided so that any application developers can build an application with a virtualized environment and the specific unikernel.
musl-libc
With this extended frankenlibc, I also made a port of musl libc in my repository to integrate with frankenlibc. With this, existing applications like
nginxcan be cleanly built with the custom libc and a build system which embedded with this custom kernel for an application.where are those going to ?
while the initial libos patches only has a hijacked library to replace network stack with its own, and this case is sometime useful because it's instantly deployable, this architecture is not well feasible and faces a lot of issues such as un-hijackable symbols in existing host libraries, difficult translations of function signatures on non-Linux hosts, hard to support
sysctlcommand due to the lack of /proc fs, etc. I tentatively concluded that creating a clean environment for the particular architecture (arch/lib or arch/lkl: more detail later) is the way to be sustainable for a long time rather than introducing an ad-hoc solution.With this, I'm going for making Linux libos to be a rump kernel.
The choice of rumprun or frankenlibc unikernel is under consideration. Will put more news in near future.
LKL (Linux kernel library)
The next big news is the patch proposal of Linux kernel library (LKL). The design and architecture of LKL is quite the same as libos and rump kernel, based on providing host_ops interfaces for various platforms, which is identical part of rump hypercall interface. It currently supports a couple of platforms including Linux, FreeBSD, and Windows (mingw).
I've been aware of its code and paper since a while ago but didn't see any activities these days: that's why I started the libos project. After looking at the LKL patch, I decided to go with them: port the libos and NUSE code into this because the design of LKL is quite clean and addresses a couple of issues such as existence of stub code, non-kbuild based build system, which the libos patch got during the patch proposal.
So I've been used lkl-linux repository since rumprun and frankenlibc integrations (above) and not almost updated my previous libos code (net-next-nuse) to the upstream.
There is still a big missing feature with LKL, which is ns-3 network simulator integration and is useful for deterministic test framework with complex network configurations. This requires the time-warping functionality, which underlying clock can skip (or warp) the time advances IF there is no scheduled events in a specific time duration. I'm not still sure how to implement this on LKL without stubbing (i.e., re-implementing timer-related functions) but may come up in a future.
Multipath TCP with libos
There is another interesting attempt of userspace network stack, with an out-of-tree implementation of Multipath TCP. The discussion on the mptcp mailing list brought me mptcp-enabled iperf without installing mptcp-enable Linux kernel on a host system. Further effort is needed to be more sustainable solutions but it works (tm)!
My trial was with libos (net-next-nuse) but it should also work with LKL.
Next steps in 2016 ?
All of above are experimental trials to study whether those interesting stuffs are possible or not (and I think the answers were yes). The linux libos project will go for this direction and make those experimental code be available more broader ways. Some of the stuffs are step-backs from previous libos code but using LKL code is the way to be sustainable and maintainable of this magic in the middle.
I will work on more to clean up the code under github.com/libos-nuse/ and eventually upstream to lkl code.
For the coming event, I will give a tutorial on netdev1.1 conference, will be held on February 2016. The talk will focus on a testing framework based on libos with Direct Code Execution (ns-3 network simulator integration) but will cover a bit about recent LKL and libos activities.
code repository of libos (as of Jan. 2016)
The text was updated successfully, but these errors were encountered: