Linux kernel source tree
C Assembly C++ Objective-C Makefile Perl Other
Switch branches/tags
Nothing to show
Clone or download
#452 Compare This branch is 799 commits ahead, 40396 commits behind torvalds:master.
thehajime Merge pull request #445 from libos-nuse/feature-clear-cache
lkl: add cache version to the source cache
Latest commit 86dd3af Jun 27, 2018
Permalink
Failed to load latest commit information.
.circleci lkl: add cache version to the source cache Jun 26, 2018
Documentation create symlink of gsed as sed and fix FreeBSD build instruction in RE… Jun 26, 2018
LICENSES LICENSES: Add MPL-1.1 license Jan 6, 2018
arch Merge pull request #440 from libos-nuse/feature-custom-make-config Jun 12, 2018
block Merge tag 'for-linus-20180302' of git://git.kernel.dk/linux-block Mar 2, 2018
certs certs/blacklist_nohashes.c: fix const confusion in certs blacklist Feb 21, 2018
crypto Merge tag 'v4.16' into merge-linux-4.16 May 14, 2018
drivers Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net Mar 31, 2018
firmware kbuild: remove all dummy assignments to obj- Nov 18, 2017
fs Merge tag 'v4.16' into merge-linux-4.16 May 14, 2018
include Merge tag 'v4.16' into merge-linux-4.16 May 14, 2018
init Merge tag 'v4.16' into merge-linux-4.16 May 14, 2018
ipc ipc/shm.c: add split function to shm_vm_ops Mar 28, 2018
kernel Merge tag 'v4.16' into merge-linux-4.16 May 14, 2018
lib Merge tag 'v4.16' into merge-linux-4.16 May 14, 2018
mm mm/kmemleak.c: wait for scan completion before disabling free Mar 28, 2018
net Merge tag 'kbuild-fixes-v4.16-3' of git://git.kernel.org/pub/scm/linu… Mar 31, 2018
samples Merge tag 'seccomp-v4.16-rc4' of https://git.kernel.org/pub/scm/linux… Feb 27, 2018
scripts Merge tag 'v4.16' into merge-linux-4.16 May 14, 2018
security integrity/security: fix digsig.c build error with header file Feb 23, 2018
sound ALSA: pcm: potential uninitialized return values Mar 27, 2018
tools create symlink of gsed as sed and fix FreeBSD build instruction in RE… Jun 26, 2018
usr initramfs: fix initramfs rebuilds w/ compression after disabling Nov 3, 2017
virt Merge tag 'kvm-arm-fixes-for-v4.16-2' of git://git.kernel.org/pub/scm… Mar 15, 2018
.cocciconfig scripts: add Linux .cocciconfig for coccinelle Jul 22, 2016
.get_maintainer.ignore Add hch to .get_maintainer.ignore Aug 21, 2015
.gitattributes .gitattributes: set git diff driver for C source code files Oct 8, 2016
.gitignore .gitignore: ignore ASN.1 auto generated files Feb 14, 2018
.mailmap mailmap: Update email address for Gregory CLEMENT Mar 9, 2018
COPYING [PATCH] update FSF address in COPYING Sep 10, 2005
CREDITS MAINTAINERS: update TPM driver infrastructure changes Nov 10, 2017
Kbuild Merge tag 'kbuild-v4.15' of git://git.kernel.org/pub/scm/linux/kernel… Nov 18, 2017
Kconfig License cleanup: add SPDX GPL-2.0 license identifier to files with no… Nov 2, 2017
MAINTAINERS Merge tag 'v4.16' into merge-linux-4.16 May 14, 2018
Makefile Merge tag 'v4.16' into merge-linux-4.16 May 14, 2018
README README: add a new README file, pointing to the Documentation/ Oct 24, 2016
README.md Redirect README.md to Documentation/lkl.txt Nov 17, 2015

README.md

Introduction

LKL (Linux Kernel Library) is aiming to allow reusing the Linux kernel code as extensively as possible with minimal effort and reduced maintenance overhead.

Examples of how LKL can be used are: creating userspace applications (running on Linux and other operating systems) that can read or write Linux filesystems or can use the Linux networking stack, creating kernel drivers for other operating systems that can read Linux filesystems, bootloaders support for reading/writing Linux filesystems, etc.

With LKL, the kernel code is compiled into an object file that can be directly linked by applications. The API offered by LKL is based on the Linux system call interface.

LKL is implemented as an architecture port in arch/lkl. It uses host operations defined by the application or a host library (tools/lkl/lib).

Supported hosts

The supported hosts for now are POSIX and Windows userspace applications.

Building LKL, the host library and LKL based tools

$ make -C tools/lkl

will build LKL as a object file, it will install it in tools/lkl/lib together with the headers files in tools/lkl/include then will build the host library, tests and a few of application examples:

  • tests/boot - a simple applications that uses LKL and exercises the basic LKL APIs

  • fs2tar - a tool that converts a filesystem image to a tar archive

  • cptofs/cpfromfs - a tool that copies files to/from a filesystem image

  • lklfuse - a tool that can mount a filesystem image in userspace, without root priviledges, using FUSE

Building LKL on FreeBSD

$ pkg install binutils gcc gnubc gmake gsed coreutils bison flex python argp-standalone

#Prefer ports binutils and GNU bc(1):
$ export PATH=/sbin:/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/usr/lib64/ccache

$ gmake -C tools/lkl

Building LKL on Ubuntu

$ sudo apt-get install libfuse-dev libarchive-dev xfsprogs

# Optional, if you would like to be able to run tests
$ sudo apt-get install btrfs-tools

$ make -C tools/lkl

# To check that everything works:
$ cd tools/lkl
$ make test

Building LKL for Windows

In order to build LKL for Win32 the mingw cross compiler needs to be installed on the host (e.g. on Ubuntu the following packages are required: binutils-mingw-w64-i686, gcc-mingw-w64-base, gcc-mingw-w64-i686 mingw-w64-common, mingw-w64-i686-dev).

Due to a bug in mingw regarding weak symbols the following patches needs to be applied to mingw-binutils:

https://sourceware.org/ml/binutils/2015-10/msg00234.html

and i686-w64-mingw32-gas, i686-w64-mingw32-ld and i686-w64-mingw32-objcopy need to be recompiled.

With that pre-requisites fullfilled you can now build LKL for Win32 with the following command:

$ make CROSS_COMPILE=i686-w64-mingw32- -C tools/lkl

Building LKL on Windows

To build on Windows, certain GNU tools need to be installed. These tools can come from several different projects, such as cygwin, unxutils, gnu-win32 or busybox-w32. Below is one minimal/modular set-up based on msys2.

Common build dependencies:

  • MSYS2 (provides GNU bash and many other utilities)
  • Extra utilities from MSYS2/pacman: bc, base-devel

General considerations:

  • No spaces in pathnames (source, prefix, destination,...)!
  • Make sure that all utilities are in the PATH.
  • Win64 (and MinGW 64-bit crt) is LLP64, which causes conflicts in size of "long" in the Linux source. Linux (and lkl) can (currently) not be built on LLP64.
  • Cygwin (and msys2) are LP64, like linux.

For MSYS2 (and Cygwin):

Msys2 will install a gcc tool chain as part of the base-devel bundle. Binutils (2.26) is already patched for NT weak externals. Using the msys2 shell, cd to the lkl sources and run:

$ make -C tools/lkl

For MinGW:

Install mingw-w64-i686-toolchain via pacman, mingw-w64-i686-binutils (2.26) is already patched for NT weak externals. Start a MinGW Win32 shell (64-bit will not work, see above) and run:

$ make -C tools/lkl

LKL hijack library

LKL hijack library (liblkl-hijack.so) is used to replace system calls used by an application on the fly so that the application can use LKL instead of the kernel of host operating system. LD_PRELOAD is used to dynamically override system calls with this library when you execute a program.

You can usually use this library via a wrapper script.

$ cd tools/lkl
$ ./bin/lkl-hijack.sh ip address show

In order to configure the behavior of LKL, a json file can be used. You can specify json file with environmental variables (LKL_HIJACK_CONFIG_FILE). If there is nothing specified, LKL tries to find with the name 'lkl-hijack.json' for the configuration file. You can also use the old-style configuration with environmental variables (e.g., LKL_HIJACK_NET_IFTYPE) but those are overridden if a json file is specified.

     $ cat conf.json
     {
       "gateway":"192.168.0.1",
       "gateway6":"2001:db8:0:f101::1",
       "debug":"1",
       "singlecpu":"1",
       "sysctl":"net.ipv4.tcp_wmem=4096 87380 2147483647",
       "boot_cmdline":"ip=dhcp",
       "interfaces":[
               {
                       "mac":"12:34:56:78:9a:bc",
                       "type":"tap",
                       "param":"tap7",
                       "ip":"192.168.0.2",
                       "masklen":"24",
                       "ifgateway":"192.168.0.1",
                       "ipv6":"2001:db8:0:f101::2",
                       "masklen6":"64",
                       "ifgateway6":"2001:db8:0:f101::1",
                       "offload":"0xc803"
               },
               {
                       "mac":"12:34:56:78:9a:bd",
                       "type":"tap",
                       "param":"tap77",
                       "ip":"192.168.1.2",
                       "masklen":"24",
                       "ifgateway":"192.168.1.1",
                       "ipv6":"2001:db8:0:f102::2",
                       "masklen6":"64",
                       "ifgateway6":"2001:db8:0:f102::1",
                       "offload":"0xc803"
               }
       ]
     }
     $ LKL_HIJACK_CONFIG_FILE="conf.json" lkl-hijack.sh ip addr s

The following are the list of keys to describe a JSON file.

  • IPv4 gateway address

    key: "gateway" value type: string

    the gateway IPv4 address of LKL network stack.

     "gateway":"192.168.0.1"
  • IPv6 gateway address

    key: "gateway6" value type: string

    the gateway IPv6 address of LKL network stack.

     "gateway6":"2001:db8:0:f101::1"
  • Debug

    key: "debug" value type: string

    Setting it causes some debug information (both from the kernel and the LKL library) to be enabled. If zero' is specified it is disabled. It is also used as a bit mask to turn on specific debugging facilities. E.g., setting it to "0x100" will cause the LKL kernel to pause after the hijack'ed app exits. This allows one to debug or collect info from the LKL kernel before it quits.

     "debug":"1"
  • Single CPU pinning

    key: "singlecpu" value type: string

    Pin LKL kernel threads on to a single host cpu. value "1" pins only LKL kernel threads while value "2" also pins polling threads.

     "singlecpu":"1"
  • SYSCTL

    key: "sysctl" value type: string

    Configure sysctl values of the booted kernel via the hijack library. Multiple entries can be specified.

     "sysctl":"net.ipv4.tcp_wmem=4096 87380 2147483647"
  • Boot command line

    key: "boot_cmdline" value type: string

    Specify the command line to the kernel boot so that change the configuration on a kernel instance. For instance, you can change the memory size with below.

     "boot_cmdline": "mem=1G"
  • Mount

    key: "mount" value type: string

     "mount": "proc,sysfs"
  • Network Interface Configuration

    key: "interfaces" value type: array of objects

    This key takes a set of sub-keys to configure a single interface. Each key is defined as follows.

         "interfaces":[{....},{....}]
    
    • Interface type

      key: "type" value type: string

      The interface type in host operating system to connect to LKL. The following example specifies a tap interface.

          "type":"tap"
    
    • Interface parameter

      key: "param" value type: string

      Additional configuration parameters for the interface specified by Interface type (type). The parameters depend on the interface type.

          "type":"tap",
          "param":"tap0"
    
    • Interface MTU size

      key: "mtu" value type: string

      the MTU size of the interface.

          "mtu":"1280"
    
    • Interface IPv4 address

      key: "ip" value type: string

      the IPv4 address of the interface. If you want to use DHCP for the IP address assignment, use "boot_cmdline" with "ip=dhcp" option.

          "ip":"192.168.0.2"
    
          "boot_cmdline":"ip=dhcp"
    
    • Interface IPv4 netmask length

      key: "masklen" value type: string

      the network mask length of the interface.

          "ip":"192.168.0.2",
          "masklen":"24"
    
    • Interface IPv4 gateway on routing policy table

      key: "ifgateway" value type: string

      If you specify this parameter, LKL adds routing policy table. And then LKL creates link local and gateway route on this table. Table SELECTOR is "from" and PREFIX is address you assigned to this interface. Table id is 2 * (interface index). This parameter could be used to configure LKL for mptcp, for example.

          "ip":"192.168.0.2",
          "masklen":"24",
          "ifgateway":"192.168.0.1"
    
    • Interface IPv6 address

      key: "ipv6" value type: string

      the IPv6 address of the interface.

          "ipv6":"2001:db8:0:f101::2"
    
    • Interface IPv6 netmask length

      key: "masklen6" value type: string

      the network mask length of the interface.

          "ipv6":"2001:db8:0:f101::2",
          "masklen":"64"
    
    • Interface IPv6 gateway on routing policy table

      key: "ifgateway6" value type: string

      If you specify this parameter, LKL adds routing policy table. And then LKL creates link local and gateway route on this table. Table SELECTOR is "from" and PREFIX is address you assigned to this interface. Table id is 2 * (interface index) + 1. This parameter could be used to configure LKL for mptcp, for example.

          "ipv6":"2001:db8:0:f101::2",
          "masklen":"64"
          "ifgateway6":"2001:db8:0:f101::1",
    
    • Interface MAC address

      key: "mac" value type: string

      the MAC address of the interface.

          "mac":"12:34:56:78:9a:bc"
    
    • Interfac neighbor entries

      key: "neigh" value type: string

      Add a list of permanent neighbor entries in the form of "ip|mac;ip|mac;...". ipv6 are supported

          "neigh":"192.168.0.1|12:34:56:78:9a:bc;2001:db8:0:f101::1|12:34:56:78:9a:be"
    
    • Interface qdisc entries

      key: "qdisc" value type: string

      Add a qdisc entry in the form of "root|type;root|type;...".

          "qdisc":"root|fq"
    
    • Interface offload

      key: "offload" value type: string

      Work as a bit mask to enable selective device offload features. E.g., to enable "mergeable RX buffer" (LKL_VIRTIO_NET_F_MRG_RXBUF) + "guest csum" (LKL_VIRTIO_NET_F_GUEST_CSUM) device features, simply set it to 0x8002. See virtio_net.h for a list of offload features and their bit masks.

          "offload":"0x8002"
    
  • Delay

    key: "delay_main" value type: string

    The delay before calling main() function of the application after the initialization of LKL. Some subsystems in Linux tree require a certain amount of time before accepting a request from application, such as delivery of address assignment to an network interface. This parameter is used in such case. The value is described as a microsecond value.

     "delay_main":"500000"

FAQ

Q: How is LKL different from UML?

A: UML prodivides a full OS environment (e.g. user/kernel separation, user processes) and also has requirements (a filesystem, processes, etc.) that makes it hard to use it for standalone applications. UML also relies heavily on Linux hosts. On the other hand LKL is designed to be linked directly with the application and hence does not have user/kernel separation which makes it easier to use it in standalone applications.

Q: How is LKL different from LibOS?

A: LibOS re-implements high-level kernel APIs for timers, softirqs, scheduling, sysctl, SLAB/SLUB, etc. LKL behaves like any arch port, implementing the arch level operations requested by the Linux kernel. LKL also offers a host interface so that support for multiple hosts can be implemented.