Skip to content
OSv, a new operating system for the cloud.
C C++ Python Java Roff Makefile Other
Branch: master
Clone or download

Latest commit

wkozaczuk httpserver: fix monitoring module Makefile to properly generate JSON …
…cc files

Signed-off-by: Waldemar Kozaczuk <jwkozaczuk@gmail.com>
Latest commit 60a5b0e Apr 8, 2020

Files

Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
.travis/cirp travis CI: setup nightly publishing of OSv artifacts Mar 9, 2020
apps @ 24577fb apps: upgraded to the latest Feb 17, 2020
arch memmove: fix a bug in memmove_backwards Mar 1, 2020
bsd zfs: default device name to vblk0.1 if vdev_path different than /dev/… Dec 23, 2019
compiler osv: use __ASSEMBLY__ instead of ASSEMBLY Jan 24, 2017
conf build: move arch-specific configuration to conf/$(arch).mk Jun 18, 2014
core mempool: use map_anon() for large allocations or when memory is fragm… Mar 29, 2020
docker travis CI: setup nightly publishing of OSv artifacts Mar 9, 2020
documentation fix permissions on LICENSE-prex Feb 6, 2018
drivers virtio: extract common logic to map capability bar Mar 19, 2020
external fs: move nfs support out of kernel in lieu of a separate pluggable mo… Mar 27, 2020
fastlz Implement almost-in-place kernel decompression Sep 29, 2018
fs fs: move nfs support out of kernel in lieu of a separate pluggable mo… Mar 27, 2020
images Remove deprecated "mgmt" submodule Jul 26, 2016
include virtio-fs: initial implementation of read-only subset Feb 9, 2020
libc libc: refine how timerfd_* functions handle wall clock jumps Mar 15, 2020
licenses travis CI: setup nightly publishing of OSv artifacts Mar 9, 2020
modules httpserver: fix monitoring module Makefile to properly generate JSON … Apr 8, 2020
musl @ 372a948 Add musl git submodule Aug 4, 2014
scripts tests: move nfs testing out to a separate module Mar 27, 2020
static/etc fs: add subset of sysfs implementation needed by numa library Sep 5, 2019
tests mempool: use map_anon() for large allocations or when memory is fragm… Mar 29, 2020
tools fs: move nfs support out of kernel in lieu of a separate pluggable mo… Mar 27, 2020
.gitignore externals: enhance setup.py to download aarch64 gcc and boost code an… Mar 1, 2020
.gitmodules fs: move nfs support out of kernel in lieu of a separate pluggable mo… Mar 27, 2020
.travis.yml travis: clean the build tree just in case Mar 13, 2020
CODINGSTYLE.md Add line breaks to CODINGSTYLE Jan 26, 2015
CONTRIBUTING Add "CONTRIBUTING" file Sep 15, 2013
Doxyfile doxygen: add include strip path Sep 16, 2014
LICENSE Add LICENSE files to the main project Sep 15, 2013
Makefile fs: move nfs support out of kernel in lieu of a separate pluggable mo… Mar 27, 2020
PORTING build.mk: enable first AArch64 stubbed support Apr 2, 2014
README.md Added patch/pull requests clarification to the main README Nov 21, 2019
bootfs.S Encapsulate boot filesystem image in an object file for direct linking Dec 23, 2012
bootfs.manifest.skel Add vdso fast syscall compatibility library Mar 18, 2018
bootfs_empty.manifest.skel Changed build script to use empty bootfs for ROFS images Jul 29, 2018
config.json Remove deprecated "mgmt" submodule Jul 26, 2016
dummy-shlib.cc loader: add a dummy shared library to the build Dec 23, 2012
gen-ctype-data.cc locale: fix std::toupper(), std::tolower(), and std::regex Feb 1, 2016
linux.cc syscall: add set_mempolicy and sched_setaffinity needed by libnuma Aug 30, 2019
loader.cc zfs: do not import extra pools unless requested Dec 23, 2019
runtime.cc Refine confstr() to conform more closely to Linux spec Jul 11, 2019
usr.manifest.skel fs: move nfs support out of kernel in lieu of a separate pluggable mo… Mar 27, 2020
usr_ramfs.manifest.skel fs: move nfs support out of kernel in lieu of a separate pluggable mo… Mar 27, 2020
usr_rofs.manifest.skel fs: move nfs support out of kernel in lieu of a separate pluggable mo… Mar 27, 2020

README.md

OSv was originally designed and implemented by Cloudius Systems (now ScyllaDB) however currently it is being maintained and enhanced by a small community of volunteers. If you are into systems programming or want to learn and help us improve OSv please contact us on OSv Google Group forum. For details on how to format and send patches, please read this wiki (we do NOT accept pull requests).

OSv

OSv is an open-source versatile modular unikernel designed to run unmodified Linux applications securely on micro-VMs in the cloud. Built from the ground up for effortless deployment and management of microservices and serverless apps, with superior performance.

OSv has new APIs for new applications, but also runs unmodified x86-64 Linux binaries as is, which effectively makes it a Linux binary compatible unikernel (for more details about Linux ABI compatibility please read this doc). In particular OSv can run many managed language runtimes including JVM, Python 2 and 3, Node.JS, Ruby, Erlang, and applications built on top of one. It can also run applications written in languages compiling directly to native machine code like C, C++, Golang and Rust as well as native images produced by GraalVM.

OSv can boot as fast as ~5 ms on Firecracker using as low as 15 MiB of memory. OSv can run on many hypervisors including QEMU/KVM, Firecracker, Xen, VMWare, VirtualBox and Hyperkit as well as open clouds like AWS EC2, GCE and OpenStack.

For more information about OSv, see the main wiki page and http://osv.io/.

Building and Running Apps on OSv

In order to run an application on OSv, one needs to build an image by fusing OSv kernel and the application files together. This, in high level can be achieved in two ways:

  • by using the script build that builds the kernel from source and fuses it with application files
  • by using the capstan tool that uses pre-built kernel and combines it with application files to produce final image

If your intention is to try to run your app on OSv with least effort, you should pursue the capstan route. For introduction please read this crash course and for more details about capstan read this documentation.

If you are comfortable with make and GCC toolchain and want to try the latest OSv code, then you should read remaining part of this page to guide you how to setup your development environment and build OSv kernel and application images.

Setting up development environment

OSv can only be built on a 64-bit x86 Linux distribution. Please note that this means the "x86_64" or "amd64" version, not the 32-bit "i386" version.

In order to build OSv kernel you need a physical or virtual machine with Linux distribution on it and GCC toolchain and all necessary packages and libraries OSv build process depends on. The easiest way to set it up is to use Docker files that OSv comes with. You can use them to build your own Docker image and then start it in order to build OSv kernel inside of it.

Otherwise, you can manually clone OSv repo and use setup.py to install GCC and all required packages, as long as it supports your Linux distribution and you have both git and python 2.7 installed on your machine:

git clone https://github.com/cloudius-systems/osv.git
cd osv && git submodule update --init --recursive
./scripts/setup.py

The setup.py recognizes and installs packages for number of Linux distributions including Fedora, Ubuntu, Debian, LinuxMint and RedHat ones (Scientific Linux, NauLinux, CentOS Linux, Red Hat Enterprise Linux, Oracle Linux). Please note that only Ubuntu and Fedora support is actively maintained and tested so your milage with other distributions may vary.

Building OSv kernel and creating images

Building OSv is as easy as using the shell script build that orchestrates the build process by delegating to the main makefile to build the kernel and by using number of Python scripts like module.py to build application and fuse it together with the kernel into a final image placed at ./build/release/usr.img (or ./build/$(arch)/usr.img in general). Please note that building app does not necessarily mean building from source as in many cases the app files would be simply located on and taken from the Linux build machine (see manifest_from_host.sh for details).

The build script can be used like so per the examples below:

# Create default image that comes with command line and REST API server
./scripts/build

# Create image with native-example app
./scripts/build -j4 fs=rofs image=native-example

# Create image with spring boot app with Java 10 JRE
./scripts/build JAVA_VERSION=10 image=openjdk-zulu-9-and-above,spring-boot-example

 # Create image with 'ls' executable taken from the host
./scripts/manifest_from_host.sh -w ls && ./script/build --append-manifest

# Create test image and run all tests in it
./script/build check

# Clean the build tree
./script/build clean

Command nproc will calculate the number of jobs/threads for make and scripts/build automatically. Alternatively, the environment variable MAKEFLAGS can be exported as follows:

export MAKEFLAGS=-j$(nproc)

In that case, make and scripts/build do not need the parameter -j.

For details on how to use the build script, please run ./scripts/build --help.

The .scripts/build creates the image build/last/usr.img in qcow2 format. To convert this image to other formats, use the scripts/convert tool, which can create an image in the vmdk, vdi or raw formats. For example:

scripts/convert raw

By default OSv builds kernel for x86_64 architecture but it is also possible to build one for ARM by adding arch parameter like so:

./scripts/build arch=aarch64

Please note that even though the aarch64 version of OSv kernel should build fine, most likely it will not run as the ARM part of OSv has not been well maintained and tested due to the lack of volunteers.

Running OSv

Running an OSv image, built by scripts/build, is as easy as:

./scripts/run.py

By default, the run.py runs OSv under KVM, with 4 VCPUs and 2GB of memory. You can control these and tens of other ones by passing relevant parameters to the run.py. For details on how to use the script please run ./scripts/run.py --help.

The run.py can run OSv image on QEMU/KVM, Xen and VMware. If running under KVM you can terminate by hitting Ctrl+A X.

Alternatively you can use ./scripts/firecracker.py to run OSv on Firecracker. This script automatically downloads firecracker and accepts number of parameters like number ot VCPUs, memory named exactly like run.py does.

Please note that in order to run OSv with best performance on Linux under QEMU or Firecracker you need KVM enabled (this is only possible on physical Linux machines, EC2 bare metal instances or VMs that support nested virtualization with KVM on). The easiest way to verify KVM is enabled is to check if /dev/kvm is present and your user can read from and write to it. Adding your user to the kvm group may be necessary like so:

usermod -aG kvm <user name>

For more information about building and running JVM, Node.JS, Python and other managed runtimes as well as Rust, Golang or C/C++ apps on OSv, please read this wiki page. For more information about various example apps you can build and run on OSv, please read the osv-apps repo README.

Networking

By default the run.py starts OSv with user networking/SLIRP on. To start OSv with more performant external networking:

sudo ./scripts/run.py -n -v

The -v is for kvm's vhost that provides better performance and its setup requires a tap and thus we use sudo.

By default OSv spawns a dhcpd that automatically configures the virtual nics. Static config can be done within OSv, configure networking like so:

ifconfig virtio-net0 192.168.122.100 netmask 255.255.255.0 up
route add default gw 192.168.122.1

Test networking:

test invoke TCPExternalCommunication

Debugging, Monitoring, Profiling OSv

  • OSv can be debugged with gdb; for more details please read this wiki
  • OSv kernel and application can be traced and profiled; for more details please read this wiki
  • OSv comes with the admin/montioring REST API server; for more details please read this and that wiki page.
You can’t perform that action at this time.