v0.9.2

• Fixed CPU usage issue caused by PIT timeouts
• Added optional file system caching
• Various fixes and cleanups.

v0.9.1

• Fixed DHCP
• Relaxed too strict IP filter
• Major improvement to testrunner
• Various bugfixes and minor improvements

v0.9.0

Platform support

• OpenStack support using IDE boot device
• Injectable support for platform specific protocols, like Unik instance registration

Instrumentation

• Backtrace from ELF symbols
• Context buffer for crashes
• Real-time stack sampling for profiling

Boot

• Stack and heap randomization
• Multiboot support

Kernel features

• ACPI tables, ACPI shutdown
• xAPIC support
• Basic SMP support
• MSI-X support
• APIC timer
• Memory map of named, fixed memory ranges
• Improved timer system
• RTC that is auto-calibrated and inexpensive
• New driver injection system
• Statman: centrally located statistics for anything

Networking

• Inet4 is now driver independent
• TCP modularized with lots of API changes and improvements

Tooling and testing

• Building kernel with minimal code and stripped symbols
• Improvements to install scripts
• Automatic builds using Jenkins for PRs and main branches
• New automatic test suite for unit testing, using LEST

Misc

• IPv4 addresses can now be created from string
• Code refactoring to make use of new C++ features and Core Guidelines
• Lots of bugfixes

Getting stable

A few bugs were found and fixed after running a new stress test. The OS seems very stable now, having survived a continous flood of Arp, ICMP and HTTP packets, run in parallel over ~9 hours. The VM showed no signs of having problems and gave a HTTP request rate of 3327.1 conn/s. after the flood had ended.

Test results:

• Ping-flood: $ sudo ping -f 10.0.0.42
101095327 packets transmitted, 101095294 received, 0% packet loss, time 31206908ms
• Arp-flood: $ sudo arping -w 100 10.0.0.42
1715415846 packets transmitted, 857201224 packets received, 50% unanswered (0 extra)
• httperf: $ time httperf --hog --server 10.0.0.42 --num-conn 100000000

Total: connections 382691 requests 382690 replies 382690 test-duration 31343.418 s` 
...
Errors: total 0 client-timo 0 socket-timo 0 connrefused 0 connreset 0
Errors: fd-unavail 0 addrunavail 0 ftab-full 0 other 0

real    522m53.450s
user    74m18.515s
sys     448m57.559s

The connection rate on http was very poor, 12.2 req/s., but I guess that’s quite understandable. Asking httperf to do a 100 million requests as fast as it possible will very likely just suck up all the available outgoing sockets / file descriptors.

• VM diagnostics:
  Current memory usage: 1122304 b, (1.122304 MB)
Recv: 23726780 Sent: 277016620

Experiment details:

• The three tests were run individually in parallell, from separate processes / screens, towards the same, singular VM with one CPU core. The tests were stopped manually after having run overnight.
• After ending the flood, without restarting the VM, it gave the following response:
  $ time httperf --hog --server 10.0.0.42 --num-conn 1000

Connection rate: 3327.1 conn/s (0.3 ms/conn, <=1 concurrent connections)
Connection time [ms]: min 0.3 avg 0.3 max 1.6 median 0.5 stddev 0.0
Connection time [ms]: connect 0.2
Connection length [replies/conn]: 1.000

Request rate: 3327.1 req/s (0.3 ms/req)
Request size [B]: 62.0

Reply rate [replies/s]: min 0.0 avg 0.0 max 0.0 stddev 0.0 (0 samples)
Reply time [ms]: response 0.1 transfer 0.0
Reply size [B]: header 234.0 content 489.0 footer 0.0 (total 723.0)
Reply status: 1xx=0 2xx=1000 3xx=0 4xx=0 5xx=0

CPU time [s]: user 0.09 system 0.21 (user 29.5% system 70.2% total 99.7%)
Net I/O: 2553.3 KB/s (20.9*10^6 bps)

Errors: total 0 client-timo 0 socket-timo 0 connrefused 0 connreset 0
Errors: fd-unavail 0 addrunavail 0 ftab-full 0 other 0

NOTE: We've seen much higher connection rates on other platforms (e.g. 16000 conn/s. has been reported), so the numbers above should not be taken as measure of IncludeOS performance.

Getting somewhere

Block device, filesystem and memdisk

• Finished filesystem interface with sync and async calls
• Support for custom filesystems
• FAT 12/16/32 filesystem implementation
• Added the Memdisk disk device, linked in as binary section
• Added Virtio Block device driver
• Added Virtio Console device driver
• Automated MemDisk builder from disk image file
• Async class for automatic handling of async file transfers

TCP - more or less complete rewrite

• State logic is now separated into classes
• "Socket" replaced with "Connection".
• Fully implemented RFC 793 "TransmissionControl Protocol".
• Retransmission, round trip time calculation (RFC 6298)
• Support options - MSS implemented.
• Congestion Control (New Reno) (RFC 5681 & 6582)
• Fixed checksum bug causing instability
• Send queue - queue several write requests which will be stored until fully acknowledged to be used for retransmission. Possible to avoid any copy by assigning a shared pointer to data.
• Fair packet distribution among connections.
• Packets with payload now released immediately after copying data once to a shared buffer, released to the user when subscribed.
• More events to subscribe to - OnConnect, OnDisconnect among others.

Time

• Async timers, one-shot and repeating (using PIT)
• We finally have a wall clock! (RTC via CMOS)

Test

• Added tests covering the major kernel and networking features
• New system for automating tests written in python
• Added support for LEST unit test framework (used in e.g. ./test/GSL)

Misc

• UDP now has send queue to help maximize bandwidth
• Improvements in packet- and buffer-management increasing bandwidth
• Added an optional async terminal with TCP and Serial backend, support for Telnet
• C++ GSL now included with project, with some limited usage in the kernel
• New safer virtio interface with GSL
• Pretty boot
• Pretty boot screen
• Tons of bugfixes
• Updated to llvm 3.8, both libc++ and clang, also for OS X.

Caveats:

• You currently can't use virtio-block devices together with virtio-net in Qemu, as they'll get the same IRQ number. This has been sorted in dev, but it's involved.
• The CMOS wall clock is super expensive - I counted about 200k cycles (!). This is because it has to do a lot of single-byte in- and out-instructions which all cause vm exits. We're working on a better way to keep time, possibly synchronizing with CMOS only now and then.

Up next:

On our dev-branch we've replaced the classic PIC-based interrupt system with APIC / IOAPIC and MSI-x. We expect this to give a major boost to resource efficiency and it does make it possible to run virtio-net together with virtio-block. We're also working on a lot of web service stuff, but that's mostly in stealth-mode as of yet - we'll open it up when we have something we think you'll like.

Thanks a lot to the budding community of contributors - we have a long way to go but it feels like we're getting somewhere :-)

v0.8.0 Release Candidate 2

Detailed release notes will come with the actual release.

v0.8.0 Release Candidate 1

Detailed release notes will come with the actual release.

IncludeOS is now free and open source

In this pre-release:

• DHCP
• ARP-resolution
• UDP Sockets (TCP sockets will be changed to match)
• New network buffer management system
• Rewritten the Dev-class with templatized getters
• Improved virtualbox run-script (etc/vboxrun.sh)
• Jenkins integration
• VGA output (somewhat sketchy - rely on the serial port)
• Successful build on Apple OSX (try ./etc/install_mac.sh)
• Lots of cleanups, and pretty printing
• Apache v2.0 license

Partial TCP, Timers, Virtualbox +++

• Partial TCP - enough to show a static webpage
• Asynchronous timers using the PIT (Polite ones - no context switches). You'll find them in "class_pit.hpp".
• Major directory structure cleanup, mostly split contents in ./src into folders.
• Lots of work towards IPv6
• VirtualBox support is back - but note, ARP is not done, so don't expect proper networking.
• Install-script finally works - way differently
  • We now install to - and build from $HOME/IncludeOS_install. Much better than installing by default to /usr/local/IncludeOS/. You can move that folder anywhere - just update the first line in the makefiles. (Yes, environment vars are probably what we'll do later, but we're also getting a package manager...)
  • The contents of the install-folder gets gzipped into a binary release bundle. This bundle can be extracted as-is, and there you have everything you need to build IncludeOS (except for local dependencies such as clang). No need to build a cross-compiler!

LLVM's libc++, clang++ and exceptions,

Finally you can throw and catch all the exceptions you like. We don't explicitly throw any inside the kernel, but containers might. You can also use std::cout << "standard strings" << std::endl, maps, vectors, lambdas etc.

NOTE:

• We still don't have threads, so no std::async.
• Locale-stuff is pretty much duct-taped in - newlib doesn't have proper support
• No timers yet, but we're working on it, towards the TCP milestone.
• Binary sizes increase pretty drastically - "Hello world" is currently at about 500Kb. As a consequence, we'll probably add EASTL back in as a module, and make an optional build-path for that.
• long double math functions are not implemented - the interface is there to satisfy the cmath header, but no implementation. We think this is better than just using them as wrappers around the double versions - which you can use.