Skip to content

Commit

Permalink
Issue #350: add support for 3.2.0
Browse files Browse the repository at this point in the history
This commit tackles the following issues:
* Update bootloader
* Remove hlt when idle and fix bug in ThreadCreate()
* Issue #418: Update README.md
  • Loading branch information
MatiasVara committed Nov 22, 2020
1 parent e0bbb6c commit 5718bf7
Show file tree
Hide file tree
Showing 17 changed files with 340 additions and 8,232 deletions.
77 changes: 61 additions & 16 deletions README.md
@@ -1,30 +1,75 @@
# Toro Kernel ![build passing](https://api.travis-ci.org/torokernel/torokernel.svg?branch=master)
# ToroMicroVM ![build passing](https://api.travis-ci.org/torokernel/torokernel.svg?branch=master)
## Introduction
Toro is a kernel dedicated to run a single application. The kernel compiles together with the user application and this results in a binary that user can run on top of a hypervisor, e.g., KVM, Xen, VirtualBox, or baremetal. To know more about Toro, visit the [blog](http://www.torokernel.io) and the [wiki](https://github.com/MatiasVara/torokernel/wiki). In the repository, the tag ToroKernel points to last commit for the ToroKernel project. Since this tag, the project focuses on microVM technologies like QEMU microvm or Firecracker, and on the devices virtio-fs for filesystem and virtio-vsocket for networking.
ToroMicroVM is a unikernel dedicated to deploy microservices as microVMs. ToroMicroVM leverages on virtio-fs and virtio-vsocket to provide a minimalistic architecture. Microservices are deployed as Toro guests in which binaries and files are distributed in a Ceph cluster. The common fileystem allows to easely launch microvms from any node of the cluster.

## Features
* Support x86-64 architecture
* Support up to 512GB of RAM
* Support KVM, Xen, HyperV, VirtualBox
* Support QEMU microvm and Firecracker
* Support QEMU-KVM microvm and Firecracker
* Cooperative and I/O bound threading scheduler
* Virtual FileSystem
* Network Stack
* Network drivers:
- Virtio-vsocket, Virtio-net, E1000, NE2000
* Disk drivers:
- Virtio-blk, ATA
* FileSystem drivers:
- Ext2, Fat, VirtioFS
* Support virtio-vsocket for networking
* Support virtio-fs for filesystem
* Fast boot up
* Tiny image

## Examples
The repository of Toro includes examples that show basic functionalities of the kernel. These examples are in the **examples** directory. Each example contains the instruction to compile it and run it on QEMU-KVM. We recommend to start with the **HelloWorld** example. Before go to the example, you need to install Lazarus and QEMU-KVM:
## How compile ToroMicroVM?

`apt-get install lazarus`
### Step 1. Install Freepascal
```bash
apt-get install fpc
```
You have to install version 3.2.0.

### Step 2. Build latest Qemu-KVM
```bash
apt-get install libcap-dev libcap-ng-dev libcurl4-gnutls-dev libgtk-3-dev libglib2.0-dev libpixman-1-dev libseccomp-dev -y
git clone https://github.com/qemu/qemu.git qemuforvmm
cd qemuforvmm
mkdir build
cd build
../configure --target-list=x86_64-softmmu
make
```

To install qemu-kvm and build a CephFS cluster you can follow these [instructions](https://github.com/torokernel/torocloudscripts).
### Step 3. Get ToroMicroVM
```bash
git clone https://github.com/torokernel/torokernel.git
```

### Step 4. Get system.pas for ToroMicroVM
```bash
git clone https://github.com/torokernel/freepascal.git -b fpc-3.2.0 fpc-3.2.0
```

### Step 5. Edit path to Qemu and FPC in CloudIt.sh
Go to `torokernel/examples` and edit `CloudIt.sh` to set the correct paths to Qemu and fpc. Optionally, you can install vsock-socat from [here](https://github.com/stefano-garzarella/socat-vsock).

## Run the HelloWorld Example
You have just to go to `examples/HelloWorld/` and execute:
```bash
../CloudIt.sh HelloWorld "-dUseSerialasConsole"
```

## Run the StaticWebServer Example
To run the StaticWebServer example, you have first to compile vsock-socat and virtiofds. The latter is built during the building of Qemu. In a terminal, launch vsock-socat by executing:

```bash
./socat TCP4-LISTEN:4000,reuseaddr,fork VSOCK-CONNECT:5:80
```
In a second terminal, execute:

```bash
./virtiofsd -d --socket-path=/tmp/vhostqemu1 -o source=/root/qemulast/build/testdir/ -o cache=always
```

Replace `source` with the directory to serve. Finally, launch the static webserver by executing:

```bash
../CloudIt.sh StaticWebServer "-dUseSerialasConsole -dShutdownWhenFinished"
```

## Create your own distributed filesystem with CephFS
To create a CephFS cluster you can follow these [instructions](https://github.com/torokernel/torocloudscripts).

## Contributing
You have many ways to contribute to Toro. One of them is by joining the Google Group [here](https://groups.google.com/forum/#!forum/torokernel). In addition, you can find more information [here](
Expand Down
6 changes: 3 additions & 3 deletions boot/pvhbootloader.asm
Expand Up @@ -18,7 +18,7 @@
;
; You should have received a copy of the GNU General Public License
; along with this program. If not, see <http://www.gnu.org/licenses/>.
global _start
global _pvstart
STACKSPACE equ 0x4000
IDT equ 3020h

Expand Down Expand Up @@ -67,7 +67,7 @@ mbinfo:
dd 0

[BITS 32]
_start:
_pvstart:
; load gdt
cli
mov esp, _sys_stack
Expand Down Expand Up @@ -268,7 +268,7 @@ SECTION .note
SECTION .note
elfnotes:
XEN_ELFNOTE_PHYS32_ENTRY equ 18
ELFNOTE "Xen", XEN_ELFNOTE_PHYS32_ENTRY, _start
ELFNOTE "Xen", XEN_ELFNOTE_PHYS32_ENTRY, _pvstart

SECTION .bss
_sys_stackend:
Expand Down
10 changes: 8 additions & 2 deletions examples/CloudIt.sh
Expand Up @@ -25,6 +25,8 @@ app="$1";
appsrc="$app.pas";
qemufile="qemu.args";
compileropt="$2";
fpcrtlsource="/root/fpc-3.2.0/rtl/";

export KERNEL_HEAD=$(git rev-parse HEAD|cut -c1-7)

# check parameters
Expand All @@ -39,7 +41,7 @@ if [ -f $qemufile ]; then
qemuparams=`cat $qemufile`
else
# parameters by default
qemuparams="-enable-kvm -M microvm,pic=off,pit=off,rtc=off -cpu host -m 16 -smp 1 -nographic -D qemu.log -d guest_errors -no-reboot"
qemuparams="-enable-kvm -M microvm,pic=off,pit=off,rtc=off -cpu host -m 32 -smp 1 -nographic -D qemu.log -d guest_errors -no-reboot"
fi

# remove all compiled files
Expand All @@ -48,8 +50,12 @@ rm -f ../../rtl/*.o ../../rtl/*.ppu ../../rtl/drivers/*.o ../../rtl/drivers/*.pp
# remove the application
rm -f $app "$app.o"

# NOTE: It is very important to set up -Fu otherwise fpc will use the installed units
if [ -f $appsrc ]; then
fpc $compileropt -Xm -Si -TLinux -O2 $appsrc -o$app -Fu../../rtl/ -Fu../../rtl/drivers -MObjfpc
# The symbols defined/undefined are the same than in system.pp
fpc -dFPC_NO_DEFAULT_MEMORYMANAGER -dHAS_MEMORYMANAGER -uFPC_HAS_INDIRECT_ENTRY_INFORMATION -dx86_64 -I$fpcrtlsource/objpas/sysutils/ -I$fpcrtlsource/linux/x86_64/ -I$fpcrtlsource/x86_64/ -I$fpcrtlsource/linux/ -I$fpcrtlsource/inc/ -I$fpcrtlsource/unix/ -Fu$fpcrtlsource/unix/ -Fu$fpcrtlsource/linux/ -MObjfpc $fpcrtlsource/linux/si_prc.pp -Fu$fpcrtlsource/objpas -Fu$fpcrtlsource/inc
fpc -Us -dx86_64 -I$fpcrtlsource/objpas/sysutils/ -I$fpcrtlsource/linux/x86_64/ -I$fpcrtlsource/x86_64/ -I$fpcrtlsource/linux/ -I$fpcrtlsource/inc/ -I$fpcrtlsource/unix/ -Fu$fpcrtlsource/unix -Fu$fpcrtlsource/linux -Fu$fpcrtlsource/objpas -Fu$fpcrtlsource/inc $fpcrtlsource/linux/system.pp
fpc -TLinux -I$fpcrtlsource/objpas/sysutils/ -I$fpcrtlsource/linux/x86_64 -I$fpcrtlsource/x86_64/ -I$fpcrtlsource/linux/ -I$fpcrtlsource/inc/ -I$fpcrtlsource/unix/ $compileropt -Xm -Si -O2 $appsrc -o$app -Fu../../rtl -Fu../../rtl/drivers -Fu$fpcrtlsource/unix -Fu$fpcrtlsource/linux -Fu$fpcrtlsource/objpas -Fu$fpcrtlsource/inc -MObjfpc -kprt0.o
~/qemulast/build/x86_64-softmmu/qemu-system-x86_64 -kernel $app $qemuparams $3
else
echo "$appsrc does not exist, exiting"
Expand Down
131 changes: 131 additions & 0 deletions examples/ExceptionHandling/ExceptionHandling.pas
@@ -0,0 +1,131 @@
//
// Exception Handling
//
// Copyright (c) 2003-2020 Matias Vara <matiasevara@gmail.com>
// All Rights Reserved
//
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// This program is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
//
// You should have received a copy of the GNU General Public License
// along with this program. If not, see <http://www.gnu.org/licenses/>.
//

program ExceptionHandling;

{$IFDEF FPC}
{$mode delphi}
{$ENDIF}

{%RunCommand qemu-system-x86_64 -m 256 -smp 2 -drive format=raw,file=ExceptionHandling.img}
{%RunFlags BUILD-}

uses
SysUtils,
Kernel,
Process,
Memory,
Debug,
Arch,
Filesystem,
Console;

var
tmp: TThreadID = 0;

{$ASMMODE intel}

// Procedure that tests Division by zero exception handler
procedure DoDivZero;
var
Q, R: Longint;
begin
try
Q := 5;
R := 0;
R := Q div R;
except
on E: Exception do
begin
WriteConsoleF('Exception Message: %s\n',[PtrUInt(@E.Message)]);
end;
end;
end;

// Procedure that tests Page Fault exception handler
procedure DoPageFault;
var
p: ^longint;
begin
// this page is not present
try
p := pointer($ffffffffffffffff);
p^ := $1234;
except
on E: Exception do
begin
WriteConsoleF('Exception Message: %s\n',[PtrUInt(@E.Message)]);
end;
end;
end;

// Procedure that tests Protection Fault exception handler
procedure DoProtectionFault;
begin
try
asm
mov ax, $20
mov ds, ax
end;
except
on E: Exception do
begin
WriteConsoleF('Exception Message: %s\n',[PtrUInt(@E.Message)]);
end;
end;
end;

// Procedure that tests Illegal instruction exception handler
procedure DoIllegalInstruction;
begin
try
asm
db $ff, $ff
end;
except
on E: Exception do
begin
WriteConsoleF('Exception Message: %s\n',[PtrUInt(@E.Message)]);
end;
end;
end;

function Exception_Core2(Param: Pointer):PtrInt;
begin
//DoDivZero;
// no registra la interruption y el raise exception termina llamando a unhandle
DoPageFault;
//DoProtectionFault;
//DoIllegalInstruction;
Result := 0;
end;

begin
//tmp:= BeginThread(nil, 4096, Exception_Core2, nil, 1, tmp);
DoDivZero;
//try
// Raise EDivException.Create ('Division by Zero would occur');
//except
// WriteConsoleF('Exception!\n',[]);
//end;
//DoPageFault;
//DoProtectionFault;
//DoIllegalInstruction;
end.
25 changes: 25 additions & 0 deletions examples/ExceptionHandling/README.md
@@ -0,0 +1,25 @@
# Exception Handling Example

In this example, we show how the handling of exceptions. The mechanism is the same that used in freepascal or delphi. You need to just wrap up your statements with the **try..except** keywords. Toro handles exceptions in the same way other kernel does. It invokes the user's code if an exception happens. In Linux, to compile and exceute the application on QEMU, go to the **ExceptionHandling** directory and execute:

`../CloudIt.sh ExceptionHandling "" "-display gtk"`

If you want to test without graphical interface execute:

`../CloudIt.sh ExceptionHandling`

If you want to redirect the screen through VNC execute:

`../CloudIt.sh ExceptionHandling "" "-vnc :0"`

You can connect the vnc client to **localhost:5900**

If you want to enable some debug symbols in the unit **Process** and check what Toro is doing execute:

`../CloudIt.sh ExceptionHandling "-dEnableDebug -dDebugProcess"`

You will see how toro initializes the unit by calling the scheduler.

## Windows Users

Windows' users should open **ExceptionHandling.lpi** and launch the compilation and execution of the application directly from the IDE by doing first **Compile** and then **Run**.
Binary file added examples/ExceptionHandling/prt0.o
Binary file not shown.
15 changes: 7 additions & 8 deletions examples/HelloWorld/HelloWorld.pas
Expand Up @@ -31,14 +31,13 @@

// only include the needed units
uses
SysUtils,
Kernel in '..\..\rtl\Kernel.pas',
Process in '..\..\rtl\Process.pas',
Memory in '..\..\rtl\Memory.pas',
Debug in '..\..\rtl\Debug.pas',
Arch in '..\..\rtl\Arch.pas',
Filesystem in '..\..\rtl\Filesystem.pas',
Console in '..\..\rtl\drivers\Console.pas';
Kernel,
Process,
Memory,
Debug,
Arch,
Filesystem,
Console;

procedure ShutdownHelloWorld;
begin
Expand Down
Binary file modified examples/HelloWorld/prt0.o
Binary file not shown.
22 changes: 11 additions & 11 deletions examples/StaticWebServer/StaticWebServer.pas
Expand Up @@ -28,17 +28,17 @@
{%RunFlags BUILD-}

uses
Kernel in '..\..\rtl\Kernel.pas',
Process in '..\..\rtl\Process.pas',
Memory in '..\..\rtl\Memory.pas',
Debug in '..\..\rtl\Debug.pas',
Arch in '..\..\rtl\Arch.pas',
Filesystem in '..\..\rtl\Filesystem.pas',
VirtIO in '..\..\rtl\drivers\VirtIO.pas',
VirtIOFS in '..\..\rtl\drivers\VirtIOFS.pas',
VirtIOVSocket in '..\..\rtl\drivers\VirtIOVSocket.pas',
Console in '..\..\rtl\drivers\Console.pas',
Network in '..\..\rtl\Network.pas';
Kernel,
Process,
Memory,
Debug,
Arch,
Filesystem,
VirtIO,
VirtIOFS,
VirtIOVSocket,
Console,
Network;

const
HeaderOK = 'HTTP/1.0 200'#13#10'Content-type: ';
Expand Down
Binary file modified examples/StaticWebServer/prt0.o
Binary file not shown.

0 comments on commit 5718bf7

Please sign in to comment.