Skip to content

Commit

Permalink
Merge pull request #8759 from fadecoder/update_docs_for_stratoVirt_VMM
Browse files Browse the repository at this point in the history
docs: Update docs for new StratoVirt VMM introduction
  • Loading branch information
GabyCT committed Jan 4, 2024
2 parents 4f9ee7b + 44b5b88 commit f056ffe
Show file tree
Hide file tree
Showing 6 changed files with 53 additions and 14 deletions.
2 changes: 1 addition & 1 deletion docs/Developer-Guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -437,7 +437,7 @@ You can build and install the guest kernel image as shown [here](../tools/packag
# Install a hypervisor

When setting up Kata using a [packaged installation method](install/README.md#installing-on-a-linux-system), the
`QEMU` VMM is installed automatically. Cloud-Hypervisor and Firecracker VMMs are available from the [release tarballs](https://github.com/kata-containers/kata-containers/releases), as well as through [`kata-deploy`](../tools/packaging/kata-deploy/README.md).
`QEMU` VMM is installed automatically. Cloud-Hypervisor, Firecracker and StratoVirt VMMs are available from the [release tarballs](https://github.com/kata-containers/kata-containers/releases), as well as through [`kata-deploy`](../tools/packaging/kata-deploy/README.md).
You may choose to manually build your VMM/hypervisor.

## Build a custom QEMU
Expand Down
17 changes: 17 additions & 0 deletions docs/design/virtualization.md
Original file line number Diff line number Diff line change
Expand Up @@ -112,10 +112,27 @@ Devices and features used:
- seccomp filters
- [HTTP OpenAPI](https://github.com/cloud-hypervisor/cloud-hypervisor/blob/main/vmm/src/api/openapi/cloud-hypervisor.yaml)

### StratoVirt/KVM

[StratoVirt](https://gitee.com/openeuler/stratovirt) is an enterprise-level open source VMM oriented to cloud data centers, implements a unified architecture to support Standard-VMs, containers and serverless (Micro-VM). StratoVirt has some competitive advantages, such as lightweight and low resource overhead, fast boot, hardware acceleration, and language-level security with Rust.

Currently, StratoVirt in Kata supports Micro-VM machine type, mainly focus on FaaS cases, supporting device hotplug (virtio block), file-system sharing through virtio fs and so on. Kata Containers with StratoVirt now use virtio-mmio bus as driver, and doesn't support CPU/memory resize nor VFIO, thus doesn't support updating container resources after booted.

Devices and features used currently:
- Micro-VM machine type for FaaS(mmio, no ACPI)
- Virtual Socket(vhost VSOCK、virtio console)
- Virtual Storage(virtio block, mmio)
- Virtual Networking(virtio net, mmio)
- Shared Filesystem(virtio fs)
- Device Hotplugging(virtio block hotplug)
- Entropy Source(virtio RNG)
- QMP API

### Summary

| Solution | release introduced | brief summary |
|-|-|-|
| Cloud Hypervisor | 1.10 | upstream Cloud Hypervisor with rich feature support, e.g. hotplug, VFIO and FS sharing|
| Firecracker | 1.5 | upstream Firecracker, rust-VMM based, no VFIO, no FS sharing, no memory/CPU hotplug |
| QEMU | 1.0 | upstream QEMU, with support for hotplug and filesystem sharing |
| StratoVirt | 3.3 | upstream StratoVirt with FS sharing and virtio block hotplug, no VFIO, no CPU/memory resize |
23 changes: 13 additions & 10 deletions docs/hypervisors.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,11 +29,12 @@ are available, their default values and how each setting can be used.
| Hypervisor | Written in | Architectures | Type | Configuration file |
|-|-|-|-|-|
[ACRN] | C | `x86_64` | Type 1 (bare metal) | `configuration-acrn.toml` |
[Cloud Hypervisor] | rust | `aarch64`, `x86_64` | Type 2 ([KVM]) | `configuration-clh.toml` |
[Firecracker] | rust | `aarch64`, `x86_64` | Type 2 ([KVM]) | `configuration-fc.toml` |
[QEMU] | C | all | Type 2 ([KVM]) | `configuration-qemu.toml` |
[`Dragonball`] | rust | `aarch64`, `x86_64` | Type 2 ([KVM]) | `configuration-dragonball.toml` |
|[ACRN] | C | `x86_64` | Type 1 (bare metal) | `configuration-acrn.toml` |
|[Cloud Hypervisor] | rust | `aarch64`, `x86_64` | Type 2 ([KVM]) | `configuration-clh.toml` |
|[Firecracker] | rust | `aarch64`, `x86_64` | Type 2 ([KVM]) | `configuration-fc.toml` |
|[QEMU] | C | all | Type 2 ([KVM]) | `configuration-qemu.toml` |
|[`Dragonball`] | rust | `aarch64`, `x86_64` | Type 2 ([KVM]) | `configuration-dragonball.toml` |
|[StratoVirt] | rust | `aarch64`, `x86_64` | Type 2 ([KVM]) | `configuration-stratovirt.toml` |

## Determine currently configured hypervisor

Expand All @@ -49,11 +50,12 @@ the hypervisors:

| Hypervisor | Summary | Features | Limitations | Container Creation speed | Memory density | Use cases | Comment |
|-|-|-|-|-|-|-|-|
[ACRN] | Safety critical and real-time workloads | | | excellent | excellent | Embedded and IOT systems | For advanced users |
[Cloud Hypervisor] | Low latency, small memory footprint, small attack surface | Minimal | | excellent | excellent | High performance modern cloud workloads | |
[Firecracker] | Very slimline | Extremely minimal | Doesn't support all device types | excellent | excellent | Serverless / FaaS | |
[QEMU] | Lots of features | Lots | | good | good | Good option for most users | | All users |
[`Dragonball`] | Built-in VMM, low CPU and memory overhead| Minimal | | excellent | excellent | Optimized for most container workloads | `out-of-the-box` Kata Containers experience |
|[ACRN] | Safety critical and real-time workloads | | | excellent | excellent | Embedded and IOT systems | For advanced users |
|[Cloud Hypervisor] | Low latency, small memory footprint, small attack surface | Minimal | | excellent | excellent | High performance modern cloud workloads | |
|[Firecracker] | Very slimline | Extremely minimal | Doesn't support all device types | excellent | excellent | Serverless / FaaS | |
|[QEMU] | Lots of features | Lots | | good | good | Good option for most users | |
|[`Dragonball`] | Built-in VMM, low CPU and memory overhead| Minimal | | excellent | excellent | Optimized for most container workloads | `out-of-the-box` Kata Containers experience |
|[StratoVirt] | Unified architecture supporting three scenarios: VM, container, and serverless | Extremely minimal(`MicroVM`) to Lots(`StandardVM`) | | excellent | excellent | Common container workloads | `StandardVM` type of StratoVirt for Kata is under development |

For further details, see the [Virtualization in Kata Containers](design/virtualization.md) document and the official documentation for each hypervisor.

Expand All @@ -63,3 +65,4 @@ For further details, see the [Virtualization in Kata Containers](design/virtuali
[KVM]: https://en.wikipedia.org/wiki/Kernel-based_Virtual_Machine
[QEMU]: http://www.qemu-project.org
[`Dragonball`]: https://github.com/kata-containers/kata-containers/blob/main/src/dragonball
[StratoVirt]: https://gitee.com/openeuler/stratovirt
1 change: 1 addition & 0 deletions tests/cmd/check-spelling/data/projects.txt
Original file line number Diff line number Diff line change
Expand Up @@ -86,6 +86,7 @@ SemaphoreCI/B
snapcraft/B
snapd/B
SQLite/B
StratoVirt/B
SUSE/B
Sysbench/B
systemd/B
Expand Down
3 changes: 2 additions & 1 deletion tests/cmd/check-spelling/kata-dictionary.dic
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
383
384
ACPI/AB
ACS/AB
API/AB
Expand Down Expand Up @@ -145,6 +145,7 @@ SUSE/B
SVG/AB
SaaS/B
SemaphoreCI/B
StratoVirt/B
Struct/A#
Sysbench/B
TBD/AB
Expand Down
21 changes: 19 additions & 2 deletions tools/packaging/kata-deploy/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -138,6 +138,15 @@ spec:
runtimeClassName: kata-fc
```

The following YAML snippet shows how to specify a workload should use Kata with StratoVirt:

```yaml
spec:
template:
spec:
runtimeClassName: kata-stratovirt
```

The following YAML snippet shows how to specify a workload should use Kata with QEMU:

```yaml
Expand All @@ -164,6 +173,12 @@ To run an example with `kata-fc`:
$ kubectl apply -f https://raw.githubusercontent.com/kata-containers/kata-containers/main/tools/packaging/kata-deploy/examples/test-deploy-kata-fc.yaml
```

To run an example with `kata-stratovirt`:

```bash
$ kubectl apply -f https://raw.githubusercontent.com/kata-containers/kata-containers/main/tools/packaging/kata-deploy/examples/test-deploy-kata-stratovirt.yaml
```

To run an example with `kata-qemu`:

```bash
Expand All @@ -176,6 +191,7 @@ The following removes the test pods:
$ kubectl delete -f https://raw.githubusercontent.com/kata-containers/kata-containers/main/tools/packaging/kata-deploy/examples/test-deploy-kata-dragonball.yaml
$ kubectl delete -f https://raw.githubusercontent.com/kata-containers/kata-containers/main/tools/packaging/kata-deploy/examples/test-deploy-kata-clh.yaml
$ kubectl delete -f https://raw.githubusercontent.com/kata-containers/kata-containers/main/tools/packaging/kata-deploy/examples/test-deploy-kata-fc.yaml
$ kubectl delete -f https://raw.githubusercontent.com/kata-containers/kata-containers/main/tools/packaging/kata-deploy/examples/test-deploy-kata-stratovirt.yaml
$ kubectl delete -f https://raw.githubusercontent.com/kata-containers/kata-containers/main/tools/packaging/kata-deploy/examples/test-deploy-kata-qemu.yaml
```

Expand Down Expand Up @@ -235,7 +251,7 @@ This image contains all the necessary artifacts for running Kata Containers, all
from the [Kata Containers release page](https://github.com/kata-containers/kata-containers/releases).

Host artifacts:
* `cloud-hypervisor`, `firecracker`, `qemu`, and supporting binaries
* `cloud-hypervisor`, `firecracker`, `qemu`, `stratovirt` and supporting binaries
* `containerd-shim-kata-v2` (go runtime and rust runtime)
* `kata-collect-data.sh`
* `kata-runtime`
Expand All @@ -254,7 +270,8 @@ applying labels to the nodes.
This DaemonSet installs the necessary Kata binaries, configuration files, and virtual machine artifacts on
the node. Once installed, the DaemonSet adds a node label `katacontainers.io/kata-runtime=true` and reconfigures
either CRI-O or containerd to register three `runtimeClasses`: `kata-clh` (for Cloud Hypervisor isolation), `kata-qemu` (for QEMU isolation),
and `kata-fc` (for Firecracker isolation). As a final step the DaemonSet restarts either CRI-O or containerd. Upon deletion,
`kata-fc` (for Firecracker isolation) and `kata-stratovirt` (for StratoVirt isolation).
As a final step the DaemonSet restarts either CRI-O or containerd. Upon deletion,
the DaemonSet removes the Kata binaries and VM artifacts and updates the node label to `katacontainers.io/kata-runtime=cleanup`.

#### Kata cleanup
Expand Down

0 comments on commit f056ffe

Please sign in to comment.