-
Notifications
You must be signed in to change notification settings - Fork 87
/
README.md
212 lines (151 loc) · 7.96 KB
/
README.md
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
![runwasi logo light mode](./art/logo/runwasi_icon1.svg#gh-light-mode-only)
![runwasi logo dark mode](./art/logo/runwasi_icon3.svg#gh-dark-mode-only)
## runwasi
> Warning: Alpha quality software, do not use in production.
This is a project to facilitate running wasm workloads managed by containerd either directly (ie. through ctr) or as directed by Kubelet via the CRI plugin.
It is intended to be a (rust) library that you can take and integrate with your wasm host.
Included in the repository is a PoC for running a plain wasi host (ie. no extra host functions except to support wasi system calls).
### Community
- If you haven't joined the CNCF slack yet, you can do so [here](https://slack.cncf.io/).
- Come join us on our [slack channel #runwasi](https://cloud-native.slack.com/archives/C04LTPB6Z0V)
on the CNCF slack.
- Public Community Call on Tuesdays at 9:00 AM PT: [Zoom](https://zoom.us/my/containerd?pwd=bENmREpnSGRNRXdBZWV5UG8wbU1oUT09), [Meeting Notes](https://docs.google.com/document/d/1aOJ-O7fgMyRowHD0kOoA2Z_4d19NyAvvdqOkZO3Su_M/edit?usp=sharing)
### Usage
runwasi is intended to be consumed as a library to be linked to from your own wasm host implementation.
There are two modes of operation supported:
1. "Normal" mode where there is 1 shim process per container or k8s pod.
2. "Shared" mode where there is a single manager service running all shims in process.
In either case you need to implement the `Instance` trait:
```rust
pub trait Instance {
// Create a new instance
fn new(id: String, cfg: Option<&InstanceConfig<Self::E>>) -> Self;
// Start the instance and return the pid
fn start(&self) -> Result<u32, Error>;
// Send the specified signal to the instance
fn kill(&self, signal: u32) -> Result<(), Error>;
// Delete the instance
fn delete(&self) -> Result<(), Error>;
// wait for the instance to exit and send the exit code and exit timestamp to the provided sender.
fn wait(&self, send: Sender<(u32, DateTime<Utc>)>) -> Result<(), Error>;
}
```
To use your implementation in "normal" mode, you'll need to create a binary which has a main that looks something like this:
```rust
use containerd_shim as shim;
use containerd_shim_wasm::sandbox::{ShimCli, Instance}
struct MyInstance {
// ...
}
impl Instance for MyInstance {
// ...
}
fn main() {
shim::run::<ShimCli<MyInstance>>("io.containerd.myshim.v1", opts);
}
```
Note you can implement your own ShimCli if you like and customize your wasm engine and other things.
I encourage you to checkout how that is implemented.
The shim binary just needs to be installed into `$PATH` (as seen by the containerd process) with a binary name like `containerd-shim-myshim-v1`.
For the shared mode:
```rust
use containerd_shim_wasm::sandbox::{Local, ManagerService, Instance};
use containerd_shim_wasm::services::sandbox_ttrpc::{create_manager, Manager};
use std::sync::Arc;
use ttrpc::{self, Server};
/// ...
struct MyInstance {
/// ...
}
impl Instance for MyInstance {
// ...
}
fn main() {
let s: ManagerService<Local<MyInstance>> =
ManagerService::new(Engine::new(Config::new().interruptable(true)).unwrap());
let s = Arc::new(Box::new(s) as Box<dyn Manager + Send + Sync>);
let service = create_manager(s);
let mut server = Server::new()
.bind("unix:///run/io.containerd.myshim.v1/manager.sock")
.unwrap()
.register_service(service);
server.start().unwrap();
let (_tx, rx) = std::sync::mpsc::channel::<()>();
rx.recv().unwrap();
}
```
This will be the host daemon that you startup and manage on your own.
You can use the provided `containerd-shim-myshim-v1` binary as the shim to specify in containerd.
Shared mode requires precise control over real threads and as such should not be used with an async runtime.
### Examples
#### Components
- **containerd-shim-[ wasmedge | wasmtime ]-v1**
This is a containerd shim which runs wasm workloads in [WasmEdge](https://github.com/WasmEdge/WasmEdge) or [Wasmtime](https://github.com/bytecodealliance/wasmtime).
You can use it with containerd's `ctr` by specifying `--runtime=io.containerd.[ wasmedge | wasmtime ].v1` when creating the container.
And make sure the shim binary must be in $PATH (that is the $PATH that containerd sees). Usually you just run `make install` after `make build`.
> build shim with wasmedge we need install library first
This shim runs one per pod.
- **containerd-shim-[ wasmedge | wasmtime ]d-v1**
A cli used to connect containerd to the `containerd-[ wasmedge | wasmtime ]d` sandbox daemon.
When containerd requests for a container to be created, it fires up this shim binary which will connect to the `containerd-[ wasmedge | wasmtime ]d` service running on the host.
The service will return a path to a unix socket which this shim binary will write back to containerd which containerd will use to connect to for shim requests.
This binary does not serve requests, it is only responsible for sending requests to the `contianerd-[ wasmedge | wasmtime ]d` daemon to create or destroy sandboxes.
- **containerd-[ wasmedge | wasmtime ]d**
This is a sandbox manager that enables running 1 wasm host for the entire node instead of one per pod (or container).
When a container is created, a request is sent to this service to create a sandbox.
The "sandbox" is a containerd task service that runs in a new thread on its own unix socket, which we return back to containerd to connect to.
The Wasmedge / Wasmtime engine is shared between all sandboxes in the service.
To use this shim, specify `io.containerd.[ wasmedge | wasmtime ]d.v1` as the runtime to use.
You will need to make sure the `containerd-[ wasmedge | wasmtime ]d` daemon has already been started.
#### Test and demo with containerd
**Attention**
Instead of enabling docker-desktop official released feature `use containerd for pulling and storing images`, you can build a local image and interact with the container locally.
- **Install WasmEdge first (If you choose Wasmedge as your wasm runtime)**
- Install WasmEdge
- Make sure the library is in the search path.
```terminal
$ curl -sSf https://raw.githubusercontent.com/WasmEdge/WasmEdge/master/utils/install.sh | bash
$ sudo -E sh -c 'echo "$HOME/.wasmedge/lib" > /etc/ld.so.conf.d/libwasmedge.conf'
$ sudo ldconfig
```
- **Run unit test**
```terminal
$ cargo test -- --nocapture
```
You should see some output like:
```terminal
running 3 tests
test instance::tests::test_maybe_open_stdio ... ok
test instance::wasitest::test_delete_after_create ... ok
test instance::wasitest::test_wasi ... ok
```
- **Build and install shim components**
```terminal
$ make build
$ sudo make install
```
- **Demo**
Now you can use the test image provided in this repo to have test with, use `make load` to load it into containerd.
- Case 1.
Run it with `sudo ctr run --rm --runtime=io.containerd.[ wasmedge | wasmtime ].v1 ghcr.io/containerd/runwasi/wasi-demo-app:latest testwasm /wasi-demo-app.wasm echo 'hello'`. You should see some output repeated like:
```terminal
$ sudo ctr run --rm --runtime=io.containerd.wasmedge.v1 ghcr.io/containerd/runwasi/wasi-demo-app:latest testwasm /wasi-demo-app.wasm echo 'hello'
hello
exiting
```
- Case 2.
Run it with `sudo ctr run --rm --runtime=io.containerd.[ wasmedge | wasmtime ].v1 docker.io/library/wasmtest:latest testwasm`.
You should see some output repeated like:
```terminal
$ sudo ctr run --rm --runtime=io.containerd.wasmedge.v1 docker.io/library/wasmtest:latest testwasm
This is a song that never ends.
Yes, it goes on and on my friends.
Some people started singing it not knowing what it was,
So they'll continue singing it forever just because...
This is a song that never ends.
Yes, it goes on and on my friends.
Some people started singing it not knowing what it was,
So they'll continue singing it forever just because...
(...)
```
To kill the process from the case 2. demo, you can run in other session: `sudo ctr task kill -s SIGKILL testwasm`. And the test binary supports full commands, check [test/image/src/main.rs](test/image/src/main.rs) to play around more.