This is an example of a container shim that exposes an HTTP API1 to control the life cycle of a container process. Theoretically, shims should be a small as possible because container managers use a shim per container process. This isn't the case of this shim, though, but also no one should be using it except for learning purposes.
👋 Make sure to follow these instructions first.
First, we need a new bundle:
$ make alpine_bundle
Let's edit the config.json
file generated in /tmp/alpine-bundle
as follows:
--- a/config.json
+++ b/config.json
@@ -6,8 +6,7 @@
"gid": 0
},
"args": [
- "sleep",
- "100"
+ "sh", "/hello-loop.sh"
],
"env": [
"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
We should also create this new file named hello-loop.sh
:
$ cat <<'EOF' > /tmp/alpine-bundle/rootfs/hello-loop.sh
#!/bin/sh
signal_handler() {
>&2 echo "bye, bye"
exit 123
}
trap 'signal_handler' TERM
while [ true ]; do
echo "[$(date)] Hello!"
sleep 1
done
EOF
This script loops forever and prints a message every second, unless it receives a SIGTERM
signal, in which case it exits with a status equal to 123
.
We are ready to manually execute yacs
(the shim) to create a new container with the yacr
runtime (the default).
$ yacs --bundle=/tmp/alpine-bundle --container-id=alpine-1
/home/gitpod/.run/yacs/alpine-1/shim.sock
Note: This step is usually automated by a container manager such as Yaman.
This should start a new shim that will automatically create a container process. We can check with yacr list
and ps
:
$ yacr list
ID STATUS CREATED PID BUNDLE
alpine-1 created 2022-06-03T22:00:00Z 44488 /tmp/alpine-bundle
$ ps auxf
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
[...]
gitpod 44458 0.0 0.0 1079856 7260 ? Ssl 22:01 0:00 yacs --bundle=/tmp/alpine-bundle --container-id=alpine-1
gitpod 44488 0.0 0.0 1076520 5780 ? Sl 22:01 0:00 \_ yacr --log-format json --log /home/gitpod/.run/yacs/alpine-1/yacr.log create container alpine-1 --root /home/gitpod/.run/yacr --bundle /tmp/alpine-bundle
When the command returns, it prints a unix socket address that can be used to query the shim using... HTTP. This isn't great but it is enough to demonstrate how a shim works.
We can use curl
to interact with the shim:
$ curl --unix-socket /home/gitpod/.run/yacs/alpine-1/shim.sock http://shim/
{
"ID": "alpine-1",
"Runtime": "yacr",
"State": {
"ociVersion": "1.0.2",
"id": "alpine-1",
"status": "created",
"pid": 44488,
"bundle": "/tmp/alpine-bundle"
},
"Status": {}
}
We can now start the container by sending the start
command (cmd
) in a POST
HTTP request:
$ curl -X POST -d 'cmd=start' --unix-socket /home/gitpod/.run/yacs/alpine-1/shim.sock http://shim/
{
"ID": "alpine-1",
"Runtime": "yacr",
"State": {
"ociVersion": "1.0.2",
"id": "alpine-1",
"status": "running",
"pid": 44488,
"bundle": "/tmp/alpine-bundle"
},
"Status": {}
}
Note: jq
was used to pretty-print the JSON responses in the different examples.
The container is now running, which we can confirm with yacr list
and ps
:
$ yacr list
ID STATUS CREATED PID BUNDLE
alpine-1 running 2022-06-03T22:00:00Z 44488 /tmp/alpine-bundle
$ ps auxf
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
[...]
gitpod 44458 0.0 0.0 1079856 7260 ? Ssl 22:01 0:00 yacs --bundle=/tmp/alpine-bundle --container-id=alpine-1
gitpod 44488 0.0 0.0 1596 4 ? S 22:01 0:00 \_ sh /hello-loop.sh
gitpod 55758 0.0 0.0 1596 4 ? S 22:02 0:00 \_ sleep 1
We can query the shim to get the container's logs:
$ curl --unix-socket /home/gitpod/.run/yacs/alpine-1/shim.sock http://shim/logs
{"m":"[Sun Jun 12 11:51:44 UTC 2022] Hello!","s":"stdout","t":"2022-06-12T11:51:44.947554491Z"}
{"m":"[Sun Jun 12 11:51:45 UTC 2022] Hello!","s":"stdout","t":"2022-06-12T11:51:45.948493454Z"}
{"m":"[Sun Jun 12 11:51:46 UTC 2022] Hello!","s":"stdout","t":"2022-06-12T11:51:46.949371235Z"}
{"m":"[Sun Jun 12 11:51:47 UTC 2022] Hello!","s":"stdout","t":"2022-06-12T11:51:47.950339068Z"}
Each entry is a JSON object with the following properties:
m
: the messages
: the stream (eitherstdout
orstderr
)t
: the timestamp
We can also use the shim HTTP API to send a signal to the container:
$ curl -X POST -d 'cmd=kill' --unix-socket /home/gitpod/.run/yacs/alpine-1/shim.sock http://shim/
{
"ID": "alpine-1",
"Runtime": "yacr",
"State": {
"ociVersion": "1.0.2",
"id": "alpine-1",
"status": "running",
"pid": 44488,
"bundle": "/tmp/alpine-bundle"
},
"Status": {}
}
In the output above, we see the status
set to running
despite our attempt to "kill" the container. This happens because of timing, the container probably didn't have time to change its state.
If we query the logs again, we can see that the container actually received the signal (SIGTERM
by default):
$ curl --unix-socket /home/gitpod/.run/yacs/alpine-1/shim.sock http://shim/logs
[...]
{"m":"[Sun Jun 12 11:52:36 UTC 2022] Hello!","s":"stdout","t":"2022-06-12T11:52:36.001548972Z"}
{"m":"[Sun Jun 12 11:52:37 UTC 2022] Hello!","s":"stdout","t":"2022-06-12T11:52:37.002608715Z"}
{"m":"[Sun Jun 12 11:52:38 UTC 2022] Hello!","s":"stdout","t":"2022-06-12T11:52:38.003658337Z"}
{"m":"bye, bye","s":"stderr","t":"2022-06-12T11:52:40.005199923Z"}
The container printed the message of the signal_handler
defined in the hello-loop.sh
script so the kill
command worked as intended.
Let's query the state of the container again:
$ curl --unix-socket /home/gitpod/.run/yacs/alpine-1/shim.sock http://shim/
{
"ID": "alpine-1",
"Runtime": "yacr",
"State": {
"ociVersion": "1.0.2",
"id": "alpine-1",
"status": "stopped",
"pid": 44488,
"bundle": "/tmp/alpine-bundle"
},
"Status": {
"exitStatus": 123,
"exited": true,
"pid": 44488,
"waitStatus": 31488
}
}
The container is now "stopped". The exitStatus
is 123
and matches what we defined in the hello-loop.sh
file created previously. Note also that the shim is still alive and we still have access to the container's full state and stdout/stderr logs. This is one of the reasons why shims are used.
yacr list
ID STATUS CREATED PID BUNDLE
alpine-1 stopped 2022-06-03T22:00:00Z 0 /tmp/alpine-bundle
We can now delete the container. This API request should not return anything (HTTP 204):
$ curl -X POST -d 'cmd=delete' --unix-socket /home/gitpod/.run/yacs/alpine-1/shim.sock http://shim/
If we query the state of the shim again, it should indicate that the container does not exist anymore:
$ curl --unix-socket /home/gitpod/.run/yacs/alpine-1/shim.sock http://shim/
container does not exist
Finally, we can terminate the shim with a DELETE
HTTP request:
$ curl -X DELETE --unix-socket /home/gitpod/.run/yacs/alpine-1/shim.sock http://shim/
BYE
👋 Make sure to follow these instructions first.
This shim should be able to use any OCI-compliant runtime like runc
(the reference implementation). Let's reproduce what was done in the previous section but with runc
.
$ yacs --bundle /tmp/alpine-bundle/ --container-id alpine-runc --runtime runc
/home/gitpod/.run/yacs/alpine-runc/shim.sock
The ps
output below shows that runc
has been invoked:
$ ps auxf
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
[...]
gitpod 4321 0.0 0.0 1079876 6776 ? Ssl 10:57 0:00 yacs --bundle /tmp/alpine-bundle/ --container-id alpine-runc --runtime runc
gitpod 4363 0.0 0.0 1083784 9980 ? Ssl 10:57 0:00 \_ runc init
$ curl -X POST -d 'cmd=start' --unix-socket /home/gitpod/.run/yacs/alpine-runc/shim.sock http://shim/
{
"id": "alpine-runc",
"runtime": "runc",
"state": {
"ociVersion": "1.0.2-dev",
"id": "alpine-runc",
"status": "running",
"pid": 4363,
"bundle": "/tmp/alpine-bundle"
},
"status": {}
}
We could use runc list
to list the containers created by runc
:
$ runc list
ID PID STATUS BUNDLE CREATED OWNER
alpine-runc 4363 running /tmp/alpine-bundle 2022-06-05T10:57:18.334165274Z gitpod
Since runc
is the reference implementation and a production-ready runtime, it has a LOT more features than yacr
. For instance, we can use runc exec
to execute a new process in the container, like spawning a shell:
$ runc exec -t alpine-runc /bin/sh
/ # ps
PID USER TIME COMMAND
1 root 0:00 sh /hello-loop.sh
229 root 0:00 /bin/sh
236 root 0:00 sleep 1
237 root 0:00 ps
/ #
Let's kill the container now:
$ curl -X POST -d 'cmd=kill' --unix-socket /home/gitpod/.run/yacs/alpine-runc/shim.sock http://shim/
{
"id": "alpine-runc",
"runtime": "runc",
"state": {
"ociVersion": "1.0.2-dev",
"id": "alpine-runc",
"status": "running",
"pid": 4363,
"bundle": "/tmp/alpine-bundle"
},
"status": {}
}
The state should be updated after the container process has exited:
$ curl --unix-socket /home/gitpod/.run/yacs/alpine-runc/shim.sock http://shim/
{
"id": "alpine-runc",
"runtime": "runc",
"state": {
"ociVersion": "1.0.2-dev",
"id": "alpine-runc",
"status": "stopped",
"bundle": "/tmp/alpine-bundle"
},
"status": {
"exitStatus": 123,
"exited": true
}
}
We can now delete the container and terminate the shim:
$ curl -X POST -d 'cmd=delete' --unix-socket /home/gitpod/.run/yacs/alpine-runc/shim.sock http://shim/
$ curl -X DELETE --unix-socket /home/gitpod/.run/yacs/alpine-runc/shim.sock http://shim/
BYE
Yacs has many configuration flags (options). This section describes some of them.
Yacs uses this log file to write the standard output (and error) of the container process. Each line is appended to the log file as a JSON object (also described in a previous section above):
{"m":"[Sun Jun 12 11:51:44 UTC 2022] Hello!","s":"stdout","t":"2022-06-12T11:51:44.947554491Z"}
When the container process exits, Yacs will call an "exit command" when --exit-command
is specified. It is also possible to specify command arguments with --exit-command-arg
.
This can be useful for daemon-less container managers (e.g., Yaman configures Yacs to call yaman container cleanup
when a container process exits so that (1) Yaman is notified of this event and (2) it can perform some clean-up tasks).
The OCI runtime to use. By default, Yacr will be used.
This is the directory where Yacs will create the FIFOs (stdio named pipes).