Repo: docker/VM setup for runtime engine testing

[camerondurham]

Use Sylvia's branch, should be able to use syscalls:

• chroot
• sethostname
• setrlimit

EDIT:

Unable to use syscalls requested with Docker settings. Recommend using VM, WSL, or native Linux install to run.

Have prepared an Ubuntu Server VM .ova file to import into VirtualBox:

• Setup guide Google Doc Link
• Virtual Machine Image runner-ubuntu-server.ova

[camerondurham]

Instructions to setup Linux VM with Go 1.17.7 installed for engine/runtime development:

Importing VM Appliance to VirtualBox

1. Download the Ubuntu Server OVA from drive runner-ubuntu-server.ova
2. Install and/or Open Virtual Box
3. Open File > Import Appliance
4. Select the runner-ubuntu-server.ova downloaded in step 1
5. Select machine settings
   1. Setup memory (2GB recommended)
   2. Setup hard disk size (20-30GB recommended)
6. Start VM
   1. User: runner
   2. Password: runner
7. Open Settings > Network > Adapter 1
   1. Select Bridged Adapter to assign the machine a local IP address
   2. Run ip a in terminal and note the 192.168.0.X address
   3. SSH or use Remote Development extension with VSCode for editing
      1. ssh runner@192.168.0.X
      2. Enter password and log in!

---

Q: Why do you need to create a VM for runtime development?
A: From testing, the setrlimit, prlimit syscalls did not allow limiting of processes within the container. Docker and other container runtimes have their own way of limiting resources for containers and this model makes it difficult to restrict processes from within the container. There may be a way to do this but I was unable to find it! Setting up a VM is faster if slightly less easy to use than Docker.

[camerondurham]

Update: Previous testing with prlimit and setrlimit was done incorrectly. TLDR is that need to create a non-root user (uid != 0) for these resource limits to actually matter then these calls will work in Docker (citation needed).

From testing with a minimal Dockerfile, this should be possible to use the runner engine within Docker with a Dockerfile like so:

FROM golang:1.17.7-buster as builder
WORKDIR /runner
COPY . .
RUN go build -v -o /runner/process /runner/engine/process/

# TODO: use slimmer docker image like ubuntu:20.04
FROM golang:1.17.7-buster

# create non-root user, assign UID != 0
RUN useradd -u 1111 -ms /bin/bash runner

# define user to start up as
USER runner

WORKDIR /home/runner
COPY --from=builder /runner/process ./
ENTRYPOINT ["/bin/bash"]

One note of caution is that the limits are per-user. I think the best solution to this would be to use cgroups instead since this offers a more granular process-by-process control. However, we can still accomplish process limiting by creating new users for each runner request (and re-use users as needed).

This post made me realize I was testing wrong before: https://unix.stackexchange.com/questions/303190/prlimit-fails-are-20-processes-not-enough-for-bash

Other helpful links:

• https://linux.die.net/man/2/prlimit
• https://www.gnu.org/software/libc/manual/html_node/Limits-on-Resources.html

[siwei-li]

Thank you for posting! Now I know how to use setrlimit properly...

Update: Previous testing with prlimit and setrlimit was done incorrectly. TLDR is that need to create a non-root user (uid != 0) for these resource limits to actually matter then these calls will work in Docker (citation needed).

From testing with a minimal Dockerfile, this should be possible to use the runner engine within Docker with a Dockerfile like so:

FROM golang:1.17.7-buster as builder
WORKDIR /runner
COPY . .
RUN go build -v -o /runner/process /runner/engine/process/

# TODO: use slimmer docker image like ubuntu:20.04
FROM golang:1.17.7-buster

# create non-root user, assign UID != 0
RUN useradd -u 1111 -ms /bin/bash runner

# define user to start up as
USER runner

WORKDIR /home/runner
COPY --from=builder /runner/process ./
ENTRYPOINT ["/bin/bash"]

One note of caution is that the limits are per-user. I think the best solution to this would be to use cgroups instead since this offers a more granular process-by-process control. However, we can still accomplish process limiting by creating new users for each runner request (and re-use users as needed).

This post made me realize I was testing wrong before: https://unix.stackexchange.com/questions/303190/prlimit-fails-are-20-processes-not-enough-for-bash

Other helpful links:

• https://linux.die.net/man/2/prlimit
• https://www.gnu.org/software/libc/manual/html_node/Limits-on-Resources.html

[camerondurham]

This comment reminded me I did not update this issue with the latest way to setup the dev environment.

The instructions I posted in this issue about setting up a VM are not necessary. Fortunately, you do not need a VM and there are updated instructions in README for how to setup Docker and VSCode for the project: https://github.com/camerondurham/runner#using-dev-containers-with-vscode-recommended