From 95ce046d6c74ce7c96a8d3679d574c507ac316dc Mon Sep 17 00:00:00 2001 From: Jun Lin Chen Date: Sun, 4 Feb 2024 21:12:14 -0500 Subject: [PATCH] fix docs --- docs/2vm.md | 389 ++++++++++++++++++ docs/deprecated-v0.1/starlight-proxy.md | 208 ++++++++++ docs/deprecated-v0.1/starlight-snapshotter.md | 207 ++++++++++ docs/deprecated-v0.1/starlight-workflow.md | 19 + docs/terraform.md | 46 +++ 5 files changed, 869 insertions(+) create mode 100644 docs/2vm.md create mode 100644 docs/deprecated-v0.1/starlight-proxy.md create mode 100644 docs/deprecated-v0.1/starlight-snapshotter.md create mode 100644 docs/deprecated-v0.1/starlight-workflow.md create mode 100644 docs/terraform.md diff --git a/docs/2vm.md b/docs/2vm.md new file mode 100644 index 0000000..faa9727 --- /dev/null +++ b/docs/2vm.md @@ -0,0 +1,389 @@ +# TL;DR All-in-one Quick Start Guide + +To finish this guide, you will need TWO machines (or VMs) far away from each other. +One acts as the Cloud, and the other acts as the Edge. You will need to identify the IP address of the Cloud server. + +The following instructions have been tested using AWS EC2 t2.micro with Ubuntu 22.04 LTS and `starlight v0.3.2`. + +`git checkout v0.6.2` + +--- + +## The "Cloud" + +In this machine you will need to set up the Starlight Proxy and a standard container registry. +If you are using AWS EC2, please add the following ports to the Security Group whitelist when you create the VM: + - TCP 8090: Starlight Proxy (set the source to be your VPC) + - TCP 8080: Adminer - for Metadata database (set the source to be your own IP) + - TCP 5000: Container Registry (set the source to be your VPC) + - TCP 80: Container Registry (set the source to be your VPC) + + +1. Change the hostname of the server (Don't copy and paste, replace `` with your server's IP address) + + ```shell + echo "cloud" | sudo tee /etc/hostname > /dev/null + sudo hostname -F /etc/hostname + echo " cloud.cluster.local" | sudo tee -a /etc/hosts > /dev/null + ``` + +2. Install [Docker](https://docs.docker.com/engine/install/ubuntu/#install-using-the-repository) and [Docker Compose](https://docs.docker.com/compose/install/) + + If using Ubuntu 22.04 LTS, you could install Docker and Docker Compose using the following commands: + ```shell + sudo apt update && \ + sudo apt upgrade -y && \ + sudo apt install -y docker-compose && \ + sudo usermod -aG docker $USER + ``` + After adding the current user to the `docker` group, you (may) **need to log out and log in** to take effect. + To confirm that Docker is working with correct permission, `docker ps` should not print any errors. + ```shell + docker ps + # CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES + ``` + +3. Clone this project and launch the registry and proxy containers from `./demo/compose/registry+proxy` + + ```shell + git clone https://github.com/mc256/starlight.git && \ + cd starlight && \ + git checkout v0.6.2 && \ + cd demo/compose/ && \ + cp docker-compose-example.yaml docker-compose.yaml && \ + docker-compose up -d + # Creating network "registryproxy_default" with the default driver + # Creating registryproxy_db_1 ... done + # Creating registryproxy_registry_1 ... done + # Creating registryproxy_dbadmin_1 ... done + # Creating registryproxy_proxy_1 ... done + ``` + The Starlight proxy writes image metadata to the Postgres database, and + the container registry saves container images to `./data_registry`. + + +3. Verify the registry and proxy are running. + ```shell + # This checks the Starlight Proxy + curl http://cloud.cluster.local:8090/ + # {"status":"OK","code":200,"message":"Starlight Proxy"} + # This checks the container registry + curl http://cloud.cluster.local:5000/v2/ + # {} + ``` + + If it does not work, please restart the containers after the database has been created (missing a db health check). + We could put a Nginx reverse proxy to handle SSL certificates or load balancing. + But for simplicity, this part is ignored in this example. + + +3. Adjust the TCP window size (Optional). + If the edge node is far away, we will need to adjust the TCP window size so that the connection can speed up to the speed limit faster. (You could calculate the best TCP window size using https://www.speedguide.net/bdp.php later) + If you skip this step, the connection will be much slower (the impact on speed is worse for other methods!). + + ```shell + cat < /dev/null + net.core.wmem_max=125829120 + net.core.rmem_max=125829120 + net.ipv4.tcp_rmem= 10240 87380 125829120 + net.ipv4.tcp_wmem= 10240 87380 125829120 + net.ipv4.tcp_window_scaling = 1 + net.ipv4.tcp_timestamps = 1 + net.ipv4.tcp_sack = 1 + net.ipv4.tcp_no_metrics_save = 1 + net.core.netdev_max_backlog = 10000 + EOT + sudo sysctl -p + ``` + +🙌 That's it. Please obtain the IP address of this machine. + +--- + +## The "Edge" + +Please get another machine (or VM), you will need to set up a container worker with Starlight Snapshotter plugin. + +### 1. Install Dependencies + +The worker machine needs `build-essential` and `containerd`. +```shell +sudo apt update && sudo apt upgrade -y && \ +sudo apt install -y build-essential containerd +``` + +Enable `containerd` +```shell +sudo systemctl enable containerd && \ +sudo systemctl start containerd +``` + +Verify `containerd` is running +```shell +sudo systemctl status containerd +# Active: active +``` + +Install Go https://go.dev/doc/install ➡️ +```shell +wget https://go.dev/dl/go1.20.8.linux-amd64.tar.gz && \ +sudo rm -rf /usr/local/go && sudo tar -C /usr/local -xzf go1.20.8.linux-amd64.tar.gz +``` + +Add Go to the environment variable (You may want to change `.zshrc` or `.bashrc` file to permanently add this folder to the `PATH` environment variable) +```shell +export PATH=$PATH:/usr/local/go/bin +``` + +Verify Go is available +```shell +go version +# go version go1.18.3 linux/amd64 +``` + +### 2. (Optional) Tune the network + +Adjust the TCP window size. If the edge node is far away, we will need to adjust the TCP window size so that the connection can speed up to the speed limit faster. (You could calculate the best TCP window size using https://www.speedguide.net/bdp.php later) + +```shell +cat < /dev/null +net.core.wmem_max=125829120 +net.core.rmem_max=125829120 +net.ipv4.tcp_rmem= 10240 87380 125829120 +net.ipv4.tcp_wmem= 10240 87380 125829120 +net.ipv4.tcp_window_scaling = 1 +net.ipv4.tcp_timestamps = 1 +net.ipv4.tcp_sack = 1 +net.ipv4.tcp_no_metrics_save = 1 +net.core.netdev_max_backlog = 10000 +EOT +sudo sysctl -p +``` + +### 3. Clone and Build +Clone the Starlight repository +```shell +git clone https://github.com/mc256/starlight.git && \ +cd starlight && \ +git checkout v0.3.1 +``` + +Build the snapshotter plugin and CLI tool +```shell +make starlight-daemon ctr-starlight +``` + +### 4. Configure Starlight Snapshotter + +Find out the IP address / DNS of the Starlight Proxy server and set these two environment variables (Don't Copy-Paste!) +```shell +# DO NOT COPY! +# This is just an example !!! Get the real address of your server !!! +echo " cloud.cluster.local" | sudo tee -a /etc/hosts > /dev/null +```` + +Verify that the Starlight proxy is accessible from the worker. +```shell +curl http://cloud.cluster.local:8090/ +# {"status":"OK","code":200,"message":"Starlight Proxy"} +curl http://cloud.cluster.local:5000/v2/ +# {} +``` +If it does not work, please check the firewall configurations, +Please add port 8090 and 5000 to the firewall whitelist, the worker has to access these ports. + +Install Starlight Snapshotter `systemd` service and CLI tool. +```shell +sudo make install install-systemd-service +``` + +Enable Starlight snapshotter service +```shell +sudo systemctl enable starlight && \ +sudo systemctl start starlight +``` + +Verify Starlight is running +```shell +sudo systemctl status starlight +# it should be "active". +``` + +Add Starlight Proxy profile to the Snapshotter's configuration file +```shell +sudo ctr-starlight add myproxy http cloud.cluster.local:8090 +``` + +Confirm that the proxy has been added +```shell +sudo ctr-starlight ls +# [starlight-shared] https://starlight.yuri.moe +# [myproxy] http://cloud.cluster.local:8090 +``` + +Test the proxy is working +```shell +sudo ctr-starlight test myproxy +# ping test success: ok! - http://cloud.cluster.local:8090 +# latency: XX ms +``` + + +### 5. Configure `contaienrd` + +Add configuration to `/etc/containerd/config.toml`. +(If you have set other `proxy_plugins`, please manually edit the file) +```shell +sudo mkdir /etc/containerd/ && \ +cat < /dev/null + [proxy_plugins] + [proxy_plugins.starlight] + type = "snapshot" + address = "/run/starlight/starlight-snapshotter.sock" +EOT +``` + +Restart `containerd` service +```shell +sudo systemctl restart containerd +``` + +Verify the Starlight snapshotter plugin is functioning +```shell +sudo ctr plugin ls | grep starlight +# io.containerd.snapshotter.v1 starlight - ok +``` + +### 6. Convert Container Image + +Convert the container image to the **Starlight format** container image and report to the Starlight proxy. + + +```shell +sudo ctr-starlight convert \ + --insecure-destination \ + --notify --profile myproxy \ + --platform linux/amd64 \ + docker.io/library/redis:6.2.1 cloud.cluster.local/redis:6.2.1-starlight && \ +sudo ctr-starlight convert \ + --insecure-destination \ + --notify --profile myproxy \ + --platform linux/amd64 \ + docker.io/library/redis:6.2.2 cloud.cluster.local/redis:6.2.2-starlight +``` + +In this example, we load two versions of the Redis container image from docker hub and convert them to the Starlight +format container image and notify the Starlight proxy (using `--notify` flag). + +### 7. Optimize Container Image + +Set `starlight` as the default containerd snapshotter in command line (optional). +```shell +export CONTAINERD_SNAPSHOTTER=starlight +``` + +Collect traces on the worker for container startup. +```shell +sudo ctr-starlight optimizer on && \ +sudo ctr-starlight pull --profile myproxy cloud.cluster.local/redis:6.2.1-starlight && \ +mkdir /tmp/test-redis-data && \ +sudo ctr c create \ + --snapshotter=starlight \ + --mount type=bind,src=/tmp/test-redis-data,dst=/data,options=rbind:rw \ + --env-file ./demo/config/all.env \ + --net-host \ + cloud.cluster.local/redis:6.2.1-starlight \ + instance1 /usr/local/bin/redis-server && \ +sudo ctr task start instance1 +``` + +You may terminate the container using `Ctrl-C`, and remove the container: +```shell +sudo ctr container rm instance1 +``` + +Repeat the same thing for `redis:6.2.2` +```shell +sudo ctr-starlight pull --profile myproxy cloud.cluster.local/redis:6.2.2-starlight && \ +sudo ctr c create \ + --snapshotter=starlight \ + --mount type=bind,src=/tmp/test-redis-data,dst=/data,options=rbind:rw \ + --env-file ./demo/config/all.env \ + --net-host \ + cloud.cluster.local/redis:6.2.2-starlight \ + instance2 /usr/local/bin/redis-server && \ +sudo ctr task start instance2 +``` + +Terminate the container using `Ctrl-C`, and remove the container: +```shell +sudo ctr container rm instance2 +``` + +Report traces to the Starlight Proxy. +```shell +sudo ctr-starlight optimizer off && \ +sudo ctr-starlight report --profile myproxy +# set optimizer: completed request +# sha256:291220ae234f1aa9655359d7e553b05fa9e288fd811b7429f229e5aecd64a181: collected 40.966s file access traces - okay +# sha256:561c5b8bb95e26feb56b2bfda1d1fe2aee3229e5d9a2cc879e7524f05ad427a8: collected 13.459s file access traces - okay +# reported traces: uploaded traces +``` + + +### 8. Clear all the cache and reset the environment +```shell +# This script will kill containerd and starlight. +# And then it removes ALL contents +# in /var/lib/containerd, /var/lib/starlight, and /tmp/test-redis-data (the redis data directory) +# and unmount all the mount points in /var/lib/starlight/mounts +sudo ./demo/reset.sh + +# Confirm that the cache is cleared +sudo ls -al /var/lib/containerd +# > ls: cannot access '/var/lib/containerd': No such file or directory +sudo ls -al /var/lib/starlight +# > ls: cannot access '/var/lib/starlight': No such file or directory + + +# restart the processes +sudo systemctl start starlight containerd +``` + +### 9. (Optional) Check the metadata database + +You could also inspect the metadata database using the Adminer Web UI at `http://:8080/` (system: `PostgreQL`,server: `db`, username: `postgres`, password is the same as the username). + +In the `file` table, you could see it records the access order of the file. + +![adminer](./images/metadatadb-screenshot.png) + + +### 10. Deploying and update container + +Start a container using Starlight (it should be much faster) +```shell +sudo ctr-starlight pull --profile myproxy cloud.cluster.local/redis:6.2.1-starlight && \ +mkdir /tmp/test-redis-data && \ +sudo ctr c create \ + --snapshotter=starlight \ + --mount type=bind,src=/tmp/test-redis-data,dst=/data,options=rbind:rw \ + --env-file ./demo/config/all.env \ + --net-host \ + cloud.cluster.local/redis:6.2.1-starlight \ + instance3 /usr/local/bin/redis-server && \ +sudo ctr task start instance3 +``` + +Update a container using Starlight (also way faster) +```shell +sudo ctr-starlight pull --profile myproxy cloud.cluster.local/redis:6.2.2-starlight && \ +sudo ctr c create \ + --snapshotter=starlight \ + --mount type=bind,src=/tmp/test-redis-data,dst=/data,options=rbind:rw \ + --env-file ./demo/config/all.env \ + --net-host \ + cloud.cluster.local/redis:6.2.2-starlight \ + instance4 /usr/local/bin/redis-server && \ +sudo ctr task start instance4 +``` diff --git a/docs/deprecated-v0.1/starlight-proxy.md b/docs/deprecated-v0.1/starlight-proxy.md new file mode 100644 index 0000000..456fb03 --- /dev/null +++ b/docs/deprecated-v0.1/starlight-proxy.md @@ -0,0 +1,208 @@ +# Starlight Proxy + +**⚠️ This document is outdated. Please use https://github.com/mc256/starlight/blob/master/docs/newbie.md instead** + +This is the **Step 1** to use Starlight: + +Set up a Starlight proxy, ideally close to the registry server you are using. +Configure the proxy server to point to the registry and run it. Starlight supports any standard registry. + +[⬅️ Back to README.md](https://github.com/mc256/starlight#getting-started) + +--- +## Method 0. Use Helm (Recommended) + +```shell +helm install my-starlight-proxy oci://ghcr.io/mc256/starlight/starlight-proxy-chart --version 0.1.2 +``` + +You may need to customize the chart. Helm chart configuration are [available here](https://github.com/mc256/starlight/blob/master/docs/helm.md). + + +## Method 1. Use Docker Compose to deploy Starlight Proxy + Container Registry (Recommended) + +This is an all-in-one example in case you don't have full access to a container registry. +We could use Docker Compose to deploy both the proxy and the registry on the same machine. + + +0. Install [Docker](https://docs.docker.com/engine/install/ubuntu/#install-using-the-repository) and [Docker Compose](https://docs.docker.com/compose/install/) + +If using Ubuntu 20.04 LTS, you could install Docker and Docker Compose using the following commands: +```shell +sudo apt update && \ +sudo apt upgrade -y && \ +sudo apt install -y docker-compose && \ +sudo usermod -aG docker $USER +``` +After adding the current user to the `docker` group, you may _need to log out and log in_ to take effect. +To confirm that Docker is working with correct permission, `docker ps` should not print any errors. +```shell +docker ps +# CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES +``` + +1. Clone this project and launch the registry and proxy containers from `./demo/compose/registry+proxy` + +```shell +git clone https://github.com/mc256/starlight.git && \ +cd starlight/demo/compose/registry+proxy && \ +docker-compose up -d +# Creating network "registryproxy_default" with the default driver +# Creating starlightproxy ... done +# Creating starlightregistry ... done +``` +The Starlight proxy writes image metadata to `./data_proxy` folder, and +the container registry saves container images to `./data_registry` + + +2. Verify the registry and proxy are running. +```shell +curl http://localhost:8090/ +# Starlight Proxy OK! +curl http://localhost:5000/v2/ +# {} +``` + +The Starlight proxy listens on port 8090. +We could put a Nginx reverse proxy to handle SSL certificates or load balancing. +But for simplicity, this part is ignored in this example. +Please add port 8090 and 5000 to the firewall whitelist, the worker has to access these ports. + +🙌 That's it. Please obtain the IP address of the server and proceed to the **Step 2**. + +```shell +# update the IP address keep this for future use. +export STARLIGHT_PROXY=:8090 +export REGISTRY=:5000 +``` + +[⬅️ Back to README.md](https://github.com/mc256/starlight#getting-started) + +--- +## Method 2. Use Docker Compose (Starlight Only) + +The prebuilt Starlight proxy container image is available at `ghcr.io/mc256/starlight/proxy:latest`. + +0. Install [Docker](https://docs.docker.com/engine/install/ubuntu/#install-using-the-repository) and [Docker Compose](https://docs.docker.com/compose/install/) + +If using Ubuntu 20.04 LTS, you could install Docker and Docker Compose using the following commands: +```shell +sudo apt update && \ +sudo apt upgrade -y && \ +sudo apt install -y docker-compose && \ +sudo usermod -aG docker $USER +``` +After adding the current user to the `docker` group, you may need to log out and log in to take effect. +To confirm that Docker is working with correct permission, `docker ps` should not print any errors. +```shell +docker ps +# CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES +``` + +1. Clone this project + +```shell +git clone https://github.com/mc256/starlight.git && \ +``` + +2. Set `REGISTRY` environment variable to your own container registry. + +```shell +echo "REGISTRY=http://starlightregistry:5000" >> ./starlight/demo/compose/proxy/.env +``` + +3. Launch the proxy +```shell +cd ./starlight/demo/compose/proxy && \ +docker-compose up -d +# Creating starlightproxy ... done +``` + +The Starlight proxy writes image metadata to `./data_proxy` folder. + +2. Verify the registry and proxy are running. +```shell +curl http://localhost:8090/ +# Starlight Proxy OK! +``` + +The Starlight proxy listens on port 8090. +We could put a Nginx reverse proxy to handle SSL certificates or load balancing. +But for simplicity, this part is ignored in this example. + +🙌 That's it. Please obtain the IP address of the server and proceed to the **Step 2**. + +```shell +# update the IP address keep this for future use. +export STARLIGHT_PROXY=:8090 +export REGISTRY=:5000 +``` + +[⬅️ Back to README.md](https://github.com/mc256/starlight#getting-started) + +--- +## Method 3. Build from source + +0. Install Go https://go.dev/doc/install ➡️ +```shell +wget https://go.dev/dl/go1.17.8.linux-amd64.tar.gz && +sudo rm -rf /usr/local/go && sudo tar -C /usr/local -xzf go1.17.8.linux-amd64.tar.gz +``` + +1. Add Go to the environment variable (You may want to change `.zshrc` or `.bashrc` file to permanently add this folder to the `PATH` environment variable) +``` +export PATH=$PATH:/usr/local/go/bin +``` + +2. Verify Go is available with `go version` +```shell +go version +#go version go1.17.8 linux/amd64 +``` + +4. Install necessary tools to build this project + +```shell +sudo apt update && \ +sudo apt upgrade -y && \ +sudo apt install build-essential +``` + +4. Clone this project. + +```shell +git clone https://github.com/mc256/starlight.git && \ +cd starlight +``` + +5. Build Starlight proxy +```shell +make starlight-proxy +``` + +6. Run Starlight +```shell +cd ./out && \ +mkdir ./data && \ +./starlight-proxy --registry=http://myregistry:5000 & +``` + +7. Verify the Starlight Proxy is working +```shell +curl http://localhost:8090/ +# Starlight Proxy OK! +``` + +The Starlight proxy listens on port 8090. +We could put a Nginx reverse proxy to handle SSL certificates or load balancing. +But for simplicity, this part is ignored in this example. + +🙌 That's it. Please obtain the IP address of the server and proceed to the **Step 2**. + +```shell +# update the IP address keep this for future use. +export STARLIGHT_PROXY=:8090 +export REGISTRY=:5000 +``` + +[⬅️ Back to README.md](https://github.com/mc256/starlight#getting-started) diff --git a/docs/deprecated-v0.1/starlight-snapshotter.md b/docs/deprecated-v0.1/starlight-snapshotter.md new file mode 100644 index 0000000..f84c15b --- /dev/null +++ b/docs/deprecated-v0.1/starlight-snapshotter.md @@ -0,0 +1,207 @@ +# Starlight Snapshotter Plugin + +**⚠️ This document is outdated. Please use https://github.com/mc256/starlight/blob/master/docs/newbie.md instead** + +This is the **Step 2** to use Starlight: + +Set up the worker to be able to run Starlight. +This involves +installing **containerd** and the **Starlight snapshotter plugin**, +configuring containerd to use the plugin, +and starting the Starlight snapshotter daemon +(you also need to tell the snapshotter the address of the proxy server). + +[⬅️ Back to README.md](https://github.com/mc256/starlight) + +--- + +## Method 1. Install Pre-built Package (Recommended) + +Pre-build deb package is available for `amd64`, `armhf`, and `arm64`. + +### 1. Install Starlight Snapshotter + +Download and install the `.deb` package from the [release page](https://github.com/mc256/starlight/releases). + +```shell +export ARCH=amd64 +export SL_VERSION=0.1.2 +wget "https://github.com/mc256/starlight/releases/download/v${SL_VERSION}/starlight-snapshotter_${SL_VERSION}_$ARCH.deb" +sudo apt install -f "./starlight-snapshotter_${SL_VERSION}_$ARCH.deb" +``` + +Update systemd service file `/lib/systemd/system/starlight.service`. +- Change `STARLIGHT_PROXY` to the address of the Starlight Proxy. +- remove `--plain-http` if the Starlight Proxy is behind a HTTPS reverse proxy. +``` +ExecStart=/usr/bin/starlight-grpc run --plain-http starlight.lan +``` + +Reload systemd service +```shell +sudo systemctl daemon-reload +sudo systemctl restart starlight-snapshotter +``` + +### 2. Configure `contaienrd` + +Add configuration to `/etc/containerd/config.toml`. +(If you have set other `proxy_plugins`, please manually edit the file) +```shell +sudo mkdir /etc/containerd/ && \ +cat < /dev/null +[proxy_plugins] + [proxy_plugins.starlight] + type = "snapshot" + address = "/run/starlight-grpc/starlight-snapshotter.socket" +EOT +``` + +Restart `containerd` service +```shell +sudo systemctl restart containerd +``` + +Verify the Starlight snapshotter plugin is functioning +```shell +sudo ctr plugin ls | grep starlight +# io.containerd.snapshotter.v1 starlight - ok +``` + + +🙌 That's it. Please proceed to the **Step 3**. + +[⬅️ Back to README.md](https://github.com/mc256/starlight#getting-started) + + + +--- + + + +## Method 2. Build from source + +### 1. Install Dependencies + +The worker machine is supposed to be far away (in latency) to the registry and proxy. +Please install **containerd** and **Starlight snapshotter** on a new machine (or VM), not the same machine that runs the proxy or the registry. + +The worker machine needs `build-essential` and `containerd`. +```shell +sudo apt update && sudo apt upgrade -y && \ +sudo apt install -y build-essential containerd +``` + +Enable `containerd` +```shell +sudo systemctl enable containerd && \ +sudo systemctl start containerd +``` + +Verify `containerd` is running +```shell +sudo systemctl status containerd +# Active: active +``` + +Install Go https://go.dev/doc/install ➡️ +```shell +wget https://go.dev/dl/go1.17.8.linux-amd64.tar.gz && \ +sudo rm -rf /usr/local/go && sudo tar -C /usr/local -xzf go1.17.8.linux-amd64.tar.gz +``` + +Add Go to the environment variable (You may want to change `.zshrc` or `.bashrc` file to permanently add this folder to the `PATH` environment variable) +```shell +export PATH=$PATH:/usr/local/go/bin +``` + +Verify Go is available +```shell +go version +# go version go1.17.8 linux/amd64 +``` + +### 2. Clone and Build +Clone the Starlight repository +```shell +git clone https://github.com/mc256/starlight.git && \ +cd starlight +``` + +Build the snapshotter plugin and CLI tool +```shell +make starlight-grpc ctr-starlight +``` + +### 3. Configure Starlight Snapshotter + +You need to find out the IP address / DNS of the Starlight Proxy server (in **Step 1**. [Find out how to install **Starlight proxy** ➡️](docs/starlight-proxy.md) ) + +```shell +# This is an example +export STARLIGHT_PROXY=172.18.1.3:8090 +export REGISTRY=172.18.1.3:5000 +``` + +Verify that the Starlight proxy is accessible from the worker. +```shell +curl http://$STARLIGHT_PROXY +# Starlight Proxy OK! +``` + +Install Starlight Snapshotter `systemd` service and CLI tool. +Please follow the prompt, enter +```shell +sudo make install install-systemd-service +#Please enter Starlight Proxy address (example: proxy.mc256.dev:8090):172.18.1.3:8090 +#Enable HTTPS Certificate (requires load balancer like Nginx) (y/N):n +#Created systemd service file (/lib/systemd/system/starlight.service) +#Reloaded systemd daemon +``` + +Enable Starlight snapshotter service +```shell +sudo systemctl enable starlight && \ +sudo systemctl start starlight +``` + +Verify Starlight is running +```shell +sudo systemctl status starlight +# it should be "active". +``` + +### 4. Configure `contaienrd` + +Add configuration to `/etc/containerd/config.toml`. +(If you have set other `proxy_plugins`, please manually edit the file) +```shell +sudo mkdir /etc/containerd/ && \ +cat < /dev/null +[proxy_plugins] + [proxy_plugins.starlight] + type = "snapshot" + address = "/run/starlight-grpc/starlight-snapshotter.socket" +EOT +``` + +Restart `containerd` service +```shell +sudo systemctl restart containerd +``` + +Verify the Starlight snapshotter plugin is functioning +```shell +sudo ctr plugin ls | grep starlight +# io.containerd.snapshotter.v1 starlight - ok +``` + + +🙌 That's it. Please proceed to the **Step 3**. + +[⬅️ Back to README.md](https://github.com/mc256/starlight#getting-started) + + +--- + +For more information, please see `ctr-starlight --help` and `starlight-grpc --help` diff --git a/docs/deprecated-v0.1/starlight-workflow.md b/docs/deprecated-v0.1/starlight-workflow.md new file mode 100644 index 0000000..e5eac96 --- /dev/null +++ b/docs/deprecated-v0.1/starlight-workflow.md @@ -0,0 +1,19 @@ +# How Starlight Works (Overview) +![starlight-workflow](images/starlight-workflow.png) + +Once the user issues a worker `PULL` command to download a set of containers ①, +the command is received by the standard **containerd** daemon. +**containerd** then forwards the command to the **Starlight snapshotter** daemon ②, +and waits for confirmation that the requested images have been found. +The Starlight snapshotter opens a connection to the **Starlight proxy** +and sends the list of requested containers as well as the list of relevant containers that already exist on the worker ③. +The proxy queries the directory database ④ for the list of files in the various layers of the +requested container image, as well in the image already available on the worker. + +The proxy will then begin computing the **delta bundle** that includes the set of distinct compressed file contents that the worker does not already have, specifically organized to speed up deployment; +In the background, the proxy also responds with HTTP 200 OK header to the snapshotter, which notifies **containerd** that the `PULL` phase has finished successfully; the snapshotter however, remains active and keeps the connection open to receive the data from the proxy. +In the background, the proxy issues a series of requests to the registry ⑦ to retrieve the compressed contents of files needed for delta bundle. +Once the contents of the delta bundle has been computed, the proxy creates a **Starlight manifest** (SLM) -- the list of file metadata, container manifests, and other required metadata -- and sends it to the snapshotter ⑤, +which notifies **containerd** that the `PULL` phase has finished successfully. + +Please read our [NSDI '22 paper](https://www.usenix.org/conference/nsdi22/presentation/chen-jun-lin) for more details. diff --git a/docs/terraform.md b/docs/terraform.md new file mode 100644 index 0000000..475ad81 --- /dev/null +++ b/docs/terraform.md @@ -0,0 +1,46 @@ +# Setup Starlight Experiment using Terraform + +## Prerequisites +- [Terraform](https://learn.hashicorp.com/tutorials/terraform/install-cli) +- [AWS account](https://aws.amazon.com/) You will need to setup programmatic access to AWS (e.g. set up credentials in `$HOME/.aws/config` and `$HOME/.aws/credentials`). + + +## Install +1. Clone the repository + ```shell + git clone https://github.com/mc256/starlight.git + cd starlight/demo/terraform + ``` + +2. Initialize Terraform + ```shell + terraform init + ``` + +3. Modify `terraform.tfvars` to your needs. + + +4. Apply the configuration + ```shell + terraform apply + ``` + +5. Wait for the infrastructure to be created. This may take a few minutes. After the infrastructure is create you can see there is a `.completed` file in the home directory. + +## Experiment + +1. SSH into the Starlight CLI Tool pods in the edge node. + ```shell + ssh -i ubuntu@ + ``` + +2. Run the experiment + + + +## Uninstall + +1. Destroy the infrastructure + ```shell + terraform destroy + ``` \ No newline at end of file