docs(tutorial): how to create a custom openWRT image for SCW Instances

tutorials/create-openwrt-image-for-scaleway/index.mdx

@@ -0,0 +1,344 @@
+---
+meta:
+  title: How to create an openWRT image for Scaleway Instances
+  description: This page shows how to create an openWRT image for Scaleway Instances
+content:
+  h1: How to create an openWRT image for Scaleway Instances
+  paragraph: This page shows how to create an openWRT image for Scaleway Instances
+categories:
+  - compute
+  - instances
+tags: openwrt image instances custom-image qcow2
+dates:
+  validation: 2022-10-24
+  posted: 2022-10-24
+---
+
+This tutorial shows how to build a custom image for Scaleway from scratch using the new export/import feature.
+
+It presents the basic needs and actions to create a custom image, but each operating system may have specifics which need to be troubleshooted individually.
+
+<Message type="important">
+
+The procedure presented in this tutorial is specific to OpenWRT, an open-source project for embedded operating systems based on Linux. It is primarily used to route network traffic.
+
+</Message>
+
+## Overview
+
+To create a custom image, you need to build a [QCOW2 image](https://www.linux-kvm.org/page/Qcow2) and [create a snapshot](/compute/instances/how-to/create-a-snapshot/) from it.
+
+The following steps are required to create the image:
+
+- Download an OS disk image
+- Convert it to QCOW2 (if not provided in this format)
+- Edit the image to fit Scaleway's ecosystem
+- Upload the image to Scaleway Object Storage
+- Convert to Snapshot through import
+- Test the image and troubleshoot via Console
+
+<Message type="requirement">
+
+  - You have an account and are logged into the [Scaleway Console](https://console.scaleway.com/)
+  - You have [generated an SSH Key](/console/my-project/how-to/create-ssh-key/)
+  - You have [created](/storage/object/how-to/create-a-bucket/) a Scaleway Object Storage Bucket
+
+</Message>
+
+The following commands are done on an Ubuntu 22.04.
+
+## Download the image
+
+The needed image must be a full disk image, not an ISO image or only a rootfs.
+
+In order to work on Scaleway Instances, the image must be using EFI (not just BIOS) to boot.
+
+OpenWRT images are available [here](https://openwrt.org/downloads), we are going to use a stable release, for `x86_64` architecture, with EFI.
+
+Download the image we are going to use in this tutorial:
+
+  ```
+  curl -sSLO https://downloads.openwrt.org/releases/22.03.2/targets/x86/64/openwrt-22.03.2-x86-64-generic-ext4-combined-efi.img.gz
+  gunzip openwrt-22.03.2-x86-64-generic-ext4-combined-efi.img.gz
+  ```
+
+## Prepare the QCOW2 image
+
+<Message type="note">
+
+QCOW2 images are container images used in [QEMU](https://www.qemu.org/) as virtual disks.
+
+</Message>
+
+1. Install the needed QEMU tools:
+
+    ```
+    apt-get install qemu-utils -y
+    ```
+
+2. Convert the image and resize the target disk size (minimum disk size at Scaleway = 1GB):
+
+    ```
+    qemu-img convert -f raw -O qcow2 openwrt-22.03.2-x86-64-generic-ext4-combined-efi.img openwrt-22.03.2-x86-64-generic-ext4-combined-efi.qcow2
+    qemu-img resize openwrt-22.03.2-x86-64-generic-ext4-combined-efi.qcow2 1G
+    ```
+
+3. Mount the QCOW2 image as a device
+
+    ```
+    modprobe nbd
+    qemu-nbd -c /dev/nbd0 openwrt-22.03.2-x86-64-generic-ext4-combined-efi.qcow2
+    ```
+
+    <Message type="important">
+
+    Conversion and resizing creates errors in the partition table which are fixed when re-writing it using fdisk. Furthermore, To resize the main partition, the partition number must be found out.
+
+    To print and fix partition table, do:
+
+    ```
+    echo 'p\nw\n' | fdisk /dev/nbd0
+    ```
+
+    </Message>
+
+4. Resize the main partition to fit the available space (partition number 2 in the example above):
+
+    ```
+    growpart /dev/nbd0 2
+    resize2fs /dev/nbd0p2
+    ```
+
+## Edit the image content
+
+<Message type="note">
+
+Network configuration and specific actions may need to be configured in order to access the image when run in Scaleway Instances.
+
+For example:
+
+* In Scaleway Instances, the first NIC (e.g. `eth0`) must be associated to the public network and need to be configured (For exapmple using: Cloud-init, DHCP, ...)
+
+* If the image provides a GUI, it may need to be configured to allow access from public interface (are specific ports or firewall rules required?)
+
+* How to manage the the SSH access (set user password or configure SSH keys?)
+
+</Message>
+
+
+1. Mount the image:
+
+    ```
+    mkdir -p /mount/temp
+    mount /dev/nbd0p2 /mount/temp
+    ```
+
+    OpenWRT needs two interfaces to work (WAN and LAN). But a default Instance only provide one interface. We are going to use the dummy module to add an interface.
+
+2. Download the package.
+
+    ```
+    curl -sSL -o /mount/temp/kmod-dummy_5.10.146-1_x86_64.ipk https://downloads.openwrt.org/releases/22.03.2/targets/x86/64/packages/kmod-dummy_5.10.146-1_x86_64.ipk
+    ```
+
+3. Chroot in the image.
+
+    ```
+    chroot /mount/temp/ /bin/ash
+    ```
+
+4. Install the package.
+
+    ```
+    mkdir -p /tmp/lock
+    opkg install kmod-dummy_5.10.146-1_x86_64.ipk
+    rm -rf /tmp/* kmod-dummy_5.10.146-1_x86_64.ipk
+    ```
+
+5. Set the root password, whilst in the chroot. In OpenWRT, it is used by the web interface for login.
+
+    ```
+    passwd
+    ```
+
+6. Configure the web server ports and redirect `http` to `https` using the `uci` CLI of OpenWRT.
+
+    ```
+    uci del uhttpd.main.listen_http
+    uci add_list uhttpd.main.listen_http='0.0.0.0:8080'
+    uci del uhttpd.main.listen_https
+    uci add_list uhttpd.main.listen_https='0.0.0.0:8443'
+    uci set uhttpd.main.redirect_https=1
+    uci commit uhttpd
+    ```
+
+7. Disable password authentication in SSH.
+
+    ```
+    uci set dropbear.@dropbear[0].PasswordAuth=off
+    uci set dropbear.@dropbear[0].RootPasswordAuth=off
+    uci commit dropbear
+    ```
+### Configure SSH access
+
+Since we have disabled password authentication in SSH, we need a way to load SSH keys when running.
+
+In this tutorial we do not set up Cloud-init, but use the same magic IP mechanism to get the keys.
+
+1. Create the `authorized_keys` file:
+
+    ```
+    touch /etc/dropbear/authorized_keys
+    chmod 600 /etc/dropbear/authorized_keys
+    ```
+
+2. Create the fetch script:
+
+    ```sh
+    cat <<EOF>/etc/init.d/fetch_ssh_keys
+    #!/bin/sh /etc/rc.common
+
+    START=97
+
+    start() {
+        echo -e "\nFetching SSH keys from Scaleway Metadata" > /dev/console
+        wget -qO- http://169.254.42.42/conf | egrep 'SSH_PUBLIC_KEYS_._KEY' | cut -d'=' -f2- | sed "s/'//g" > /etc/dropbear/authorized_keys
+    }
+
+    reload() {
+        start
+    }
+    EOF
+    chmod +x /etc/init.d/fetch_ssh_keys
+    ```
+
+3. Add the script to `rc.d`.
+
+    ```sh
+    ln -s /etc/init.d/fetch_ssh_keys /etc/rc.d/S97fetch_ssh_keys
+    ```
+
+4. Exit the chroot.
+
+    ```
+    exit
+    ```
+
+### Configure the network
+
+1. Configure the network to be able to use the image on a Scaleway Instance:
+
+    ```
+    cat <<EOF>/mount/temp/etc/config/network
+    config interface 'loopback'
+    option device 'lo'
+    option proto 'static'
+    option ipaddr '127.0.0.1'
+    option netmask '255.0.0.0'
+
+    config interface 'wan'
+    option device 'eth0'
+    option proto 'dhcp'
+
+    config interface 'lan'
+    option device 'dummy0'
+    option proto 'static'
+    option ipaddr '192.168.0.1'
+    option netmask '255.255.255.0'
+    EOF
+    ```
+
+    <Message type="important">
+
+    `eth0` is the first interface associated by Scaleway Instances to the public network (WAN) and need to be available to our DHCP server for configuration.
+
+    </Message>
+
+2. Add firewall rules to redirect WAN ports to LAN ports. This is required, as accessing the UI and SSH in OpenWRT is normally only possible through the LAN interface.
+
+    ```
+    cat <<EOF>>/mount/temp/etc/config/firewall
+    # port redirect UI ACCESS and SSH
+    config redirect
+    option name             HTTP-UI
+    option src              wan
+    option src_dport        8080
+    option desti            lan
+    option dest_ip          192.168.0.1
+    option dest_port        8080
+    option proto            tcp
+
+    config redirect
+    option name             HTTPS-UI
+    option src              wan
+    option src_dport        8443
+    option dest             lan
+    option dest_ip          192.168.0.1
+    option dest_port        8443
+    option proto            tcp
+
+    config redirect
+    option name             SSH
+    option src              wan
+    option src_dport        22
+    option dest             lan
+    option dest_ip          192.168.0.1
+    option dest_port        22
+    option proto            tcp
+    EOF
+    ```
+
+3. Unmount the image container to finish editing its content.
+
+    ```
+    umount /mount/temp
+    qemu-nbd -d /dev/nbd0
+    ```
+
+## Import the image
+
+You can use the Scaleway console or your favorite S3 CLI to upload objects into a bucket.
+
+In this example, we use the [AWS CLI](https://www.scaleway.com/en/docs/storage/object/api-cli/object-storage-aws-cli/).
+
+1. Import the image into your bucket: 
+```
+aws s3 cp openwrt-22.03.2-x86-64-generic-ext4-combined-efi.qcow2 s3://my-import-bucket/openwrt.qcow2
+```
+
+2. Trigger the import of the image as a snapshot in one of the regions where the bucket is located, using [SCW CLI](https://github.com/scaleway/scaleway-cli):
+
+```
+scw instance snapshot create zone=fr-par-1 name=openwrt volume-type=unified bucket=my-import-bucket key=openwrt.qcow2
+```
+
+3. Create the corresponding Instance image:
+
+```
+scw instance image create zone=fr-par-1 name=openwrt arch=x86_64 snapshot-id=$(scw instance snapshot list | grep -m1 openwrt | awk '{print $1}')
+```
+
+
+## Testing and troubleshooting the image
+
+1. Create an Instance using the snapshot image:
+
+```
+scw instance server create type=DEV1-S zone=fr-par-1 name=scw-openwrt ip=new image=$(scw instance image list | grep -m1 openwrt | awk '{print $1}')
+```
+
+2. Troubleshoot the boot behaviour and configuration using the console of your Instance:
+
+```
+scw instance server console zone=fr-par-1 $(scw instance server list | grep -m1 scw-openwrt | awk '{print $1}')
+```
+
+3. Exit the console using `Ctrl+q`.
+
+You can access the web console of OpenWRT using `https://<public ip of instance>:8443`
+
+<Message type="important">
+
+Scaleway does not provide support for these kind of images. Building and running of custom images is managed by the users themselves.
+
+</Message>