Cloud-init for On-Prem appliances using TinyCore Linux
It is split into 3 sub-directories:
- extensions: Build scripts for creating the required TinyCore
- overlay: File and directory structure which should be copied directly into the rootfs
- .github/workflows: GitHub Actions workflow to automatically build each
.tczextension, and publish them to Bintray
Cloud-init on TinyCore Linux
The current version of cloud-init (
v20.2) does not support TinyCore Linux. It is designed for systems such as Debian, CentOS, FreeBSD, etc. Lack of support for TinyCore leads us to two options:
- Modify cloud-init and its Python code to work with TinyCore Linux
- Create custom scripts for operations specific to TinyCore Linux
Our use-case is to support immutable On-Prem TinyCore appliances, which presents another set of problems since the system state is reset at every boot. The Python interpreter is slow and the programming language is less than ideal, so tweaking it to work for TinyCore was not a good choice.
For this reason, we've created a set of simple custom Shell scripts to handle most initialization tasks, while leveraging the few tasks cloud-init handles well, such as detecting metadata and userdata from various cloud providers.
- TinyCore Linux 10.x x86-64
- overlay directory structure added to rootfs
- python3.6.tcz extension
- ifupdown.tcz, cloud-init.tcz, cloud-init-deps.tcz extensions
How it works
/opt/bootsync.sh will run
cloud-init and then try to setup networking through
/opt/network.sh, which is simply calling
cloud-init once more with different arguments. The reason for this is to provide backward compatibility for existing On-Prem TinyCore deployments.
/usr/local/bin/cloud-init modules --mode config
Each step in the cloud-init runs can be found in
config modules are
bootcmd which are the actual Shell scripts found in
Customizations can and should be added to
custom-once.sh. If it's not immediately obvious, the
custom-once.sh scripts will only run once per boot. Unlike typical cloud-init installations, the once scripts will run again on reboot (because the OS is immutable).
What do these scripts configure
The scripts will configure the
NTP server address,
DNS nameservers, the
hosts file, the
network config for
iptables firewall, and additional
storage (disk2) via
LVM, NFS, or AoE.
storage will not be configured unless the
nfs-utils.tcz extensions are loaded.
Backups and restoring data
The data stored in
/run/cloud-init should not be backed up, as it will prevent userdata and metadata from being altered on the Host OS, and any customized settings will not be retained on reboot.
The following files can be backed up (added to
If they are backed up, on reboot they will overwrite any values from
vendordata, and will help speed up the boot process. Customizations to values (ex:
etc/hostname) will also be retained.
If they are not backed up, the initial config will take a bit more time.
Userdata, Metadata, Vendordata
The Shell scripts provided for this cloud-init deployment will read settings from various files on the system, including any userdata or vendordata provided to the system.
The order of precendence for reading configuration values are:
- backed up files from
- userdata from
- vendordata from
Note: Metadata is only currently used by cloud-init to configure
SSH public-keys. All other metadata is ignored, but still accessible to applications running on the TinyCore Linux appliance.
EC2, Proxmox, and NoCloud, the userdata will be discovered by cloud-init and read directly from
/var/lib/cloud/instance/user-data.txt. From testing, it appears cloud-init handles Base64 decoding of userdata automatically provided by EC2.
Note: Userdata is not parsed by cloud-init, but rather read manually by various Shell scripts. This means it is not possible for
userdata to overwrite values from
/etc/cloud/cloud.cfg, and it is not possible to provide
userdata which contains a shell script or other malicious features. Userdata should only contain
key: value pairs.
vendor-data should be added to the
/var/lib/cloud/seed/nocloud-net/ directory on the rootfs. It is also possible to add a
network-config (v1 config) to that directory for pre-configuring the network.
Note: Similar to
vendor-data is not parsed by cloud-init and should only contain
key: value pairs (except for
network-config, which is parsed by cloud-init's
All extensions are built from a
Makefile under their respective directory. The extension's config values are stored in
config.make (ex: version number, source URL). The extension's Open Source LICENSE file is also included.
Makefile will load the
build.make file which performs generic tasks such as adjusting file permissions, creating the
.tcz squashfs file, and cleaning up.
ifupdown.tcz is taken directly from the Debian repositories, and built with a patch to remove the need for
run-parts. This provides the
ifup, ifdown, ifquery commands to TinyCore Linux, working with the typical Debian
/etc/network/interfaces file, however it can not run scripts, for security reasons.
cloud-init.tcz is fetched from the official Canonical Cloud-Init repository, and built with standard Python 3 commands with no changes or patches. It contains the entire
cloud-init installation without its dependencies.
cloud-init-deps.tcz is built similarly to
cloud-init.tcz and contains all the dependencies for
First, it is important to load the required build tools in a TinyCore Linux environment:
tce-load -wicl git compiletc coreutils
To manually build an extension,
cd into the directory of the extension and type:
11.0-x86_64, if that variable is omitted.
The files should be output to
$HOME/artifacts in a subdirectory with the name of the extension and version of TinyCore. It will contain a simple
.info file, the
md5 and sha256 hashes, the
.dep files, and of course the
.tcz extension file.
The GitHub workflow runs three concurrent builds, one for each extension. It downloads the JFrog CLI tool if the extension was built correctly, and uploads the files to Bintray for further review (they are not published automatically).
Everything in this repository is licensed under the MIT License, except for the
LICENSE file of each individual extension.
Copyright (c) 2020 Alexander Williams, Unscramble email@example.com