diff --git a/README.md b/README.md
index 260d1d7..b5d32c5 100644
--- a/README.md
+++ b/README.md
@@ -19,6 +19,7 @@ Following features and tools are supported:
* 🐟 Fish Shell
* 📷 AzCopy
* 🪪 Certificates
+* 📨 cmctl
* ⚙️ Direnv
* ⛵️ Helm
* 🛠 JQ
@@ -30,6 +31,7 @@ Following features and tools are supported:
* 📦 Packages
* 📦 Packer
* 👟 Run
+* 🔑 sops
* 📜 Stern
* 🌏 Terraform
* 🐗 Terragrunt
@@ -53,6 +55,7 @@ Following features and tools are supported:
* [Fish Shell](#_fish)
* [AzCopy](#azcopy)
* [Certificates](#certificates)
+ * [cmctl](#cmctl)
* [Direnv](#direnv)
* [Helm](#helm)
* [JQ](#jq)
@@ -64,6 +67,7 @@ Following features and tools are supported:
* [Packages](#packages)
* [Packer](#packer)
* [Run](#run)
+ * [sops](#sops)
* [Stern](#stern)
* [Terraform](#terraform)
* [Terragrunt](#terragrunt)
@@ -393,6 +397,17 @@ Adds specified trusted certificate authorities into the container
(optional). Defaults to `/certificates`. If something different than the default is used, the volume-target needs to be adapted to
the same directory
+### cmctl
+
+Installs the cert-manager Command Line Tool
+
+#### Configuration
+
+* USE_cmctl: Enable this feature
+* DEBUG_cmctl: Debug this feature
+* Environment CMCTL_VERSION: Version of cmctl to install (optional)
+ Defaults to `latest`
+
### Direnv
Installs [Direnv](https://direnv.net/)
@@ -540,6 +555,17 @@ Runs commands inside the shell when entering the cloud control container
* DEBUG_run: Debug this feature
* Environment RUN_COMMANDS: Valid shell commands to run
+### sops
+
+Installs [sops](https://github.com/getsops/sops)
+
+#### Configuration
+
+* USE_sops: Enable this feature
+* DEBUG_sops: Debug this feature
+* Environment SOPS_VERSION (required): Valid sops version (e.g. 3.8.1)
+* Environment specific for the key you use, see [sops documentation](https://github.com/getsops/sops?tab=readme-ov-file#22encrypting-using-age)
+
### Stern
Installs [stern](https://github.com/stern/stern), a multi pod and container log tailing for Kubernetes
@@ -630,14 +656,24 @@ Installs the [YAML parser and processor yq](https://github.com/mikefarah/yq)
## Development
-*CloudControl* supports a decoupled development of features and flavours. If you're missing something, just fork this
-repository, create a subfolder for your new feature under "features" and add these files:
+*CloudControl* supports a decoupled development of features and flavours.
+
+### Features
+
+If you're missing a feature, just fork this repository, copy the feature template from features/.template into a
+new subfolder, check out the comments in the example files, and modify them to your needs.
+
+These files make up a feature:
* `feature.yaml`: A descriptor for your feature with a title, a description and configuration notes
* `install.sh`: A shell script that is run by CloudControlCenter and should install everything you need
for your new feature
* `motd.sh`: (optional) If you want to show some information to the users upon login, put them here.
+And optional, but recommended [integration tests](integration-testing) in a `.goss` folder.
+
+### Flavours
+
If you need another flavour (aka cloud provider), add a new subdirectory under "flavour" and add a flavour.yaml describing
your flavour the same way as a feature. For the rest of the files, please check out existing flavours for details. Please,
include a sample configuration for your flavour to make it easier for other people to work with it.
@@ -775,7 +811,7 @@ you can inspect the failing container as well.
To rebuild this documentation, first compile the documentation maker:
- docker run --rm -e GOOS=[os, e.g. darwin, linux, windows] -e GOARCH=[architecture, e.g. arm64, amd64] -v "$PWD":/usr/src/myapp -w /usr/src/myapp golang:1.19-alpine go run cmd/doc/mkdoc
+ docker run --rm -e GOOS=[os, e.g. darwin, linux, windows] -e GOARCH=[architecture, e.g. arm64, amd64] -v "$PWD":/usr/src/myapp -w /usr/src/myapp golang:1.19-alpine go build cmd/doc/mkdoc.go
Then run it to rebuild README.md based on README.md.gotmpl:
diff --git a/README.md.gotmpl b/README.md.gotmpl
index 07fb2ee..7ba0d0d 100644
--- a/README.md.gotmpl
+++ b/README.md.gotmpl
@@ -283,14 +283,24 @@ environment variable in the docker-compose file. Then you can debug with the run
## Development
-*CloudControl* supports a decoupled development of features and flavours. If you're missing something, just fork this
-repository, create a subfolder for your new feature under "features" and add these files:
+*CloudControl* supports a decoupled development of features and flavours.
+
+### Features
+
+If you're missing a feature, just fork this repository, copy the feature template from features/.template into a
+new subfolder, check out the comments in the example files, and modify them to your needs.
+
+These files make up a feature:
* `feature.yaml`: A descriptor for your feature with a title, a description and configuration notes
* `install.sh`: A shell script that is run by CloudControlCenter and should install everything you need
for your new feature
* `motd.sh`: (optional) If you want to show some information to the users upon login, put them here.
+And optional, but recommended [integration tests](integration-testing) in a `.goss` folder.
+
+### Flavours
+
If you need another flavour (aka cloud provider), add a new subdirectory under "flavour" and add a flavour.yaml describing
your flavour the same way as a feature. For the rest of the files, please check out existing flavours for details. Please,
include a sample configuration for your flavour to make it easier for other people to work with it.
@@ -428,7 +438,7 @@ you can inspect the failing container as well.
To rebuild this documentation, first compile the documentation maker:
- docker run --rm -e GOOS=[os, e.g. darwin, linux, windows] -e GOARCH=[architecture, e.g. arm64, amd64] -v "$PWD":/usr/src/myapp -w /usr/src/myapp golang:1.19-alpine go run cmd/doc/mkdoc
+ docker run --rm -e GOOS=[os, e.g. darwin, linux, windows] -e GOARCH=[architecture, e.g. arm64, amd64] -v "$PWD":/usr/src/myapp -w /usr/src/myapp golang:1.19-alpine go build cmd/doc/mkdoc.go
Then run it to rebuild README.md based on README.md.gotmpl:
diff --git a/assets/feature-installer-utils.sh b/assets/feature-installer-utils.sh
index 5eb02f6..d0f2763 100644
--- a/assets/feature-installer-utils.sh
+++ b/assets/feature-installer-utils.sh
@@ -1,4 +1,29 @@
-# Execute a command and if it fails, return its output and exit
+# Set some informative variables
+
+# The flavour we're running on
+FLAVOUR="$(cat /home/cloudcontrol/flavour)"
+export FLAVOUR
+
+# The path to install software binaries to
+export BINPATH="/home/cloudcontrol/bin"
+
+export TEMPDIR=""
+
+# Prepare feature installation. Will create a temporary directory and change to it
+function prepare {
+ TEMPDIR=$(mktemp -d)
+ cd "${TEMPDIR}" || exit
+}
+
+# Cleanup the previously generated temporary directory
+function cleanup {
+ cd - &>/dev/null || exit
+ rm -rf "${TEMPDIR}"
+}
+
+# Usage: execHandle MESSAGE COMMAND...
+#
+# Output MESSAGE and then execute COMMAND. If it fails, return its output and exit
function execHandle {
TITLE=$1
shift
@@ -38,6 +63,7 @@ function waitForMfaCode {
echo "[VALID_CODE] Valid code entered. Thank you."
}
+# Get the hardware platform we're on and translate it to the usual platform names used in most software
function getPlatform {
if [ "$(uname -m)" == 'aarch64' ]
then
@@ -50,18 +76,32 @@ function getPlatform {
fi
}
+# Usage: checkAndCleanVersion VERSION
+#
+# Includes checks for version numbers and removes the "v" prefix from it to have a homogeneous version scheme
+# throughout CloudControl.
function checkAndCleanVersion {
VERSION=$1
if [ "${VERSION:0:1}" == "v" ]
then
- echo "[DEPRECATION WARNING] Versions with a \"v\" prefix are deprecated and will be removed in CloudControl 4.0. Please only use versions without the \"v\" prefix. (Got \"${VERSION}\")" >&2
+ echo "[DEPRECATION WARNING] Versions with a \"v\" prefix are deprecated and will be removed in CloudControl 6.0.0 Please only use versions without the \"v\" prefix. (Got \"${VERSION}\")" >&2
echo "${VERSION/#v/}"
else
echo "${VERSION}"
fi
}
+# Usage: download URL FILENAME
+#
+# Downloads the given URL into the provided FILENAME
+function download {
+ URL=$1
+ FILENAME=$2
+ execHandle "Downloading ${FILENAME} from ${URL}" curl -f -s -L "${URL}" -o "${FILENAME}"
+}
+
# Usage: downloadFromGithub USER REPO VERSION PACKAGE_PREFIX PACKAGE_SUFFIX TARGET
+#
# Downloads a release package from github using the common architecture names
# The package will be downloaded from github.com/USER/REPO/releases/VERSION/download/PACKAGE to the given TARGET file
# where PACKAGE consists of PACKAGE_PREFIXARCHITECTURE.PACKAGE_SUFFIX.
@@ -77,4 +117,4 @@ function downloadFromGithub {
ARCH="$(uname -m | sed -e 's/x86_64/amd64/' -e 's/\(arm\)\(64\)\?.*/\1\2/' -e 's/aarch64$/arm64/')"
PACKAGE="${PACKAGE_PREFIX}${ARCH}.${PACKAGE_SUFFIX}"
execHandle "Downloading ${USER}/${REPO}@${VERSION}" curl -f -s -L "https://github.com/${USER}/${REPO}/releases/${VERSION}/download/${PACKAGE}" --output "${TARGET}"
-}
\ No newline at end of file
+}
diff --git a/build.sh b/build.sh
index d171681..53b513d 100644
--- a/build.sh
+++ b/build.sh
@@ -1,5 +1,7 @@
#!/usr/bin/env bash
+set -euo pipefail
+
# CloudControl build script
# Usage:
#
@@ -27,8 +29,8 @@ for FLAVOUR in ${FLAVOURS}
do
cat build/Dockerfile.prefix > Dockerfile
cat "flavour/${FLAVOUR}/Dockerfile.flavour" >> Dockerfile
- cat build/Dockerfile.suffix.mo | docker run --rm -i -e FLAVOUR=${FLAVOUR} -e BUILD_DATE=$(date -Iseconds) metal3d/mo >> Dockerfile
- docker build --pull . -t "ghcr.io/dodevops/cloudcontrol-${FLAVOUR}:${TAG}"
+ cat build/Dockerfile.suffix.mo | docker run --rm -i -e FLAVOUR=${FLAVOUR} -e BUILD_DATE=$(date -Iseconds) ghcr.io/tests-always-included/mo:3.0.5 >> Dockerfile
+ docker build --pull . --no-cache -t "ghcr.io/dodevops/cloudcontrol-${FLAVOUR}:${TAG}"
done
if [ -e Dockerfile.sav ] ; then
diff --git a/cmd/doc/mkdoc.go b/cmd/doc/mkdoc.go
index cdc8d98..86cf6c2 100644
--- a/cmd/doc/mkdoc.go
+++ b/cmd/doc/mkdoc.go
@@ -23,6 +23,9 @@ func fetchDocData(basePath string, filenamePattern string) (map[string]internal.
return docDatas, err
} else {
for _, dir := range subDirs {
+ if filepath.Base(dir) == ".template" {
+ continue
+ }
if _, err := os.Stat(filepath.Join(dir, filenamePattern)); err == nil {
if yamlFile, err := os.ReadFile(filepath.Join(dir, filenamePattern)); err != nil {
return docDatas, err
diff --git a/cmd/tests/test-features.go b/cmd/tests/test-features.go
index 7250e7a..1f9f93a 100644
--- a/cmd/tests/test-features.go
+++ b/cmd/tests/test-features.go
@@ -77,7 +77,7 @@ func getFeatures(flavour string) []lib.Feature {
funk.FilterString(
featuresGlob,
func(value string) bool {
- if correctPathRegexp.Match([]byte(filepath.Dir(value))) {
+ if correctPathRegexp.Match([]byte(filepath.Base(value))) {
if yamlFile, err := os.ReadFile(filepath.Join(value, "feature.yaml")); err != nil {
return false
} else {
@@ -225,8 +225,10 @@ func main() {
os.Exit(1)
}
+ features := getFeatures(*flavour)
+
for _, includeFeature := range *includeFeatures {
- if !funk.Contains(getFeatures(*flavour), func(feature lib.Feature) bool {
+ if !funk.Contains(features, func(feature lib.Feature) bool {
return feature.Name == includeFeature
}) {
logrus.Errorf("%s is not a known feature", includeFeature)
@@ -236,7 +238,7 @@ func main() {
}
for _, excludeFeature := range *excludeFeatures {
- if !funk.Contains(getFeatures(*flavour), func(feature lib.Feature) bool {
+ if !funk.Contains(features, func(feature lib.Feature) bool {
return feature.Name == excludeFeature
}) {
logrus.Errorf("%s is not a known feature", excludeFeature)
diff --git a/feature/.template/feature.yaml b/feature/.template/feature.yaml
new file mode 100644
index 0000000..cfc1f24
--- /dev/null
+++ b/feature/.template/feature.yaml
@@ -0,0 +1,18 @@
+# This is the metadata manifest for a feature. Configure it for the feature to be included in CloudControl
+# In your new feature, please remove all descriptive comments from the template.
+
+# Please use an emoji character as a meaningful logo for your feature
+icon: "🫥"
+
+# Set the title for this feature
+title: "Template"
+
+# Set the description of the feature in Markdown. Be sure to include links to websites of the software the feature
+# installs or configures
+description: "Does some awesome stuff"
+
+# Use the configuration key to list additional configuration options that you're providing in your feature
+configuration:
+ #- |
+ # Environment EXAMPLE_ENVIRONMENT: This environment variable is used in the feature to do xyz (optional)
+ # Defaults to `abc`
diff --git a/feature/.template/goss/.envrc b/feature/.template/goss/.envrc
new file mode 100644
index 0000000..ed8b5fe
--- /dev/null
+++ b/feature/.template/goss/.envrc
@@ -0,0 +1,2 @@
+# Setup required environment for test
+export EXAMPLE_ENVIRONMENT=something
diff --git a/feature/.template/goss/goss.yaml b/feature/.template/goss/goss.yaml
new file mode 100644
index 0000000..b803253
--- /dev/null
+++ b/feature/.template/goss/goss.yaml
@@ -0,0 +1,5 @@
+command:
+ test:
+ # Run ls / after installing the feature and if that returns successfully, the test is successful
+ exec: "ls /"
+ exit-status: 0
diff --git a/feature/.template/install.sh b/feature/.template/install.sh
new file mode 100644
index 0000000..eccc716
--- /dev/null
+++ b/feature/.template/install.sh
@@ -0,0 +1,27 @@
+# The installer script for your feature, which is called in the init phase of CloudControl
+
+# Include the feature installer utils which include helpers for the installation of your feature
+# See /assets/feature-installer-utils.sh
+
+. /feature-installer-utils.sh
+
+# Prepare a workspace for installing the feature
+prepare
+
+if [ "X${FLAVOUR}X" == "XsimpleX" ]
+then
+ # Do specific things for the simple flavour
+ download "https://example.com/my-tool-for-simple.tar.gz" tool.tar.gz
+elif [[ "X${FLAVOR}X" =~ X(azure|gcloud)X ]]
+then
+ # Do specific things for other flavours
+ download "https://example.com/my-tool-for-others.tar.gz" tool.tar.gz
+fi
+
+# Do other things
+execHandle "Extracting tool" tar xzf tool.tar.gz
+execHandle "Making tool executable" chmod +x tool
+execHandle "Copying tool to ${BINPATH}" cp tool "${BINPATH}"
+
+# Cleanup what we did
+cleanup
\ No newline at end of file
diff --git a/feature/cmctl/feature.yaml b/feature/cmctl/feature.yaml
new file mode 100644
index 0000000..765eca6
--- /dev/null
+++ b/feature/cmctl/feature.yaml
@@ -0,0 +1,7 @@
+icon: "📨"
+title: "cmctl"
+description: "Installs the cert-manager Command Line Tool"
+configuration:
+ - |
+ Environment CMCTL_VERSION: Version of cmctl to install (optional)
+ Defaults to `latest`
diff --git a/feature/cmctl/goss/goss.yaml b/feature/cmctl/goss/goss.yaml
new file mode 100644
index 0000000..8a02ad4
--- /dev/null
+++ b/feature/cmctl/goss/goss.yaml
@@ -0,0 +1,6 @@
+command:
+ cmctl:
+ exec: "/home/cloudcontrol/bin/cmctl version --client"
+ stdout:
+ - "Client Version"
+ exit-status: 0
diff --git a/feature/cmctl/install.sh b/feature/cmctl/install.sh
new file mode 100644
index 0000000..6d6e489
--- /dev/null
+++ b/feature/cmctl/install.sh
@@ -0,0 +1,9 @@
+. /feature-installer-utils.sh
+
+# Prepare a workspace for installing the feature
+prepare
+
+download "https://github.com/cert-manager/cmctl/releases/${CMCTL_VERSION:-latest}/download/cmctl_linux_$(getPlatform)" cmctl
+execHandle "Making cmctl executable" chmod +x cmctl
+execHandle "Copying tool to ${BINPATH}" cp cmctl "${BINPATH}"
+cleanup
\ No newline at end of file
diff --git a/feature/sops/feature.yaml b/feature/sops/feature.yaml
new file mode 100644
index 0000000..5fe5405
--- /dev/null
+++ b/feature/sops/feature.yaml
@@ -0,0 +1,6 @@
+icon: "🔑"
+title: "sops"
+description: "Installs [sops](https://github.com/getsops/sops)"
+configuration:
+ - "Environment SOPS_VERSION (required): Valid sops version (e.g. 3.8.1)"
+ - "Environment specific for the key you use, see [sops documentation](https://github.com/getsops/sops?tab=readme-ov-file#22encrypting-using-age)"
diff --git a/feature/sops/goss/.env b/feature/sops/goss/.env
new file mode 100644
index 0000000..4a9e6df
--- /dev/null
+++ b/feature/sops/goss/.env
@@ -0,0 +1 @@
+SOPS_VERSION=3.8.1
\ No newline at end of file
diff --git a/feature/sops/goss/goss.yaml b/feature/sops/goss/goss.yaml
new file mode 100644
index 0000000..6ecf167
--- /dev/null
+++ b/feature/sops/goss/goss.yaml
@@ -0,0 +1,6 @@
+command:
+ sops:
+ exec: "/home/cloudcontrol/bin/sops --version"
+ exit-status: 0
+ stdout:
+ - "sops"
diff --git a/feature/sops/install.sh b/feature/sops/install.sh
new file mode 100644
index 0000000..471cc21
--- /dev/null
+++ b/feature/sops/install.sh
@@ -0,0 +1,23 @@
+. /feature-installer-utils.sh
+
+if [ -z "${SOPS_VERSION}" ]
+then
+ echo "The sops feature requires a version set using SOPS_VERSION. See https://github.com/getsops/sops/releases/ for valid versions"
+ exit 1
+fi
+
+SOPS_VERSION=$(checkAndCleanVersion "${SOPS_VERSION}")
+
+TEMPDIR=$(mktemp -d)
+cd "${TEMPDIR}" || exit
+
+execHandle "Downloading sops" curl -f -s -L "https://github.com/getsops/sops/releases/download/v${SOPS_VERSION}/sops-v${SOPS_VERSION}.linux.$(getPlatform)" --output sops
+execHandle "Installing sops" mv sops /home/cloudcontrol/bin
+execHandle "Making sops executable" chmod +x /home/cloudcontrol/bin/sops
+
+cd - &>/dev/null || exit
+rm -rf "${TEMPDIR}"
+
+
+
+