AiMLops Project 3.2 Automation Script

MLOps Platform Setup Scripts: Technical Documentation

Overview

This document describes a set of shell scripts designed to automate the setup of a local development environment for the MLOps platform, primarily focusing on creating a Kubernetes cluster using Kind, configuring networking, and setting up automated SSL/TLS certificate management using cert-manager.

These scripts handle user configuration, hostname detection, cluster creation, and SSL provider setup (Let's Encrypt or ZeroSSL).

Prerequisites

Before running these scripts, ensure the following tools are installed on your system:

• Docker: Required by Kind to run Kubernetes nodes as containers.
• Kind (Kubernetes in Docker): The tool used to create the local Kubernetes cluster.
• kubectl: The Kubernetes command-line tool for interacting with the cluster.
• curl: Used by Finding_Hostname.sh to detect the public IP address.
• dig: Used by Finding_Hostname.sh for reverse DNS lookups. (Installed automatically by the script if missing and using apt or yum).

Configuration (config.env)

Several scripts read from and write to configuration files, primarily config.env located in the root directory relative to the scripts, and component-specific config.env files within the deployment directories. Key variables include:

• INSTALL_TYPE: Set to local or cloud. Determines if cloud-specific SSL configuration is needed.
• SSL_PROVIDER: Set to letsencrypt or zerossl. Specifies the certificate authority.
• EMAIL: The email address used for SSL certificate registration and notifications.
• ZEROSSL_EAB_HMAC_KEY: ZeroSSL External Account Binding (EAB) HMAC Key (if using ZeroSSL).
• ZEROSSL_ACCESS_KEY_ID: ZeroSSL EAB Key ID (if using ZeroSSL).
• DOMAIN: The detected public hostname or IP address, used for accessing services and configuring SSL certificates.
• HOST_IP: (Environment Variable for create_cluster.sh) The IP address the Kind API server should bind to.
• CLUSTER_NAME: (Environment Variable for create_cluster.sh) The name for the Kind cluster.
• INSTALL_LOCAL_REGISTRY: (Environment Variable for create_cluster.sh, boolean true/false) Whether to configure the Kind cluster to use a local registry mirror.

Script Breakdown

1. SSL_Details.sh

• Purpose: Gathers essential configuration details from the user regarding the installation environment and SSL preferences. Orchestrates configuration file updates and triggers hostname detection.
• Functionality:
  • Prompts the user for INSTALL_TYPE (local or cloud).
  • If local, updates config.env and exits.
  • If cloud:
    • Prompts for SSL_PROVIDER (Let's Encrypt [default] or ZeroSSL).
    • If ZeroSSL, prompts for EAB credentials (ZEROSSL_EAB_HMAC_KEY, ZEROSSL_ACCESS_KEY_ID).
    • Prompts for a valid EMAIL address.
    • Uses the update_config_var function to create/update variables in the main config.env and component-specific config.env files (Cert-manager, MLflow, Kubeflow Pipelines, Grafana, Prometheus).
    • Executes Finding_Hostname.sh to detect the domain/IP and propagate it.
• Key Function: update_config_var: Safely adds or updates key-value pairs in specified configuration files, creating directories and files if they don't exist.

2. Finding_Hostname.sh

• Purpose: Automatically detects the public IP address and corresponding hostname (if available) of the machine where the script is run and updates configuration files.
• Functionality:
  • Ensures curl and dig commands are available, installing them via apt or yum if necessary (ensure_tool_installed function).
  • Uses curl -s ifconfig.me to retrieve the public IP address.
  • Performs a reverse DNS lookup (dig +short -x <IP_ADDRESS>) to find the hostname.
  • If no hostname is found via reverse DNS, it defaults to using the detected IP address as the DOMAIN.
  • Reads SSL_PROVIDER from config.env to determine the correct path for the cert-manager overlay configuration.
  • Uses the update_config_var function (defined within this script) to update the DOMAIN variable in various component configuration files.

4. SSL_Creation.sh

• Purpose: Installs and configures cert-manager within the created Kubernetes cluster to handle automated SSL certificate provisioning based on the choices made during SSL_Details.sh.
• Functionality:
  • Sources configuration from .platform/.config (expected to be a copy of config.env).
  • Validates that INSTALL_TYPE (and SSL_PROVIDER if cloud) are set.
  • Performs a cleanup of any existing cert-manager resources (Deployments, Services, Secrets, Namespace, CRDs, Webhooks) using kubectl delete.
  • Installs cert-manager v1.12.0 CRDs.
  • Applies cert-manager base manifests using kubectl apply -k.
  • Temporarily deletes the cert-manager validating and mutating webhooks to allow ClusterIssuer resources to be created before the webhook service is fully ready.
  • Waits for the cert-manager-webhook deployment to become available.
  • If INSTALL_TYPE is cloud:
    • Applies the appropriate ClusterIssuer and related resources based on the selected SSL_PROVIDER (Let's Encrypt or ZeroSSL) using kubectl apply -k on the corresponding overlay directory (deployment/kubeflow/manifests/common/cert-manager/cert-manager/overlay/$SSL_PROVIDER). It includes logic to apply individual files first and retry the kustomization apply if needed.
    • Re-applies the base manifests to ensure webhooks are correctly configured.
    • Waits and re-applies the overlay manifests again to ensure resources are reconciled with webhooks fully active.
    • Prints the status of ClusterIssuers and Certificates.
  • If INSTALL_TYPE is local, it skips the SSL provider-specific configuration steps.

Workflow / Execution Order

A typical execution flow would involve running SSL_Details.sh first.

1. SSL_Details.sh: Gathers user input, updates config files. If cloud is selected, it calls:
2. Finding_Hostname.sh: Detects public address, updates config files with the DOMAIN.
3. create_cluster.sh: (Run separately, likely requires HOST_IP and CLUSTER_NAME env vars) Creates the Kind cluster.
4. SSL_Creation.sh: (Run after cluster creation) Installs cert-manager and configures the SSL issuer based on previously saved settings.

The result is a Kind cluster ready with automated HTTPS certificate management for deployed services accessible via the determined DOMAIN.