CI/CD GitHub Actions Automation, Helper Scripts, Container Creation Updates, Bug Fixes, Etc.

.gitmodules

@@ -0,0 +1,9 @@
+[submodule "LDAP/pown"]
+	path = LDAP/pown
+	url = https://github.com/anishapant21/pown.sh.git
+[submodule "LDAP/LDAPServer"]
+	path = LDAP/LDAPServer
+	url = https://github.com/mieweb/LDAPServer.git
+[submodule "ci-cd automation/Proxmox-Launchpad"]
+	path = ci-cd automation/Proxmox-Launchpad
+    url = https://github.com/maxklema/proxmox-launchpad.git

LDAP/README.md

@@ -0,0 +1 @@
+## LDAP

README.md

@@ -1,65 +1,132 @@
 # opensource-mieweb
 
-Configuration storage for the [opensource.mieweb.com](https://opensource.mieweb.com) Proxmox project.
+Configuration storage for the [opensource.mieweb.org](https://opensource.mieweb.org:8006) Proxmox project.
 
-This repository contains configuration files and scripts for managing a Proxmox-based container hosting environment, including automated DNS, NGINX reverse proxy, and dynamic port mapping.
+This repository contains configuration files and scripts for managing a Proxmox-based container hosting environment, including automated DNS, NGINX reverse proxy, dynamic port mapping, and the Proxmox LaunchPad GitHub Action for automated container deployment.
 
+## Cluster Graph
+
+```mermaid
+
+graph TD
+    %% Repository Structure
+    REPO[opensource-mieweb Repository]
+
+    %% All Main Folders
+    REPO --> CICD[ci-cd automation]
+    REPO --> CC[container creation]
+    REPO --> DNS[dnsmasq service]
+    REPO --> GW[gateway]
+    REPO --> LDAP[LDAP]
+    REPO --> NGINX[nginx reverse proxy]
+    REPO --> PL[proxmox-launchpad]
+
+    %% Core Workflow Connections
+    CC --> |creates| CONTAINER[LXC Container]
+    CONTAINER --> |Updates Container Map| NGINX
+    CONTAINER --> |updates| DNS
+    CONTAINER --> | Updates IP Tables| GW
+    CONTAINER --> |authenticates via| LDAP
+
+    %% CI/CD Operations
+    CICD --> |manages| CONTAINER
+    PL --> |automates| CC
+    PL --> |uses| CICD
+
+    %% User Access Flow
+    USER[User Access] --> DNS
+    DNS --> NGINX
+    NGINX --> CONTAINER
+
+    %% Styling
+    classDef folder fill:#e3f2fd,stroke:#1976d2,stroke-width:2px
+    classDef system fill:#f1f8e9,stroke:#689f38,stroke-width:2px
+    classDef user fill:#fff3e0,stroke:#f57c00,stroke-width:2px
+
+    class CICD,CC,DNS,GW,LDAP,NGINX,PL folder
+    class CONTAINER system
+    class USER user
 ```
-│   README.md
-│   
-├───intern-dnsmasq
-│       dnsmasq.conf
-│       
-├───intern-nginx
-│       nginx.conf
-│       port_map.js
-│       reverse_proxy.conf
-│
-└───intern-phxdc-pve1
-        register-container.sh
-        register_proxy_hook.sh
-```
-## Repository Structure
 
-- [`intern-dnsmasq/dnsmasq.conf`](intern-dnsmasq/dnsmasq.conf):  
-  Dnsmasq configuration for DHCP and DNS, including wildcard routing for the reverse proxy.
+### Core Infrastructure
+
+- [`dnsmasq service/`](dnsmasq%20service/):  
+  Contains Dnsmasq configuration for DHCP and DNS services, including wildcard routing for the reverse proxy and container network management.
+
+- [`nginx reverse proxy/`](nginx%20reverse%20proxy/):  
+  Houses NGINX configuration files for the reverse proxy setup, including JavaScript modules for dynamic backend resolution and SSL certificate management.
 
-- [`intern-nginx/nginx.conf`](intern-nginx/nginx.conf):  
-  Main NGINX configuration, loading the JavaScript module for dynamic backend resolution.
+- [`gateway/`](gateway/):  
+  Gateway configuration and management scripts for network routing and access control between the internal container network and external traffic. Also contains daily clean up scripts for the cluster.
 
-- [`intern-nginx/reverse_proxy.conf`](intern-nginx/reverse_proxy.conf):  
-  NGINX reverse proxy config using dynamic JavaScript lookups for backend containers.
+### Container Management
 
-- [`intern-nginx/port_map.js`](intern-nginx/port_map.js):  
-  JavaScript module for NGINX to map subdomains to backend container IPs and ports using a JSON file.
+- [`container creation/`](container%20creation/):  
+  Contains comprehensive scripts for LXC container lifecycle management, including creation, LDAP configuration, service deployment, and registration with the proxy infrastructure.
 
-- [`intern-phxdc-pve1/register-container.sh`](intern-phxdc-pve1/register-container.sh):  
-  Proxmox hook script to register new containers with the NGINX proxy and assign HTTP/SSH ports.
+- [`ci-cd automation/`](ci-cd%20automation/):  
+  Automation scripts for continuous integration and deployment workflows, including container existence checks, updates, and cleanup operations with helper utilities.
 
-- [`intern-phxdc-pve1/register_proxy_hook.sh`](intern-phxdc-pve1/register_proxy_hook.sh):  
-  Proxmox event hook to trigger container registration on startup.
+### Authentication & Directory Services
+
+- [`LDAP/`](LDAP/):  
+  Contains LDAP authentication infrastructure including a custom Node.js LDAP server that bridges database user management with LDAP protocols, and automated LDAP client configuration tools for seamless container authentication integration. LDAP Server configured to reference the [Proxmox VE Users @pve realm](https://pve.proxmox.com/wiki/User_Management) with optional [Push Notification 2FA](https://github.com/mieweb/mieweb_auth_app)
+
+### GitHub Action Integration
+
+- [`proxmox-launchpad/`](proxmox-launchpad/):  
+  The Proxmox LaunchPad GitHub Action for automated container deployment directly from GitHub repositories, supporting both single and multi-component applications.
 
 ## How It Works
 
-- **DNS**: All `*.opensource.mieweb.com` requests are routed to the NGINX proxy via Dnsmasq.
-- **Reverse Proxy**: NGINX uses a JavaScript module to dynamically resolve the backend IP and port for each subdomain, based on `/etc/nginx/port_map.json`.
-- **Container Registration**: When a new container starts, Proxmox runs a hook script that:
-  - Waits for the container to get a DHCP lease.
-  - Assigns available HTTP and SSH ports.
-  - Updates the NGINX port map and reloads NGINX.
-  - Sets up port forwarding for SSH access.
+- **DNS**: All `*.opensource.mieweb.com` requests are routed to the NGINX proxy via Dnsmasq, providing automatic subdomain resolution for containers.
+- **Reverse Proxy**: NGINX uses JavaScript modules to dynamically resolve backend IP addresses and ports for each subdomain, based on the container registry in `/etc/nginx/port_map.json`.
+- **Container Lifecycle**: When containers start, Proxmox hooks automatically:
+  - Wait for DHCP lease assignment
+  - Allocate available HTTP and SSH ports
+  - Update the NGINX port mapping and reload configuration
+  - Configure iptables rules for SSH port forwarding
+- **GitHub Integration**: The Proxmox LaunchPad action automates the entire process from repository push to live deployment, including dependency installation, service configuration, and application startup.
+- **CI/CD Pipeline**: Automated scripts used by [Proxmox LaunchPad](#proxmox-launchpad) to handle container updates, existence checks, and cleanup operations to maintain a clean and efficient hosting environment.
+- **LDAP Server**: All LXC Container Authentication is handled by a centralized LDAP server housed in the cluster. Each Container is configured with SSSD, which communicates with the LDAP server to verify/authenitcate user credentials. This approach is more secure than housing credentials locally.
+
 
-## Usage
+## Proxmox LaunchPad
+
+The Proxmox LaunchPad is a powerful GitHub Action that automatically creates, manages, and deploys LXC containers on the Proxmox cluster based on your repository's branch activity. It supports:
+
+- **Automatic Container Creation**: Creates new containers when branches are created or pushed to
+- **Multi-Component Deployments**: Supports applications with multiple services (e.g., frontend + backend)
+- **Service Integration**: Automatically installs and configures services like MongoDB, Docker, Redis, PostgreSQL, and more
+- **Branch-Based Environments**: Each branch gets its own isolated container environment
+- **Automatic Cleanup**: Deletes containers when branches are deleted (e.g., after PR merges)
+
+The action integrates with the existing infrastructure to provide automatic DNS registration, reverse proxy configuration, and port mapping for seamless access to deployed applications.
+
+## Opensource Cluster Usage
+
+### For Infrastructure Management
 
 1. **Clone this repository** to your Proxmox host or configuration management system.
 2. **Deploy the configuration files** to their respective locations on your infrastructure.
 3. **Ensure dependencies**:
-   - Proxmox VE with container support.
-   - NGINX with the `ngx_http_js_module`.
-   - Dnsmasq.
-   - `jq` for JSON manipulation.
-4. **Register new containers** using the provided hook scripts for automatic proxy and DNS integration.
+   - Proxmox VE with LXC container support
+   - NGINX with the `ngx_http_js_module`
+   - Dnsmasq for DNS and DHCP services
+4. **Set up LDAP authentication** using the provided LDAP server and client configuration tools.
+5. **Configure container templates** and network settings according to your environment.
+6. **Register new containers** using the provided hook scripts for automatic proxy and DNS integration.
+
+### For GitHub Action Deployment
+
+1. **Add the Proxmox LaunchPad action** to your repository's workflow file.
+2. **Configure repository secrets** for Proxmox credentials and optionally a GitHub PAT.
+3. **Set up trigger events** for push, create, and delete operations in your workflow.
+4. **Configure deployment properties** in your workflow file for automatic application deployment.
+5. **Push to your repository** and watch as containers are automatically created and your application is deployed.
+
+See the [`proxmox-launchpad/README.md`](proxmox-launchpad/README.md) for detailed setup instructions and configuration options.
 
 ---
 
-*Current SME: Carter Myers and other contributors to opensource.mieweb.com *
+Contributors: Carter Myers, Maxwell Klema, and Anisha Pant

ci-cd automation/README.md

@@ -0,0 +1 @@
+# CI/CD Automation

ci-cd automation/check-container-exists.sh

@@ -0,0 +1,46 @@
+#!/bin/bash
+# Script to check if a container exists, and if so, whether it needs to be updated or cloned.
+# Last Modified by Maxwell Klema on July 13th, 2025
+# -----------------------------------------------------
+
+outputError() {
+	echo -e "${BOLD}━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━${RESET}"
+	echo -e "${BOLD}${MAGENTA}❌ Script Failed. Exiting... ${RESET}"
+	echo -e "$2"
+	echo -e "${BOLD}━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━${RESET}"
+  exit $1
+}
+
+RESET="\033[0m"
+BOLD="\033[1m"
+MAGENTA='\033[35m'
+
+echo -e "${BOLD}━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━${RESET}"
+echo -e "${BOLD}${MAGENTA}🔎 Check Container Exists ${RESET}"
+echo -e "${BOLD}━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━${RESET}"
+
+TYPE_RUNNER="true"
+source /var/lib/vz/snippets/helper-scripts/PVE_user_authentication.sh
+source /var/lib/vz/snippets/helper-scripts/verify_container_ownership.sh
+
+STATUS=$?
+if [ "$STATUS" != 0 ]; then
+    exit 1;
+fi
+
+REPO_BASE_NAME=$(basename -s .git "$PROJECT_REPOSITORY")
+
+# Check if repository folder is present.
+if [ "$PVE1" == "true" ]; then
+    if pct exec $CONTAINER_ID -- test -f /root/container-updates.log; then
+        exit 2; # Update Repository
+    else
+        exit 0; # Clone Repository
+    fi
+else
+    if ssh 10.15.0.5 "pct exec $CONTAINER_ID -- test -f /root/container-updates.log"; then
+        exit 2; # Update Repository
+    else
+        exit 0; # Clone Repository
+    fi
+fi

ci-cd automation/delete-container.sh

@@ -0,0 +1,29 @@
+#!/bin/bash
+# Script to delete a container permanently
+# Last Modified by Maxwell Klema on July 13th, 2025
+# -----------------------------------------------------
+
+RESET="\033[0m"
+BOLD="\033[1m"
+MAGENTA='\033[35m'
+
+echo -e "${BOLD}━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━${RESET}"
+echo -e "${BOLD}${MAGENTA}🗑️  Delete Container ${RESET}"
+echo -e "${BOLD}━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━${RESET}"
+
+CMD=(
+bash /var/lib/vz/snippets/helper-scripts/delete-runner.sh
+"$PROJECT_REPOSITORY"
+"$GITHUB_PAT"
+"$PROXMOX_USERNAME"
+"$PROXMOX_PASSWORD"
+"$CONTAINER_NAME"
+)
+
+# Safely quote each argument for the shell
+QUOTED_CMD=$(printf ' %q' "${CMD[@]}")
+
+tmux new-session -d -s delete-runner "$QUOTED_CMD"
+
+echo "✅ Container with name \"$CONTAINER_NAME\" will be permanently deleted."
+exit 0 # Container Deleted Successfully

ci-cd automation/helper-scripts/PVE_user_authentication.sh

@@ -0,0 +1,23 @@
+#!/bin/bash
+# Script that checks if a user is authenticated in Proxmox PVE Realm @ opensource.mieweb.org 
+# Last Modified by Maxwell Klema on July 13th, 2025
+# -----------------------------------------------------
+
+# Authenticate User (Only Valid Users can Create Containers)
+
+if [ -z "$PROXMOX_USERNAME" ]; then
+	read -p  "Enter Proxmox Username →  " PROXMOX_USERNAME
+fi
+
+if [ -z "$PROXMOX_PASSWORD" ]; then
+	read -sp "Enter Proxmox Password →  " PROXMOX_PASSWORD
+	echo ""
+fi
+
+USER_AUTHENTICATED=$(ssh root@10.15.234.122 "node /root/bin/js/runner.js authenticateUser \"$PROXMOX_USERNAME\" \"$PROXMOX_PASSWORD\"")
+
+if [ $USER_AUTHENTICATED == 'false' ]; then
+	outputError 1 "Your Proxmox account, $PROXMOX_USERNAME@pve, was not authenticated. Retry with valid credentials."
+fi
+
+echo "🎉 Your proxmox account, $PROXMOX_USERNAME@pve, has been authenticated"

ci-cd automation/helper-scripts/create-template.sh

@@ -0,0 +1,56 @@
+#!/bin/bash
+# Creates a template of a LXC container
+# Last modified by Maxwell Klema on July 23rd, 2025.
+# --------------------------------------------------
+
+if [ "${DEPLOY_ON_START^^}" != "Y" ] || [ "${GH_ACTION^^}" != "Y" ]; then
+    return 0
+fi
+
+DEFAULT_BRANCH=$(curl -s https://api.github.com/repos/$REPO_BASE_NAME_WITH_OWNER/$REPO_BASE_NAME | jq -r '.default_branch')
+
+if [ "$DEFAULT_BRANCH" != "$PROJECT_BRANCH" ]; then
+    return 0
+fi
+
+echo "📝 Creating Container Template..."
+
+# Check if template already exists, and if it does, destroy it =====
+
+TEMPLATE_NAME="template-$REPO_BASE_NAME-$REPO_BASE_NAME_WITH_OWNER"
+TEMPLATE_CONTAINER_ID=$( { pct list; ssh root@10.15.0.5 'pct list'; } | awk -v name="$TEMPLATE_NAME" '$3 == name {print $1}')
+
+if [ ! -z "$TEMPLATE_CONTAINER_ID" ]; then
+    pct destroy $TEMPLATE_CONTAINER_ID | true
+fi
+
+# Clone LXC container and convert it into a template =====
+
+NEXT_ID=$(pvesh get /cluster/nextid)
+
+if (( $CONTAINER_ID % 2 == 101 )); then
+    ssh root@10.15.0.5 "
+    pct clone $CONTAINER_ID $NEXT_ID \
+        --hostname "$TEMPLATE_NAME" \
+        --full true
+    pct migrate $NEXT_ID intern-phxdc-pve1 --target-storage containers-pve1
+    " > /dev/null 2>&1
+else
+    pct clone $CONTAINER_ID $NEXT_ID \
+        --hostname "$TEMPLATE_NAME" \
+        --full true
+fi
+
+# AUTH_TOKEN_RESPONSE=$(curl --location --request POST https://api.github.com/repos/$REPO_BASE_NAME_WITH_OWNER/$REPO_BASE_NAME/actions/runners/registration-token --header "Authorization: token $GITHUB_PAT")
+# TOKEN=$(echo "$AUTH_TOKEN_RESPONSE" | jq -r '.token')
+
+# Remove rsa keys ====
+pct start $NEXT_ID
+pct enter $NEXT_ID <<EOF
+rm -rf /root/.ssh/id_rsa.pub && rm -rf /root/.ssh/id_rsa && \
+rm -rf /root/container-updates.log
+EOF
+
+pct stop $NEXT_ID
+pct set $NEXT_ID --tags "$PROXMOX_USERNAME"
+pct template $NEXT_ID