Rancher DevOps Operator

---

A Kubernetes operator for managing Rancher projects and namespaces declaratively using Custom Resource Definitions (CRDs).

Features

• Declarative Project Management: Define Rancher projects using Kubernetes CRDs
• Namespace Management: Automatically create and link namespaces to projects
• Member Management: Configure project members and their roles
• Cluster Name Resolution: Reference clusters by name instead of cluster IDs
• Resource Quotas: Set project-level resource quotas (optional)
• Flexible Authentication: Supports both API tokens and username/password with automatic token management
• Observability: Prometheus metrics and Kubernetes events for full visibility

Prerequisites

• Kubernetes cluster (1.24+)
• Rancher (2.7+)
• Helm 3
• Rancher authentication (either API token or username/password)

Quick Start

See AUTHENTICATION.md for detailed authentication setup guide.

Installation

Using Helm

1. Clone the repository:

git clone https://github.com/Jasonrve/rancher-devops-operator.git
cd rancher-devops-operator

2. Create a values file or update the default values:

Option 1: Using API Token (Recommended for Production)

rancher:
  url: "https://your-rancher-server"
  token: "your-rancher-api-token"
  allowInsecureSsl: false  # Set to true for development with self-signed certs

env:
  cleanupNamespaces: false  # Set to true to delete namespaces when removed from CRD

image:
  repository: ghcr.io/Jasonrve/rancher-devops-operator
  tag: "latest"

Option 2: Using Username/Password (Auto-creates tokens)

rancher:
  url: "https://your-rancher-server"
  username: "admin"
  password: "your-password"
  allowInsecureSsl: false

env:
  cleanupNamespaces: false  # Set to true to delete namespaces when removed from CRD

image:
  repository: ghcr.io/Jasonrve/rancher-devops-operator
  tag: "latest"

Option 3: Using Existing Secret

# Create a secret with token
kubectl create secret generic rancher-creds \
  --namespace rancher-devops-system \
  --from-literal=token=your-token

# Or with username/password
kubectl create secret generic rancher-creds \
  --namespace rancher-devops-system \
  --from-literal=username=admin \
  --from-literal=password=your-password

rancher:
  url: "https://your-rancher-server"
  existingSecret: "rancher-creds"
  allowInsecureSsl: false

image:
  repository: ghcr.io/Jasonrve/rancher-devops-operator
  tag: "latest"

3. Install the operator:

helm install rancher-devops-operator ./helm/rancher-devops-operator \
  --namespace rancher-devops-system \
  --create-namespace \
  --values your-values.yaml

Usage

Creating a Rancher Project

Create a Project custom resource:

apiVersion: rancher.devops.io/v1
kind: Project
metadata:
  name: my-project
spec:
  clusterName: "local"  # Name of your Rancher cluster
  displayName: "My Application Project"
  description: "Project for my application"
  
  # Optional: Management policies (defaults to ["Create", "Delete"])
  # - Create: Allows creating projects/namespaces/members
  # - Delete: Allows deleting projects and removing namespace associations
  # - Observe: Discovers and imports existing namespaces/members from project into CRD
  managementPolicies:
    - Create
    - Delete
  
  namespaces:
    - app-frontend
    - app-backend
    - app-database
  
  members:
    - principalId: "local://user-abc123"
      role: "project-owner"
    - principalId: "local://user-def456"
      role: "project-member"
  
  resourceQuota:
    limitsCpu: "10"
    limitsMemory: "20Gi"
    requestsCpu: "2"
    requestsMemory: "4Gi"

Management Policy Examples

Default Behavior (Create-only)

# Omit managementPolicies or leave empty for default
spec:
  clusterName: "local"
  displayName: "My Project"
  # Creates project/namespaces/members. Deletion and removals require explicit policy.
  # To allow deletions (project deletion, member removals, namespace disassociation/deletion), add:
  # managementPolicies:
  #   - Delete
  # Namespace-specific behavior can be controlled separately:
  # namespaceManagementPolicies:
  #   - Create        # default when omitted
  #   # - Delete      # enables namespace disassociation (and deletion only when CleanupNamespaces=true)

Take Over Existing Project (with Observe)

spec:
  clusterName: "local"
  displayName: "Existing Project"  # Must match existing project name
  managementPolicies:
    - Observe  # Discovers and adds existing namespaces/members to CRD
    - Create
    # - Delete
  namespaces: []  # Will be populated automatically
  members: []     # Will be populated automatically

Monitor-Only Mode (Observe without modifications)

spec:
  clusterName: "local"
  displayName: "Monitored Project"  # Must match existing project name
  managementPolicies:
    - Observe  # Only discovers and tracks, never creates or deletes
  namespaces: []  # Auto-populated and kept in sync via WebSocket events
  members: []     # Auto-populated from Rancher

The Observe policy enables real-time monitoring of Rancher projects:

• Initial Discovery: On first reconciliation, the operator queries Rancher for all namespaces and members in the project and updates the CRD spec
• Real-Time Updates: A WebSocket connection to Rancher monitors namespace creation events. When a new namespace is created in an observed project, the CRD is automatically updated
• No Modifications: With Observe-only mode (no Create or Delete), the operator never modifies Rancher resources—it only tracks changes
• Conflict-Free: Multiple CRDs can observe the same project without conflicts, ideal for read-only monitoring dashboards

Create-Only (No Deletion)

spec:
  clusterName: "local"
  displayName: "Protected Project"
  managementPolicies:
    - Create  # Only creates, never deletes
  namespaces:
    - protected-app

Apply the resource:

kubectl apply -f rancher-project.yaml

Checking Status

kubectl get projects
kubectl describe project my-project

Common Roles

• project-owner: Full access to the project
• project-member: Standard member access
• read-only: Read-only access

Development

Prerequisites

• .NET 9 SDK
• Docker
• Kubernetes cluster (k3d, kind, minikube, or real cluster)

Building

dotnet build

Running Locally

1. Update appsettings.json or set environment variables:

Option 1: Using Token

{
  "Rancher": {
    "Url": "https://rancher.local",
    "Token": "your-token-here",
    "AllowInsecureSsl": true,
    "CleanupNamespaces": false
  }
}

Option 2: Using Username/Password

{
  "Rancher": {
    "Url": "https://rancher.local",
    "Username": "admin",
    "Password": "your-password",
    "AllowInsecureSsl": true,
    "CleanupNamespaces": false
  }
}

Or set environment variables:

# Using token
export Rancher__Url="https://rancher.local"
export Rancher__Token="your-token-here"
export Rancher__AllowInsecureSsl=true
export Rancher__CleanupNamespaces=false

# Or using username/password
export Rancher__Url="https://rancher.local"
export Rancher__Username="admin"
export Rancher__Password="your-password"
export Rancher__AllowInsecureSsl=true
export Rancher__CleanupNamespaces=false

2. Run the operator:

cd rancher-devops-operator
dotnet run

Building Docker Image

docker build -t rancher-devops-operator:latest .

Running Tests

dotnet test

Architecture

The operator consists of:

• V1Project CRD: Defines the desired state of Rancher projects
• ProjectController: Reconciles the CRD with actual Rancher state
• RancherApiService: Handles communication with the Rancher API
• RancherAuthService: Manages authentication (token or username/password)
• RancherWebSocketService: Monitors Rancher namespace creation events in real-time via WebSocket API
• MetricsService: Exposes Prometheus metrics on port 9090
• KubernetesEventService: Creates Kubernetes events for visibility

Real-Time Event Monitoring

The operator maintains a WebSocket connection to Rancher's /v3/subscribe API to monitor namespace creation events. When the Observe management policy is enabled on a Project CRD, the operator automatically updates the CRD's namespace list whenever a matching namespace is created in Rancher. This enables:

• Event-Driven Updates: No polling required—namespace additions are reflected immediately
• Selective Monitoring: Only CRDs with Observe policy receive updates
• Connection Resilience: Automatic reconnection with exponential backoff on failures

Observability

The operator provides comprehensive observability through:

• Prometheus Metrics: Exposed on port 9090 (/metrics)
  • Reconciliation success/failure rates
  • API call duration and error rates
  • Active resource counts
  • Token creation metrics
• Kubernetes Events: Created for all major operations
  • Project creation/deletion
  • Namespace management
  • Member additions
  • Errors and warnings

See OBSERVABILITY.md for detailed metrics documentation and Grafana dashboard examples.

Quick Metrics Access

# Port-forward to metrics endpoint
kubectl port-forward -n rancher-devops-system deployment/rancher-devops-operator 9090:9090

# View metrics
curl http://localhost:9090/metrics

View Events

# For a specific project
kubectl describe project my-project

# All operator events
kubectl get events --field-selector source=rancher-devops-operator

Configuration

Environment Variables

• Rancher__Url: Rancher server URL
• Rancher__Token: Rancher API token
• Rancher__AllowInsecureSsl: Allow insecure SSL (development only)
• Rancher__CleanupNamespaces: Delete namespaces when removed from CRD (default: false)

Namespace Cleanup Behavior

By default (CleanupNamespaces=false), namespaces are never deleted. When a namespace is removed from a Project CRD spec:

• The namespace is disassociated from the project only if namespaceManagementPolicies includes Delete
• The namespace itself remains in the cluster
• Workloads continue running

When CleanupNamespaces=true and namespaceManagementPolicies includes Delete, namespaces are deleted when removed from the CRD spec or when the CRD is deleted. This includes deleting all resources within the namespace.

Notes:

• Use namespaceManagementPolicies to control namespace create vs delete independently of project/member policies.
• namespaceManagementPolicies defaults to Create when omitted. Add Delete to enable disassociation and (optional) deletion.

Status Fields

• status.createdNamespaces: Namespaces created by the operator during reconcile (excludes moved/assigned namespaces).
• status.manuallyRemovedNamespaces: Namespaces that should not be recreated even if present in spec.namespaces.
  • Populated automatically when:
    • A namespace is deleted (watch Deleted event) in the target cluster
    • Poll mode detects a namespace missing compared to the previous snapshot
    • Reconcile finds a desired namespace not present in the Rancher project
  • Can also be managed manually via kubectl (examples below)
• status.createdTimestamp: First successful creation/takeover timestamp for the project.
• status.lastUpdatedTimestamp: Timestamp of the last successful reconcile status update.

Manual Removals: Prevent Recreation

If a namespace is removed directly in Rancher or Kubernetes, the operator marks it as manually removed to avoid recreating it.

Notes:

• When a name exists in status.manuallyRemovedNamespaces, the operator skips creation/assignment for that namespace even if present in spec.namespaces.
• Remove the entry to resume management; on the next reconcile the operator will create/assign according to policies.

Rancher API Token

The operator requires a Rancher API token with the following permissions:

• Manage projects
• Manage namespaces
• Manage project members

To create a token:

1. Log in to Rancher
2. Go to User Settings → API & Keys
3. Create a new API key
4. Save the token securely

Troubleshooting

Check operator logs:

kubectl logs -n rancher-devops-system deployment/rancher-devops-operator

Common Issues

1. Cluster not found: Ensure clusterName matches the exact cluster name in Rancher
2. Authentication errors: Verify your Rancher API token is valid and has correct permissions
3. SSL errors: Set allowInsecureSsl: true for self-signed certificates (development only)

Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

License

[Your License Here]

Support

For issues and questions, please open an issue on GitHub.