RunStack

A lightweight distributed platform written in Go for executing jobs and deploying long-running applications.

RunStack allows you to register nodes, discover machine capabilities, execute remote shell tasks, and manage long-running containerized applications across a cluster. It aims for a clean, deterministic architecture without the overwhelming complexity of full container orchestrators like Kubernetes.

Architecture

Control Plane
    ├── Node Registry
    ├── Job Registry
    ├── Application Registry
    ├── Deployment Registry
    ├── Instance Registry
    ├── Secret Registry
    ├── Job Scheduler
    ├── Instance Scheduler
    ├── Instance Reconciler
    ├── Routing Reconciler
    └── HTTP API

Agents
    ├── Registration & Heartbeats
    ├── Job Polling & Claiming
    ├── Job Execution & Result Reporting
    ├── Instance Polling & Claiming
    └── Container Runtime (Docker/Podman)

Core Capabilities

• Node Discovery: Agents automatically report OS, CPU, RAM, and container runtimes (Docker/Podman). The Control Plane aggressively detects offline agents via heartbeat timeouts.
• Job Execution: A deterministic scheduler pushes one-off PENDING work to available ONLINE nodes. Agents pull assigned jobs, claim them securely, execute them safely via os/exec, and report standard output/errors back.
• Application Deployment (PaaS): Manage desired application state (e.g., number of replicas, image configuration). Features immutable deployments and automatic rollouts.
• Instance Reconciliation: The Control Plane automatically reconciles desired deployments with actual runtime instances, scheduling new instances or tearing down excess ones.
• Container Lifecycle: Agents natively interface with container runtimes (Docker/Podman) to run isolated application instances.
• Failure Recovery: Node-aware failure recovery, stale execution fencing, and robust retry policies ensure workloads recover from agent crashes or timeouts.
• Service Routing & Ingress: Zero-downtime service routing and custom domain ingress support.
• Secrets Management: Application-scoped, in-memory secret registry with safe runtime resolution during instance claims to prevent plaintext leaks.

Domain Models

Jobs (One-off Tasks)

Jobs strictly follow a state machine governed by the Control Plane:

PENDING (Created) → ASSIGNED (Scheduler) → RUNNING (Agent Claim) → SUCCEEDED / FAILED (Agent Report or Recovery)

Applications & Instances (Long-running Services)

Applications use a declarative model mapping desired state to actual instances:

Application (Desired State)
    → Deployment (Immutable Snapshot)
    → Instances (Runtime Units)

Instances follow a similar lifecycle to Jobs:

PENDING → ASSIGNED → RUNNING → STOPPED / FAILED

Running RunStack

RunStack provides a unified CLI and a Makefile for streamlined development.

1. Build the project:

   make dev

2. Start the Control Plane:

   make control-plane

3. Start an Agent:

   make agent

4. Check nodes via CLI:

   ./bin/runstack nodes

   You can also run a health check using ./bin/runstack doctor.

Example Usage

Submitting a Job

curl -X POST http://localhost:8080/api/v1/jobs \
     -H "Content-Type: application/json" \
     -d '{"name":"my-job","command":"echo hello_world", "maxRetries": 3}'

Deploying an Application

curl -X POST http://localhost:8080/api/v1/apps \
     -H "Content-Type: application/json" \
     -d '{
           "name": "web-server",
           "spec": {
             "image": "nginx:latest",
             "replicas": 3,
             "ports": [80]
           }
         }'

Current Limitations (V1)

• In-Memory State: Registries live entirely in memory. If the Control Plane restarts, historical data is lost.
• Command Parsing: Agents run job commands via strings.Fields. Complex shell quotes (echo "hello world") are not yet parsed natively to avoid arbitrary /bin/sh shell injection.
• Single Job Concurrency: An Agent executes exactly one job at a time.
• Single Control Plane: Single-node control plane instance, no multi-node CP high-availability.
• No Persistent Volumes: Strictly a stateless application runtime.

Project Structure & Documentation

For a full breakdown of the architecture, roadmap, and development guidelines, please refer to the files below:

• AGENTS.md - Rules for AI coding assistants.
• MEMORY.md - Technical state and completed milestones.
• SKILLS.md - Subsystem capability mapping.
• docs/ARCHITECTURE.md - Deep architectural diagrams.
• docs/ROADMAP.md - Future features and Milestone planning.
• docs/API.md - HTTP endpoints.
• docs/DEVELOPMENT.md - Local development guidelines.
• docs/TROUBLESHOOTING.md - Common issues.

Development Commands

Run make help to see all available commands. Before committing, always validate your changes:

make check

Milestone 8L: Automatic TLS & HTTPS Ingress (COMPLETED)

• Automatic Let's Encrypt / ACME HTTP-01 certificate provision.
• TLS SNI handshake validation and connection rejection for unknown domains.
• In-memory certificate cache mapped directly to Autocert.
• Automatic HTTP -> HTTPS redirection.
• Control Plane restart volatility inherently acknowledged and guarded with bounded issuance.

Milestone 8M: Declarative Manifests & Idempotent Apply (DESIGN PHASE)

• See docs/MILESTONE_8M_DESIGN.md for the authoritative design.
• Introduces runstack.yaml as the durable source of desired state.
• Retains the V1 "No Database" constraint while solving Control Plane restart volatility.