Skip to content

Deployment

Jump to bottom
TheMinecraftGuyGuru edited this page Jun 8, 2026 · 4 revisions

Deployment

MuxCore scales from a single machine to a globally distributed cluster without changing the architecture. Start simple — one binary, sidecar modules. Grow to multiple machines when you need to.

Phase 1: Single Machine

The simplest setup — one machine, one core binary, modules as separate processes.

docker compose up

What You Get

┌──────────────────────────┐
│      Docker Host          │
│                           │
│  ┌──────────────────────┐ │
│  │  MuxCore (the loom)  │ │
│  │                       │ │
│  │  HTTP API  → :8080   │ │
│  │  gRPC Mesh → :9090   │ │
│  │  Admin UI             │ │
│  └──────────────────────┘ │
│                           │
│  ┌──────────────────────┐ │
│  │  Sidecar Modules      │ │
│  │  - downloader         │ │
│  │  - media-manager      │ │
│  │  - transcoder         │ │
│  │  - notification       │ │
│  └──────────────────────┘ │
└──────────────────────────┘
  • Core binary starts the loom
  • Modules are separate binaries spawned by core (or run independently)
  • In-memory event bus (zero external dependencies)
  • Local filesystem storage
  • No database needed for single-node
  • Just as easy as installing Sonarr

Sidecar Modules

Modules are standalone binaries. Core spawns them as child processes, or you run them separately:

# Start core (fetches and spawns modules from the spool)
muxcored --tag default

# Or — start core bare, connect modules manually
muxcored
./downloader-qbittorrent --muxcore-mesh-addr :9090 --muxcore-module-id downloader-qbittorrent

Configuration

All settings go in muxcore.json or environment variables. See Getting Started for the full config reference.

Phase 2: Multiple Machines

Add worker nodes for distributed transcoding, storage, and failover.

┌─────────────────────┐
│   MuxCore Node 1    │  (primary — API, UI, event bus)
│   - Core services   │
│   - Database module │
│   - Cache module    │
└─────────┬───────────┘
          │
    ┌─────┴─────────────┐
    │                   │
┌───▼─────────┐  ┌──────▼────────┐
│ Worker 1    │  │ Worker 2      │
│ - Transcoder│  │ - Transcoder  │
│ - GPU: RTX  │  │ - GPU: GTX    │
│ - Local SSD  │  │ - Local SSD   │
└─────────────┘  └───────────────┘

New Capabilities

  • Distributed transcoding — tasks go to the least-loaded GPU
  • Worker failover — if worker 1 dies, tasks move to worker 2
  • Module failover — modules reconnect to surviving cores after a node death
  • Shared task queue — backed by cache module (Redis/Valkey)
  • Shared metadata — backed by database module (Postgres/SQLite)
  • Module mobility — run a module on any machine that connects to the mesh

Database and Cache Are Modules

PostgreSQL and Redis are not built into core. They're modules you install:

# docker-compose.yml
services:
  muxcore:
    image: muxcore-media/core:latest
    environment:
      - MUXCORE_DATABASE_DRIVER=postgres
      - MUXCORE_DATABASE_URL=postgres://...
      - MUXCORE_CACHE_DRIVER=redis
      - MUXCORE_CACHE_URL=redis://...

  postgres:
    image: postgres:16

  redis:
    image: redis:7

Pick the database and cache that fit your setup — SQLite for single-node, Postgres for multi-node, Redis or Valkey for caching.

Phase 3: Kubernetes

Full cloud-native orchestration:

┌───────────────────────────────────────┐
│         Kubernetes Cluster             │
│                                       │
│  ┌──────────┐  ┌──────────┐          │
│  │ MuxCore  │  │ MuxCore  │  (HA)    │
│  │ Pod 1    │  │ Pod 2    │          │
│  └──────────┘  └──────────┘          │
│                                       │
│  ┌──────────────────────────────┐     │
│  │     GPU Worker Pool          │     │
│  │  ┌──────┐ ┌──────┐ ┌──────┐ │     │
│  │  │GPU 1 │ │GPU 2 │ │GPU 3 │ │     │
│  │  └──────┘ └──────┘ └──────┘ │     │
│  └──────────────────────────────┘     │
│                                       │
│  ┌──────────────────────────────┐     │
│  │     Storage                   │     │
│  │  ┌─────────┐ ┌──────────┐    │     │
│  │  │ Rook/   │ │ MinIO    │    │     │
│  │  │ Ceph    │ │ (S3)     │    │     │
│  │  └─────────┘ └──────────┘    │     │
│  └──────────────────────────────┘     │
└───────────────────────────────────────┘

New Capabilities

  • Auto-scaling worker pools
  • Multi-node database (HA Postgres)
  • Redis/Valkey cluster
  • Helm chart deployment
  • GitOps (Flux/ArgoCD)
  • Multi-tenant support

Clustering: Auto-Join

When you add a new node, it can automatically join the existing cluster:

On the New Node

Set seed nodes and join token in muxcore.json:

{
  "grpc": {
    "addr": ":9090",
    "seed_nodes": ["primary-node:9090"],
    "join_token": "my-secret-cluster-token"
  }
}

Or with environment variables:

export MUXCORE_GRPC_SEED_NODES="primary-node:9090,backup-node:9090"
export MUXCORE_CLUSTER_JOIN_TOKEN="my-sec...oken"

What Happens

  1. New node starts → contacts seed nodes
  2. Presents join token → seed node validates it
  3. Receives cluster member list + leader ID
  4. Registers with the cluster
  5. Other nodes discover it via cluster events
  6. Heartbeat loop starts — the new node sends heartbeats every 10 seconds carrying its module list
  7. Existing nodes sync — they learn the new node's modules from incoming heartbeats

Heartbeat and Module Tracking

Every core instance sends a heartbeat to every other core every 10 seconds. Heartbeats carry the node's current module list — every module ID registered on that node. This is how the cluster maintains an accurate map of which modules are on which machine.

If a node stops sending heartbeats for 30 seconds, it is evicted from the cluster. Its module list is removed, and mesh routing stops sending calls to it.

Node Failure and Module Recovery

When a node dies:

  1. Surviving nodes detect heartbeat loss → evict after 30s
  2. Modules on the dead node lose their gRPC connection
  3. Modules query DiscoveryService.Members() to find a surviving core
  4. Modules reconnect, re-register, and resume
  5. The new core's next heartbeat advertises the module
  6. Mesh routing updates — other modules never noticed the outage

Module Deployment (Multi-Machine)

Modules run as separate containers or binaries, connecting to whichever core instance is closest:

# docker-compose.yml
services:
  muxcore:
    image: muxcore-media/core:latest

  downloader-qbittorrent:
    image: muxcore-media/downloader-qbittorrent:latest
    command:
      - --muxcore-mesh-addr=muxcore:9090
      - --muxcore-module-id=downloader-qbittorrent

  transcoder-ffmpeg:
    image: muxcore-media/transcoder-ffmpeg:latest
    command:
      - --muxcore-mesh-addr=muxcore:9090
      - --muxcore-module-id=transcoder-ffmpeg
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: 1
              capabilities: [gpu]

Deployment Environments

Environment Phase 1 Phase 2 Phase 3
Home lab Docker Compose Cluster K3s
Seedbox Single binary + modules Cluster K8s
Enterprise — Cluster K8s/Nomad
Media provider — — Full K8s

Observability

Phase What You Get
Phase 1 Structured logging, /health endpoint, trace ID propagation
Phase 2 Prometheus metrics, Grafana dashboards, audit logs
Phase 3 Distributed tracing, centralized logging (Loki/ELK)

Next Steps

Wiki Navigation

Clone this wiki locally