Skip to content

Installation Linux Kubernetes

Jump to bottom
boyism80 edited this page Jul 4, 2026 · 3 revisions

Linux — Kubernetes

Deploy fb to a Kubernetes cluster using Docker images and Kubernetes manifests (kubectl, GitOps, or Helm).

Parent guide: Linux Installation

For a single host without Kubernetes, use Docker Compose.

Overview

Step What happens
Build Docker images from server/*/Dockerfile
Publish Push to a registry the cluster can reach (e.g. GHCR)
Deploy Apply manifests — Deployments, StatefulSets, Services, ConfigMaps, Secrets
Configure Client IP/hostname, ports, world layout

Checked-in manifests: infra/k8s/ (kustomize). See infra/k8s/README.md.

Prerequisites

Cluster

Requirement Notes
Kubernetes 1.28+ k3s, kubeadm, managed EKS/GKE/AKS, …
kubectl Configured kubecontext
NodePort range Client TCP services (gateway, login, game)
Storage Host path /mnt/fb/ on infra nodes (same layout as Pulumi)

Recommended node layout (adjust to your environment):

Requirement Used by
Node label infra=true MySQL, Redis, RabbitMQ StatefulSets (preferred in dev overlay; required in Pulumi)
Node label cpu=epyc (optional) Gateway, Login, Game — preferred scheduling
Host path /mnt/fb/mysql, /mnt/fb/redis Durable MySQL/Redis data

Build machine

Tool Purpose
Docker + buildx Multi-stage image builds
Git Repo + submodules
Registry login Push images the cluster can pull

1. Build Docker images

All services are containerized. Build from the repository root:

BUILD_TYPE=Release   # or Debug

docker buildx build --push -t ghcr.io/<owner>/fb/data:latest \
  -f server/fb/data/Dockerfile .

docker buildx build --push -t ghcr.io/<owner>/fb/build:latest \
  --build-arg BUILD_TYPE=$BUILD_TYPE -f server/fb/Dockerfile .

docker buildx build --push -t ghcr.io/<owner>/fb/gateway:latest \
  --build-arg BUILD_TYPE=$BUILD_TYPE -f server/gateway/Dockerfile .

docker buildx build --push -t ghcr.io/<owner>/fb/login:latest \
  --build-arg BUILD_TYPE=$BUILD_TYPE -f server/login/Dockerfile .

docker buildx build --push -t ghcr.io/<owner>/fb/game:latest \
  --build-arg BUILD_TYPE=$BUILD_TYPE -f server/game/Dockerfile .

# .NET services (same Dockerfile, different SERVICE arg)
for svc in internal write-back log marketplace admin-tool; do
  docker buildx build --push -t ghcr.io/<owner>/fb/${svc}:latest \
    --build-arg BUILD_TYPE=$BUILD_TYPE --build-arg SERVICE=$svc \
    -f server/http/Dockerfile .
done

CI runs the same image build sequence in .github/workflows/build.yml (build-linux and per-service jobs).

Local load (no registry) for a single-node test cluster:

docker buildx build --load -t fb/game:local -f server/game/Dockerfile .

Ensure the cluster can pull your tags (imagePullSecrets if using a private registry).

2. Deploy with kubectl

Dev overlay (single world):

bash tools/compose-build.sh
# kind/minikube: load fb/*:local images into the cluster

cp infra/k8s/config/fb-host.env.example infra/k8s/config/fb-host.env
bash tools/k8s-config.sh
kubectl apply -k infra/k8s/overlays/dev
kubectl -n fb get pods
Path Contents
infra/k8s/base/ Namespace fb
infra/k8s/infra/ mysql, redis, rabbitmq (StatefulSets)
infra/k8s/apps/ gateway, login, game (StatefulSets) + .NET (Deployments)
infra/k8s/config/ Source templates and appsettings.k8s.json
infra/k8s/overlays/dev/ kustomize entry point
infra/k8s/overlays/dev/config/ Generated configs (written by tools/k8s-config.sh)

The dev overlay mirrors Pulumi resource shapes (infra/pulumi/): same StatefulSet/Service naming, split RabbitMQ (rabbitmq-internal, rabbitmq-log), and per-shard MySQL/Redis — with a single world (dev) and one shard each (~19 pods). Production scales shards/replicas via Pulumi develop.json.

Resources created:

Resource Services
Namespace fb
StatefulSet + Service MySQL (unified + per-world global/data/log), Redis (unified + per-world), RabbitMQ (internal + log), Gateway, Login, Game
Deployment Internal, Write-back, Log, Marketplace, Admin Tool
NodePort / ClusterIP Gateway, Login, Game (client TCP); infra NodePorts optional
ConfigMap / Secret mysql-secret, per-shard configs, C++ config.json, appsettings.k8s.json

Boot order

  1. Namespace
  2. MySQL, Redis, RabbitMQ
  3. Internal, Write-back, Admin Tool, Log, Marketplace
  4. Gateway, Login, Game

Wait for infrastructure StatefulSets to become ready before application pods start.

Client-facing address

Gateway, Login, and Game must advertise an IP or hostname that game clients can reach — not in-cluster DNS names like gateway.fb.svc.cluster.local.

tools/k8s-config.sh reads infra/k8s/config/fb-host.env, renders C++ configs from templates, and syncs all configs into overlays/dev/config/ for kustomize. Clients connect via NodePort on the node IP (gateway 30000).

3. Configuration

Setting Notes
Client host tools/k8s-config.sh + infra/k8s/config/fb-host.env
NodePorts infra/k8s/apps/* and infra/k8s/infra/* Services
database.autoMigration Enabled in internal appsettings.k8s.json
Image registry overlays/dev/kustomization.yaml images:
World name dev — resources like login-dev, game-dev-0, mysql-dev-global

Default NodePorts (dev overlay)

Service NodePort
Gateway 30000
Login (login-dev) 30010
Game (game-dev-0) 30030
Internal (HTTP) 30200
Admin tool 30210
Marketplace 30220
MySQL unified 31000
Redis unified 31010
RabbitMQ internal (AMQP) 31020

Add offset ports for additional worlds/shards (same pattern as Pulumi develop.json).

Infra NodePorts should not collide with client services.

4. Verify deployment

kubectl -n fb get pods
kubectl -n fb get svc
nc -zv <client-host> 30000

With auto-migration enabled, Internal applies SQL from infra/db/migrations/ on first start.

Troubleshooting

Symptom Likely cause
Pods Pending (MySQL/Redis) Missing /mnt/fb/ host path, or no schedulable node
ImagePullBackOff Registry auth, wrong tag, or private registry without imagePullSecrets
Client cannot connect Wrong client host in config, firewall, or NodePort not on reachable node IP
NodePort conflict Duplicate port in Service definitions
CrashLoop after deploy Infra not ready; check boot order and connection strings in ConfigMaps

Related files

Path Role
server/*/Dockerfile Image definitions
.github/workflows/build.yml CI image build reference
infra/db/migrations/ SQL applied by Internal on startup
infra/k8s/ Kustomize manifests and configs
tools/k8s-config.sh Render C++ configs and sync overlays/dev/config/

Related pages

Home

Server

Setup

Scripting

Clone this wiki locally