Installation Linux Kubernetes

Linux — Kubernetes

Deploy fb to a Kubernetes cluster using Docker images and Kubernetes manifests (kubectl, GitOps, or Helm).

For a single host without Kubernetes, use Docker Compose.

Overview

Step	What happens
Build	Docker images from `server/*/Dockerfile`
Publish	Push to a registry the cluster can reach (e.g. GHCR)
Deploy	Apply manifests — Deployments, StatefulSets, Services, ConfigMaps, Secrets
Configure	Client IP/hostname, ports, world layout

Checked-in manifests: infra/k8s/ (kustomize). See infra/k8s/README.md.

Prerequisites

Cluster

Requirement	Notes
Kubernetes 1.28+	k3s, kubeadm, managed EKS/GKE/AKS, …
kubectl	Configured kubecontext
NodePort range	Client TCP services (gateway, login, game)
Storage	Host path `/mnt/fb/` on infra nodes (same layout as Pulumi)

Recommended node layout (adjust to your environment):

Requirement	Used by
Node label `infra=true`	MySQL, Redis, RabbitMQ StatefulSets (preferred in dev overlay; required in Pulumi)
Node label `cpu=epyc` (optional)	Gateway, Login, Game — preferred scheduling
Host path `/mnt/fb/mysql`, `/mnt/fb/redis`	Durable MySQL/Redis data

Build machine

Tool	Purpose
Docker + buildx	Multi-stage image builds
Git	Repo + submodules
Registry login	Push images the cluster can pull

1. Build Docker images

All services are containerized. Build from the repository root:

BUILD_TYPE=Release   # or Debug

docker buildx build --push -t ghcr.io/<owner>/fb/data:latest \
  -f server/fb/data/Dockerfile .

docker buildx build --push -t ghcr.io/<owner>/fb/build:latest \
  --build-arg BUILD_TYPE=$BUILD_TYPE -f server/fb/Dockerfile .

docker buildx build --push -t ghcr.io/<owner>/fb/gateway:latest \
  --build-arg BUILD_TYPE=$BUILD_TYPE -f server/gateway/Dockerfile .

docker buildx build --push -t ghcr.io/<owner>/fb/login:latest \
  --build-arg BUILD_TYPE=$BUILD_TYPE -f server/login/Dockerfile .

docker buildx build --push -t ghcr.io/<owner>/fb/game:latest \
  --build-arg BUILD_TYPE=$BUILD_TYPE -f server/game/Dockerfile .

# .NET services (same Dockerfile, different SERVICE arg)
for svc in internal write-back log marketplace admin-tool; do
  docker buildx build --push -t ghcr.io/<owner>/fb/${svc}:latest \
    --build-arg BUILD_TYPE=$BUILD_TYPE --build-arg SERVICE=$svc \
    -f server/http/Dockerfile .
done

CI runs the same image build sequence in .github/workflows/build.yml (build-linux and per-service jobs).

Local load (no registry) for a single-node test cluster:

docker buildx build --load -t fb/game:local -f server/game/Dockerfile .

Ensure the cluster can pull your tags (imagePullSecrets if using a private registry).

2. Deploy with kubectl

Dev overlay (single world):

bash tools/compose-build.sh
# kind/minikube: load fb/*:local images into the cluster

cp infra/k8s/config/fb-host.env.example infra/k8s/config/fb-host.env
bash tools/k8s-config.sh
kubectl apply -k infra/k8s/overlays/dev
kubectl -n fb get pods

Path	Contents
`infra/k8s/base/`	Namespace `fb`
`infra/k8s/infra/`	mysql, redis, rabbitmq (StatefulSets)
`infra/k8s/apps/`	gateway, login, game (StatefulSets) + .NET (Deployments)
`infra/k8s/config/`	Source templates and `appsettings.k8s.json`
`infra/k8s/overlays/dev/`	kustomize entry point
`infra/k8s/overlays/dev/config/`	Generated configs (written by `tools/k8s-config.sh`)

The dev overlay mirrors Pulumi resource shapes (infra/pulumi/): same StatefulSet/Service naming, split RabbitMQ (rabbitmq-internal, rabbitmq-log), and per-shard MySQL/Redis — with a single world (dev) and minimal shard counts. Production scales shards/replicas via Pulumi develop.json.

Image strategy

The dev overlay defaults to fb/*:local with imagePullPolicy: IfNotPresent.

Cluster type	Approach
Single-node (kind, minikube, k3s on one host)	`bash tools/compose-build.sh`, then load images into the cluster runtime (e.g. `kind load docker-image fb/game:local`)
Multi-node (containerd, no shared Docker)	Push to a registry the nodes can pull, or retag GHCR images in `overlays/dev/kustomization.yaml` `images:`
Quick smoke test	Pull public GHCR tags and patch at apply time (see Verify deployment)

On multi-node clusters, Docker-built fb/*:local tags are not visible to kubelet (containerd). Importing into containerd (ctr -n k8s.io images import) requires root on each node — prefer a registry in production.

Resources created:

Resource	Services
Namespace	`fb`
StatefulSet + Service	MySQL (unified + per-world global/data/log), Redis (unified + per-world), RabbitMQ (internal + log), Gateway, Login, Game
Deployment	Internal, Write-back, Log, Marketplace, Admin Tool
NodePort / ClusterIP	Gateway, Login, Game (client TCP); infra NodePorts optional
ConfigMap / Secret	`mysql-secret`, per-shard configs, C++ `config.json`, `appsettings.k8s.json`

Boot order

Namespace
MySQL, Redis, RabbitMQ
Internal, Write-back, Admin Tool, Log, Marketplace
Gateway, Login, Game

Wait for infrastructure StatefulSets to become ready before application pods start.

Client-facing address

Gateway, Login, and Game must advertise an IP or hostname that game clients can reach — not in-cluster DNS names like gateway.fb.svc.cluster.local.

tools/k8s-config.sh reads infra/k8s/config/fb-host.env, renders C++ configs from templates, and syncs all configs into overlays/dev/config/ for kustomize. Clients connect via NodePort on the node IP (gateway 30000).

3. Configuration

Setting	Notes
Client `host`	`tools/k8s-config.sh` + `infra/k8s/config/fb-host.env`
NodePorts	`infra/k8s/apps/` and `infra/k8s/infra/` Services
`database.autoMigration`	Enabled in internal `appsettings.k8s.json`
Image registry	`overlays/dev/kustomization.yaml` `images:`
World name	`dev` — resources like `login-dev`, `game-dev-0`, `mysql-dev-global`

Default NodePorts (dev overlay)

Service	NodePort
Gateway	`30000`
Login (`login-dev`)	`30010`
Game (`game-dev-0`)	`30030`
Internal (HTTP)	`30200`
Admin tool	`30210`
Marketplace	`30220`
MySQL unified	`31000`
Redis unified	`31010`
RabbitMQ internal (AMQP)	`31020`

Add offset ports for additional worlds/shards (same pattern as Pulumi develop.json).

Infra NodePorts should not collide with client services.

4. Verify deployment

Replace <host> with the node IP clients reach (same value as FB_HOST in fb-host.env).

kubectl -n fb get pods -o wide
kubectl -n fb get svc

# HTTP (NodePort)
curl -sf http://<host>:30200/health    # internal
curl -sf http://<host>:30220/health    # marketplace

# Client TCP (NodePort — note ports differ from Compose)
nc -zv <host> 30000   # gateway
nc -zv <host> 30010   # login
nc -zv <host> 30030   # game

Quick test with GHCR images (multi-node, no local build):

FB_HOST=<host> bash tools/k8s-config.sh
kubectl kustomize infra/k8s/overlays/dev \
  | sed -e 's|fb/\([^:]*\):local|ghcr.io/boyism80/fb/\1:latest|g' \
        -e 's|imagePullPolicy: IfNotPresent|imagePullPolicy: Always|g' \
  | kubectl apply -f -

On multi-node clusters, ensure infra pods schedule on nodes with /mnt/fb/ (label infra=true on ubuntu-1-style workers), or cordon nodes without storage during testing.

With auto-migration enabled, Internal applies SQL from infra/db/migrations/ on first start. Allow ~60 seconds for MySQL StatefulSets and .NET migrations before expecting all pods Ready.

Troubleshooting

Symptom	Likely cause
Pods Pending (MySQL/Redis)	Missing `/mnt/fb/` host path, or no schedulable node
`ImagePullBackOff`	`fb/*:local` not in node containerd — use registry or `kind load`; see Image strategy
Client cannot connect	Wrong `FB_HOST` in C++ configs, firewall, or NodePort on unreachable node
NodePort conflict	Duplicate port in Service definitions
CrashLoop after deploy	Infra not ready; .NET services restart until MySQL migrations finish
Pods on wrong node	Multi-node cluster without `/mnt/fb` on that node — add `infra=true` label or adjust affinity

Tear down

kubectl delete -k infra/k8s/overlays/dev
# or: kubectl delete namespace fb

Re-run tools/k8s-config.sh before the next apply if FB_HOST changed.

Related files

Path	Role
`server/*/Dockerfile`	Image definitions
`.github/workflows/build.yml`	CI image build reference
`infra/db/migrations/`	SQL applied by Internal on startup
`infra/k8s/`	Kustomize manifests and configs
`tools/k8s-config.sh`	Render C++ configs and sync `overlays/dev/config/`

Related pages

Home

Server

Setup

Windows
- Runner
- Build from source
Linux
- Docker Compose
- Kubernetes
- Pulumi (optional)

Scripting

Builtin functions

Installation Linux Kubernetes

Linux — Kubernetes

Overview

Prerequisites

Cluster

Build machine

1. Build Docker images

2. Deploy with kubectl

Image strategy

Boot order

Client-facing address

3. Configuration

Default NodePorts (dev overlay)

4. Verify deployment

Troubleshooting

Tear down

Related files

Related pages

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Clone this wiki locally