Permalink
Browse files

Packed: Fleet scaled down removes nodes from least used Nodes

This implements the strategy such that when a Fleet is scaled down, and
has the "Packed" strategy, it removed GameServers from Nodes that have the
least GameServers running on them.

This also fixes up some issues with the OpenAPI validation as well.

THis looks like it's working. Go team.
  • Loading branch information...
markmandel committed Oct 23, 2018
1 parent cccdc8a commit 860ebdafe82074c0fec470d9502aee8db3ebcf8f
@@ -7,14 +7,23 @@
Table of Contents
=================
* [Fleet Autoscaling](#fleet-autoscaling)
* [Autoscalng Concepts](#autoscalng-concepts)
* [Allocation Scheduling](#allocation-scheduling)
* [Fleet Scheduling](#fleet-scheduling)
* [Packed](#packed)
* [Allocation Scheduling Strategy](#allocation-scheduling-strategy)
* [Distributed](#distributed)
* [Allocation Scheduling Stategy](#allocation-scheduling-stategy)
* [Scheduling and Autoscaling](#scheduling-and-autoscaling)
* [Table of Contents](#table-of-contents)
* [Fleet Autoscaling](#fleet-autoscaling)
* [Autoscaling Concepts](#autoscaling-concepts)
* [Allocation Scheduling](#allocation-scheduling)
* [Pod Scheduling](#pod-scheduling)
* [Fleet Scale Down Strategy](#fleet-scale-down-strategy)
* [Fleet Scheduling](#fleet-scheduling)
* [Packed](#packed)
* [Allocation Scheduling Strategy](#allocation-scheduling-strategy)
* [Pod Scheduling Strategy](#pod-scheduling-strategy)
* [Fleet Scale Down Strategy](#fleet-scale-down-strategy-1)
* [Distributed](#distributed)
* [Allocation Scheduling Strategy](#allocation-scheduling-strategy-1)
* [Pod Scheduling Strategy](#pod-scheduling-strategy-1)
* [Fleet Scale Down Strategy](#fleet-scale-down-strategy-2)
Scheduling and autoscaling go hand in hand, as where in the cluster `GameServers` are provisioned
impacts how to autoscale fleets up and down (or if you would even want to)
@@ -42,6 +51,11 @@ Each `GameServer` is backed by a Kubernetes [`Pod`](https://kubernetes.io/docs/c
refers to the strategy that is in place that determines which node in the Kubernetes cluster the Pod is assigned to,
when it is created.
### Fleet Scale Down Strategy
Fleet Scale Down strategy refers to the order in which the `GameServers` that belong to a `Fleet` are deleted,
when Fleets are shrunk in size.
## Fleet Scheduling
There are two scheduling strategies for Fleets - each designed for different types of Kubernetes Environments.
@@ -74,8 +88,7 @@ for the infrastructure you use.
It attempts to _pack_ as much as possible into the smallest set of nodes, to make
scaling infrastructure down as easy as possible.
Currently, Allocation scheduling is the only aspect this strategy affects, but in future releases it will
also affect `GameServer` `Pod` scheduling, and `Fleet` scale down scheduling as well.
This affects Allocation Scheduling, Pod Scheduling and Fleet Scale Down Scheduling.
#### Allocation Scheduling Strategy
@@ -91,6 +104,11 @@ topology. This attempts to group together `GameServer` Pods within as few nodes
> The default Kubernetes scheduler doesn't do a perfect job of packing, but it's a good enough job for what we need -
at least at this stage.
#### Fleet Scale Down Strategy
With the "Packed" strategy, Fleets will remove `Ready` `GameServers` from Nodes with the _least_ number of `Ready` and
`Allocated` `GameServers` on them. Attempting to empty Nodes so that they can be safely removed.
### Distributed
```yaml
@@ -118,8 +136,7 @@ on bare metal, and the cluster size rarely changes, if at all.
This attempts to distribute the load across the entire cluster as much as possible, to take advantage of the static
size of the cluster.
Currently, the only thing the scheduling strategy affects is Allocation scheduling, but in future releases it will
also affect `GameServer` `Pod` scheduling, and `Fleet` scaledown scheduling as well.
This affects Allocation Scheduling, Pod Scheduling and Fleet Scale Down Scheduling.
#### Allocation Scheduling Strategy
@@ -129,4 +146,9 @@ number of allocated `GameServers` on them.
#### Pod Scheduling Strategy
Under the "Distributed" strategy, `Pod` scheduling is provided by the default Kubernetes scheduler, which will attempt
to distribute the `GameServer` `Pods` across as many nodes as possible.
to distribute the `GameServer` `Pods` across as many nodes as possible.
#### Fleet Scale Down Strategy
With the "Distributed" strategy, Fleets will remove `Ready` `GameServers` from Nodes with at random, to ensure
a distributed load is maintained.
@@ -25,7 +25,6 @@ spec:
spec:
ports:
- name: default
portPolicy: "dynamic"
containerPort: 7654
template:
spec:
@@ -57,7 +57,7 @@ properties:
ports:
title: array of ports to expose on the game server container
type: array
minItems: 0 # make this 1 in 0.4.0
minItems: 1
required:
- containerPort
items:
@@ -91,34 +91,11 @@ properties:
type: integer
minimum: 1
maximum: 65535
portPolicy: # remove this in 0.4.0
title: the port policy that will be applied to the game server
description: |
portPolicy has two options:
- "dynamic" (default) the system allocates a free hostPort for the gameserver, for game clients to connect to
- "static", user defines the hostPort that the game client will connect to. Then onus is on the user to ensure that the
port is available. When static is the policy specified, `hostPort` is required to be populated
scheduling:
type: string
enum:
- dynamic
- static
protocol: # remove this in 0.4.0
title: Protocol being used. Defaults to UDP. TCP is the only other option
type: string
enum:
- UDP
- TCP
containerPort: # remove this in 0.4.0
title: The port that is being opened on the game server process
type: integer
minimum: 1
maximum: 65535
hostPort: # remove this in 0.4.0
title: The port exposed on the host
description: Only required when `portPolicy` is "static". Overwritten when portPolicy is "dynamic".
type: integer
minimum: 1
maximum: 65535
- Packed
- Distributed
health:
type: object
title: Health checking for the running game server
@@ -43,9 +43,15 @@ spec:
replicas:
type: integer
minimum: 0
scheduling:
type: string
enum:
- Packed
- Distributed
strategy:
properties:
type:
type: string
enum:
- Recreate
- RollingUpdate
@@ -50,8 +50,9 @@ spec:
- type
properties:
type:
type: string
enum:
- Buffer
- Buffer
buffer:
required:
- maxReplicas
@@ -44,5 +44,10 @@ spec:
replicas:
type: integer
minimum: 0
scheduling:
type: string
enum:
- Packed
- Distributed
template:
{{- include "gameserver.validation" . | indent 14 }}
View
@@ -184,9 +184,15 @@ spec:
replicas:
type: integer
minimum: 0
scheduling:
type: string
enum:
- Packed
- Distributed
strategy:
properties:
type:
type: string
enum:
- Recreate
- RollingUpdate
@@ -234,7 +240,7 @@ spec:
ports:
title: array of ports to expose on the game server container
type: array
minItems: 0 # make this 1 in 0.4.0
minItems: 1
required:
- containerPort
items:
@@ -268,34 +274,11 @@ spec:
type: integer
minimum: 1
maximum: 65535
portPolicy: # remove this in 0.4.0
title: the port policy that will be applied to the game server
description: |
portPolicy has two options:
- "dynamic" (default) the system allocates a free hostPort for the gameserver, for game clients to connect to
- "static", user defines the hostPort that the game client will connect to. Then onus is on the user to ensure that the
port is available. When static is the policy specified, `hostPort` is required to be populated
scheduling:
type: string
enum:
- dynamic
- static
protocol: # remove this in 0.4.0
title: Protocol being used. Defaults to UDP. TCP is the only other option
type: string
enum:
- UDP
- TCP
containerPort: # remove this in 0.4.0
title: The port that is being opened on the game server process
type: integer
minimum: 1
maximum: 65535
hostPort: # remove this in 0.4.0
title: The port exposed on the host
description: Only required when `portPolicy` is "static". Overwritten when portPolicy is "dynamic".
type: integer
minimum: 1
maximum: 65535
- Packed
- Distributed
health:
type: object
title: Health checking for the running game server
@@ -421,8 +404,9 @@ spec:
- type
properties:
type:
type: string
enum:
- Buffer
- Buffer
buffer:
required:
- maxReplicas
@@ -515,7 +499,7 @@ spec:
ports:
title: array of ports to expose on the game server container
type: array
minItems: 0 # make this 1 in 0.4.0
minItems: 1
required:
- containerPort
items:
@@ -549,34 +533,11 @@ spec:
type: integer
minimum: 1
maximum: 65535
portPolicy: # remove this in 0.4.0
title: the port policy that will be applied to the game server
description: |
portPolicy has two options:
- "dynamic" (default) the system allocates a free hostPort for the gameserver, for game clients to connect to
- "static", user defines the hostPort that the game client will connect to. Then onus is on the user to ensure that the
port is available. When static is the policy specified, `hostPort` is required to be populated
type: string
enum:
- dynamic
- static
protocol: # remove this in 0.4.0
title: Protocol being used. Defaults to UDP. TCP is the only other option
scheduling:
type: string
enum:
- UDP
- TCP
containerPort: # remove this in 0.4.0
title: The port that is being opened on the game server process
type: integer
minimum: 1
maximum: 65535
hostPort: # remove this in 0.4.0
title: The port exposed on the host
description: Only required when `portPolicy` is "static". Overwritten when portPolicy is "dynamic".
type: integer
minimum: 1
maximum: 65535
- Packed
- Distributed
health:
type: object
title: Health checking for the running game server
@@ -647,6 +608,11 @@ spec:
replicas:
type: integer
minimum: 0
scheduling:
type: string
enum:
- Packed
- Distributed
template:
required:
- spec
@@ -691,7 +657,7 @@ spec:
ports:
title: array of ports to expose on the game server container
type: array
minItems: 0 # make this 1 in 0.4.0
minItems: 1
required:
- containerPort
items:
@@ -725,34 +691,11 @@ spec:
type: integer
minimum: 1
maximum: 65535
portPolicy: # remove this in 0.4.0
title: the port policy that will be applied to the game server
description: |
portPolicy has two options:
- "dynamic" (default) the system allocates a free hostPort for the gameserver, for game clients to connect to
- "static", user defines the hostPort that the game client will connect to. Then onus is on the user to ensure that the
port is available. When static is the policy specified, `hostPort` is required to be populated
scheduling:
type: string
enum:
- dynamic
- static
protocol: # remove this in 0.4.0
title: Protocol being used. Defaults to UDP. TCP is the only other option
type: string
enum:
- UDP
- TCP
containerPort: # remove this in 0.4.0
title: The port that is being opened on the game server process
type: integer
minimum: 1
maximum: 65535
hostPort: # remove this in 0.4.0
title: The port exposed on the host
description: Only required when `portPolicy` is "static". Overwritten when portPolicy is "dynamic".
type: integer
minimum: 1
maximum: 65535
- Packed
- Distributed
health:
type: object
title: Health checking for the running game server
@@ -118,7 +118,7 @@ type GameServerSpec struct {
// Health configures health checking
Health Health `json:"health,omitempty"`
// Scheduling strategy. Defaults to "Packed".
Scheduling SchedulingStrategy `json:"scheduling"`
Scheduling SchedulingStrategy `json:"scheduling,omitempty"`
// Template describes the Pod that will be created for the GameServer
Template corev1.PodTemplateSpec `json:"template"`
}
@@ -57,7 +57,7 @@ type GameServerSetSpec struct {
// Replicas are the number of GameServers that should be in this set
Replicas int32 `json:"replicas"`
// Scheduling strategy. Defaults to "Packed".
Scheduling SchedulingStrategy `json:"scheduling"`
Scheduling SchedulingStrategy `json:"scheduling,omitempty"`
// Template the GameServer template to apply for this GameServerSet
Template GameServerTemplateSpec `json:"template"`
}
Oops, something went wrong.

0 comments on commit 860ebda

Please sign in to comment.