/
doc.go
36 lines (36 loc) · 1.91 KB
/
doc.go
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
// Package scheduler implements filter logic that changes the http
// request scheduling behavior of the proxy.
//
// The proxy has as default an unbounded scheduler that does not limit
// inflight requests. Goroutines with parsed request data consume
// memory. The unbounded handler could spike in memory, if you have
// traffic on a backend that has too big response times. You can check
// the number of goroutines from skipper metrics, if you have this
// problem.
//
// The scheduler filter package has two implementations of bounded
// queue, the lifo and lifoGroup filter. Both lifo filters will, use a
// last in first out queue to handle most requests fast and if skipper
// is in an overrun mode, it will serve some requests fast and some
// will timeout. This scheduler implementation makes sure that one
// route will not interfere with other routes, if these routes are not
// in the same scheduler group. LifoGroup has a user specified
// scheduler group and lifo will get a per route unique scheduler
// group.
//
// Bounded schedulers were tested in Kubernetes with 3 proxy instances
// with 500m CPU and 500Mi memory resources. The load test was done
// with 500 requests per second to backends with 25 seconds latency
// and a second load test was done in parallel with 50 150 250 .. 1000
// requests per second to backends with no additional latency. For the
// workload without additional latency there was no additional latency
// measurable. The memory was at maximum 350Mi with the bounded
// scheduler. The unbounded scheduler spiked in memory to above 500Mi,
// which caused an out of memory (OOM) kill by the operating system.
//
// Bounded schedulers will respond to requests with server status error
// codes in case of overrun. The scheduler returns HTTP status code:
//
// - 502, if it can not get a request from data structure fast enough
// - 503, if the data structure is full and reached its boundary
package scheduler