Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Ratio algorithm for dynamic load balancing #718

Merged
merged 39 commits into from
Jun 3, 2017
Merged

Ratio algorithm for dynamic load balancing #718

Show file tree
Hide file tree
Changes from 38 commits
Commits
Show all changes
39 commits
Select commit Hold shift + click to select a range
9cd0798
Separate percentile numbers from percentile values.
keshonok Dec 17, 2016
6423d3f
Remove the possibility of negative array indices.
keshonok Dec 17, 2016
2951467
Put all APM stats together in the same indexed array.
keshonok Dec 18, 2016
d276901
Implement scheduler and server options related to dynamic sheduling.
keshonok Dec 20, 2016
2967e12
Rework processing of server-related configuration entries.
keshonok Mar 22, 2017
183af75
Initial barebone version of ratio scheduler.
keshonok Mar 26, 2017
aa8f44f
Incorporate ALB algorithm suggested by @krizhanovsky.
keshonok Mar 27, 2017
c126598
Remove locks from tfw_sg_release_all().
keshonok Mar 30, 2017
7fadba9
Better sequence tracking for APM stats.
keshonok Mar 30, 2017
03b7a50
Get rid of duplicate EXPORT_SYMBOL(tfw_apm_stats) in unit tests.
keshonok Mar 30, 2017
1f8266b
Full support of static or dynamic weights of different values.
keshonok Mar 30, 2017
5172354
Merge branch 'master' into ab-alb-2
keshonok Apr 3, 2017
f204ebb
Bugfix: Adjust the sum of ratios after min and max ratios are exchanged.
keshonok Apr 3, 2017
b000209
Unit tests for Ratio Scheduler based on unit tests for RR scheduler.
keshonok Apr 4, 2017
5640772
Update the docs with new options related to ratio scheduler.
keshonok Apr 4, 2017
c55bf5f
Better description of ratio scheduler behavior and the weight option.
keshonok Apr 4, 2017
bab5e18
Remove RR scheduler. It's now replaced by Ratio scheduler.
keshonok Apr 4, 2017
ff0cb23
Remove .add_conn() callback. The functionality is moved to .add_grp().
keshonok Apr 7, 2017
f8c11b3
Address code review comments.
keshonok Apr 7, 2017
4340f5d
Predictive Scheduler implementation (simple linear regression).
keshonok Apr 18, 2017
8c4e508
Wait for RCU callbacks to complete before releasing memory.
keshonok Apr 19, 2017
7d167e6
Remove a tiny piece of dead code.
keshonok Apr 23, 2017
3eaab0f
Count the remainder from division by two (even/odd numbers).
keshonok Apr 24, 2017
50d3a83
Fix the initial value of RBCTL's entry_cnt in APM.
keshonok Apr 27, 2017
12280c9
Fix a WARNING() spewed by rcu_process_callbacks().
keshonok Apr 27, 2017
3d75192
Serialize APM stats data updates and APM stats values calculation.
keshonok Apr 28, 2017
a163fcc
Address code review issues raised by @ikoveshnikov.
keshonok May 2, 2017
8ee11de
Correctly release server's APM data on cleanup.
keshonok May 3, 2017
e8473a0
Fix schedulers unit tests - set sg->flags before sched registration.
keshonok May 5, 2017
bf850b8
Multiple minor changes to fix issues raised in code reviews.
keshonok May 15, 2017
7ef13ec
Merge branch 'master' into ab-alb-2
keshonok May 15, 2017
f6e11be
Fix the implementation of predictive ratio algorithm.
keshonok May 18, 2017
1850314
Replace ratio->free with ratio->busy for better clarity.
keshonok May 18, 2017
42140b3
Code rework to address comments in recent code review.
keshonok May 28, 2017
d893c06
Don't mix signed and unsigned types in calculations.
keshonok May 28, 2017
5c37ed1
A bit more data for servers in /proc/tempesta/servers/*.
keshonok May 28, 2017
a333f4c
Rework APM modules to work with per-CPU arrays for incoming data.
keshonok Jun 1, 2017
d834587
Add list_head argument to tfw_cfg_sg_ratio_adjust() for unit tests.
keshonok Jun 2, 2017
a044aae
Small cleanups in apm.c.
keshonok Jun 3, 2017
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
95 changes: 83 additions & 12 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -365,13 +365,17 @@ location prefix "/society/" {
A back end HTTP server is defined with `server` directive. The full syntax is
as follows:
```
server <IPADDR>[:<PORT>] [conns_n=<N>];
server <IPADDR>[:<PORT>] [conns_n=<N>] [weight=<N>];
```
`IPADDR` can be either IPv4 or IPv6 address. Hostnames are not allowed.
* `IPADDR` can be either IPv4 or IPv6 address. Hostnames are not allowed.
IPv6 address must be enclosed in square brackets (e.g. "[::0]" but not "::0").
`PORT` defaults to 80 if not specified.
`conns_n=<N>` is the number of parallel connections to the server.
* `PORT` defaults to 80 if not specified.
* `conns_n=<N>` is the number of parallel connections to the server.
`N` defaults to 32 if not specified.
* `weight=<N>` is the static weight of the server. The weight must be
in the range of 1 to 100. If not specified, then the default weight of 50
is used with the static ratio scheduler. Just the weight that differs from
default value may be specified for convenience.

Multiple back end servers may be defined. For example:
```
Expand Down Expand Up @@ -442,11 +446,12 @@ with this directive. If not specified, the queue size is set to 1000.
Back end servers can be grouped together into a single unit for the purpose of
load balancing. Servers within a group are considered interchangeable.
The load is distributed evenly among servers within a group.
If a server goes offline, other servers in a group take the load.
If a server goes offline, then other servers in a group take the load.
The full syntax is as follows:
```
srv_group <NAME> {
server <IPADDR>[:<PORT>] [conns_n=<N>];
sched <SCHED_NAME>;
server <IPADDR>[:<PORT>] [conns_n=<N>] [weight=<N>];
...
}
```
Expand Down Expand Up @@ -479,14 +484,18 @@ Scheduler is used to distribute load among servers within a group. The group
can be either explicit, defined with `srv_group` directive, or implicit.
The syntax is as follows:
```
sched <SCHED_NAME>;
sched <SCHED_NAME> [OPTIONS];
```
`SCHED_NAME` is the name of a scheduler available in Tempesta.
`OPTIONS` are optional. Not all schedulers have additional options.

Currently there are two schedulers available:
* **round-robin** - Rotates all servers in a group in round-robin manner so
that requests are distributed uniformly across servers. This is the default
scheduler.
* **ratio** - Balances the load across servers in a group based on each
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This patch replaces round-robin scheduler with ratio. Description of new options in Readme is great, but the sample configuration file etc/tempesta_fw.conf was not updated at all.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Functional tests sets scheduler to a default value (which was round-robin) in this file . This should be fixed, otherwise almost all the functional tests will be broken.

server's weight. Requests are forwarded more to servers with more weight,
and less to servers with less weight. As a result, each server in a group
receives an optimal load. In default configuration where weights are not
specified, servers weights are considered equal, and the scheduler works
in pure round-robin fashion. This is the default scheduler.
* **hash** - Chooses a server based on a URI/Host hash of a request.
Requests are distributed uniformly, and requests with the same URI/Host are
always sent to the same server.
Expand All @@ -504,8 +513,70 @@ A scheduler defined for the implicit group becomes the scheduler for an
explicit group defined with `srv_group` directive if the explicit group
is missing the `sched` directive.

If no scheduler is defined for a group, then scheduler defaults
to `round-robin`.
If no scheduler is defined, then scheduler defaults to `ratio`.

**ratio** scheduler may have the following options:
* **static** - The weight of each server in a group is defined statically
with `[weight=<NN>]` option of the `server` directive. This is the default
`ratio` scheduler option.
* **dynamic** - The weight of each server in a group is defined dynamically.
Specific type of dynamic weight is specified with additional options:
* **minimum** - The current minimum response time from a server;
* **maximum** - The current maximum response time from a server;
* **average** - The current average response time from a server;
* **percentile `[<NN>]`** - The current response time from a server that
is within specified percentile. The percentile may be one of 50, 75, 90,
95, 99. If none is given, then the default percentile of 90 is used.
If a specific type of dynamic weight is not specified, then the default type
of `average` is used.
* **predict** - The weight of each server in a group is predicted dynamically
for a time in the future, based on server's behavior in the past. Additional
options include those that are defined for **dynamic** weight, as well as
the following options:
* **past** - Period of time (in seconds) to keep past response time
values from a server. The default value is 30 seconds.
* **rate** - Rate (times per second) of retrieval of past response time
values. The default value is 20 times per second.
* **ahead** - Period of time (in seconds) for which to make a prediction;
It can't be more than half of **past**. The default value is 15 seconds.

Naturally, if a Dynamic Scheduler is specified for a group, and there's
a server in that group with the `weight` option, then an error is produced
as that combination is incompatible. Same is true for Predictive Scheduler.

The following are examples of scheduler specification in configuration.
Again, only one `sched` directive is allowed per group.
```
# Use hash scheduler
sched hash;
# Use ratio scheduler. By default, static weight distribution is used.
sched ratio;
# Use ratio scheduler with static weight distribution.
sched ratio static;
# Use dynamic scheduler. By default, current average response time is used
# for weight distribution.
sched dynamic;
# Use dynamic scheduler with maximum response time for weight distribution.
sched dynamic maximum;
# Use dynamic scheduler, default percentile of 90 is used.
sched dynamic percentile;
# Use dynamic scheduler, percentile of 75 is used for weight distribution.
sched dynamic percentile 75;
# Use predictive scheduler, percentile of 75 is used for weight distribution.
# The values of weights of each server are collected for past 60 seconds
# at the rate of 20 times per second, the weight of each server in predicted
# for the time of 2 seconds ahead.
sched predict percentile 75 past=60 rate=20 ahead=2;
```

Servers should be grouped together with proper care. Server groups should
be created with servers that handle similar resources. For instance, if
servers with static content that is served quickly are grouped together
with servers with dynamic content that is I/O bound, then the quick
response times from servers with static content will be nearly invisible
in comparison to longer response times from servers with dynamic content.
In that case the distribution of load among these servers will be severely
skewed.


#### HTTP Scheduler
Expand Down
56 changes: 47 additions & 9 deletions etc/tempesta_fw.conf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,17 +8,49 @@
# a group.
#
# Syntax:
# sched SCHED_NAME;
# sched SCHED_NAME [OPTIONS];
#
# SCHED_NAME is a name of a scheduler module that distributes the load
# among servers within a group. There are two schedulers available:
# - "round-robin" (default) - rotates all servers in the group in
# the round-robin manner, so requests are distributed uniformly across
# servers.
# - "ratio" (default) - Balances the load across servers in a group based
# on each server's weight. Requests are forwarded more to servers with
# more weight, and less to servers with less weight. As a result, each
# server in a group receives an optimal load. In default configuration
# where weights are not specified, servers weights are considered equal,
# and the scheduler works in pure round-robin fashion.
# - "hash" - chooses a server based on a URI/Host hash of a request.
# Requests are still distributed uniformly, but a request with the same
# URI/Host is always sent to the same server.
#
# OPTIONS are optional. Not all schedulers have additional options.
#
# "ratio" scheduler may have the following options:
# - static - The weight of each server in a group is defined statically
# with [weight=<NN>] option of the `server` directive. This is the
# default Ratio scheduler option.
# - dynamic - The weight of each server in a group is defined dynamically.
# Specific type of dynamic weight is specified with additional options:
# - minimum - The current minimum response time from a server;
# - maximum - The current maximum response time from a server;
# - average - The current average response time from a server;
# - percentile [<NN>] - The current response time from a server
# that is within specified percentile. The percentile may be
# one of 50, 75, 90, 95, 99. If none is given, then the default
# percentile of 90 is used.
# If a specific type of dynamic weight is not specified, then
# the default type of "average" is used.
# - predict - The weight of each server in a group is predicted dynamically
# for a time in the future, based on server's behavior in the past.
# Additional options include those that are defined for "dynamic" weight,
# as well as the following options:
# - past - Period of time (in seconds) to keep past response time
# values from a server. The default value is 30 seconds.
# - rate - Rate (times per second) of retrieval of past response time
# values. The default value is 20 times per second.
# - ahead - Period of time (in seconds) for which to make a prediction;
# It can't be more than half of **past**. The default value is 15
# seconds.
#
# Note that there's also the HTTP scheduler. It dispatches requests among
# server groups only. Round-robin or hash scheduler must be used to select
# a server within a group.
Expand All @@ -30,25 +62,31 @@
# the `sched` directive.
#
# Default:
# sched round-robin;
# sched ratio;
#

# TAG: server.
#
# Specifies an IP address/port of a back-end HTTP server.
#
# Syntax:
# server IPADDR[:PORT] [conns_n=N]
# server IPADDR[:PORT] [conns_n=N] [weight=N];
#
# IPADDR may be either IPv4 or IPv6 address, hostnames are not allowed.
# IPv6 address must be enclosed in square brackets (e.g. "[::0]" but not "::0").
# PORT defaults to 80 if not set.
#
# conns_n=N is the number of parallel connections to the server.
# The N defaults to 32 if not set.
# The N defaults to 32 if the option is not specified.
#
# weight=N is the static weight of the server. The weight must be in
# the range of 1 to 100. If not specified, then the default weight of 50
# is used with the static ratio scheduler. Just the weight that differs
# from default value may be specified for convenience.
#
#
# Multiple back-end servers may be specified, for example:
# server 10.1.0.1:80
# server 10.1.0.1:80;
# server [fc00::1]:80;
#
# Default:
Expand All @@ -60,7 +98,7 @@
# Defines a request that is considered non-idempotent.
#
# Syntax:
# nonidempotent <METHOD> <OP> <string>
# nonidempotent <METHOD> <OP> <string>;
#
# <METHOD> is one of supported HTTP methods, such as GET, HEAD, POST, etc.
# <OP> is a string matching operator, one of "eq", "prefix", "suffix", or "*".
Expand Down
Loading