Skip to content

Commit 0003c56

Browse files
justincastillaJustin CastillaSimon Prickett
authored
Adds RU301 - Running Redis at Scale to developer.redis.com (#318)
* initial ru301 section additions, stubs and profiles * Added link to Discord. * Formatting / occasional light copy edit. * Added front matter to disable the edit this page button. * adds conclusion page for RU301 Co-authored-by: Justin Castilla <pxlperfection@gmail.com> Co-authored-by: Simon Prickett <simon@redislabs.com>
1 parent 1963bef commit 0003c56

File tree

36 files changed

+2073
-347
lines changed

36 files changed

+2073
-347
lines changed

‎docs/operate/index-operate.mdx

+7-1
Original file line numberDiff line numberDiff line change
@@ -40,7 +40,13 @@ The following links demonstrate various ways to provision Redis and accelerate a
4040
page="/operate/provisioning"
4141
/>
4242
</div>
43-
43+
<div class="col scale">
44+
<RedisCard
45+
title="Running Redis at Scale"
46+
description="Create a performant, stable, and secure deployment of Redis"
47+
page="/operate/redis-at-scale"
48+
/>
49+
</div>
4450
<div class="col orchestration">
4551
<RedisCard
4652
title="Orchestration"

‎docs/operate/index-operate.mdx.orig

+7-1
Original file line numberDiff line numberDiff line change
@@ -57,7 +57,13 @@ The following links provide information about how to operate your Redis database
5757
page="/operate/provisioning"
5858
/>
5959
</div>
60-
60+
<div class="col scale">
61+
<RedisCard
62+
title="Running Redis at Scale"
63+
description="Create a performant, stable, and secure deployment of Redis"
64+
page="/operate/redis-at-scale"
65+
/>
66+
</div>
6167
<div class="col">
6268
<RedisCard
6369
title="Orchestration"

‎docs/operate/redis-at-scale/course-wrap-up/index-wrap-up.mdx

+32
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,32 @@
1+
---
2+
id: index-wrap-up
3+
title: Conclusion of Running Redis at Scale
4+
sidebar_label: Course Wrap-up
5+
slug: /operate/redis-at-scale/course-wrap-up/
6+
authors: [justin, elena, kurt]
7+
---
8+
9+
10+
You've made it! Thanks again for trying out this course on the Redis Developer Site. We hope you've enjoyed it, and we hope it's provided you with the tools you need to successfully scale Redis with your applications.
11+
<br/>
12+
<img src="https://university.redis.com/assets/images/redis-university-banner.jpg" width="50%" height="50%" />
13+
<br/>
14+
15+
If you would like to receive a certificate of completion for this course, head to [Redis University](https://university.redis.com/courses/ru301/) to enroll in the full-format class which includes homework for each course section. If you pass the course with a grade of sixty-five percent or greater, you'll be able to generate your certificate and post it to your LinkedIn profile.
16+
17+
<div align="center">
18+
<iframe width="50%" height="300" src="https://www.youtube.com/embed/V0wmD_y03iM" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>
19+
</div>
20+
<br/>
21+
22+
Please consider subscribing to our [Youtube Channel](https://www.youtube.com/redisinc) to stay up to date with all of our latest tutorials, interviews, and general news.
23+
24+
And if you have any feedback or insights you want to share with the Redis University team, don't hesitate to leave a note in our online chat on our Discord server found [here](https://discord.gg/WT6fbUMuWU).
25+
26+
Again, we're grateful you've taken the time to work through our course. Happy learning and see you next time!
27+
28+
Best wishes,
29+
30+
Elena, Kurt, and Justin
31+
32+

‎docs/operate/redis-at-scale/high-availability/basic-replication/index-basic-replication.mdx

+11
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
---
2+
id: index-basic-replication
3+
title: 3.1 Basic Replication
4+
sidebar_label: 3.1 Basic Replication
5+
slug: /operate/redis-at-scale/high-availability/basic-replication
6+
isEditable: false
7+
---
8+
9+
import useBaseUrl from '@docusaurus/useBaseUrl';
10+
11+
<iframe width="586" height="330" src="https://www.youtube.com/embed/-osCdf90tRA" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>

‎docs/operate/redis-at-scale/high-availability/exercise-1/index-exercise-1.mdx

+91
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,91 @@
1+
---
2+
id: index-exercise-1
3+
title: 3.2 Exercise - Enabling Basic Replication
4+
sidebar_label: 3.2 Exercise - Enabling Basic Replication
5+
slug: /operate/redis-at-scale/high-availability/exercise-1
6+
isEditable: false
7+
---
8+
9+
import useBaseUrl from '@docusaurus/useBaseUrl';
10+
11+
## Step 1
12+
First let’s create and configure the primary instance. We’ll start with a few configuration changes in its `primary.conf` configuration file.
13+
14+
```bash
15+
$ touch primary.conf # Create the configuration file
16+
```
17+
18+
Now open the `primary.conf` file with your favorite text editor and set the following configuration directives:
19+
20+
```bash
21+
# Create a strong password here
22+
requirepass a_strong_password
23+
24+
# AUTH password of the primary instance in case this instance becomes a replica
25+
masterauth a_strong_password
26+
27+
# Enable AOF file persistence
28+
appendonly yes
29+
30+
# Choose a name for the AOF file
31+
appendfilename "primary.aof"
32+
```
33+
34+
Finally, let’s start the primary instance:
35+
36+
```bash
37+
$ redis-server ./primary.conf
38+
```
39+
40+
## Step 2
41+
Next, let’s prepare the configuration file for the replica:
42+
```bash
43+
$ touch replica.conf
44+
```
45+
Let’s add some settings to the file we just created:
46+
```bash
47+
# Port on which the replica should run
48+
port 6380
49+
50+
# Address of the primary instance
51+
replicaof 127.0.0.1 6379
52+
53+
# AUTH password of the primary instance
54+
masterauth a_strong_password
55+
56+
# AUTH password for the replica instance
57+
requirepass a_strong_password
58+
```
59+
And let’s start the replica:
60+
```bash
61+
$ redis-server ./replica.conf
62+
```
63+
## Step 3
64+
Open two terminal tabs and use them to start connections to the primary and replica instances:
65+
```bash
66+
# Tab 1 (primary)
67+
$ redis-cli
68+
```
69+
```bash
70+
# Tab 2 (replica)
71+
$ redis-cli -p 6380
72+
```
73+
Authenticate on both tabs by running the command `AUTH` followed by your password:
74+
```bash
75+
AUTH a_strong_password
76+
```
77+
78+
On the second (replica) tab run the `MONITOR` command which will allow you to see every command executed against that instance.
79+
80+
Go back to the first (primary) tab and execute any write command, for example
81+
82+
```bash
83+
127.0.0.1:6379> SET foo bar
84+
```
85+
In the second tab you should see that the command was already sent to the replica:
86+
```bash
87+
1617230062.389077 [0 127.0.0.1:6379] "SELECT" "0"
88+
1617230062.389092 [0 127.0.0.1:6379] "set" "foo" "bar"
89+
```
90+
## Step 4
91+
Keep the instances running, or at least their configuration files around. We’ll need them for the next exercise.

‎docs/operate/redis-at-scale/high-availability/exercise-2/index-exercise-2.mdx

+84
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,84 @@
1+
---
2+
id: index-exercise-2
3+
title: 3.4 Exercise - Sentinel Hands-on
4+
sidebar_label: 3.4 Exercise - Sentinel Hands-on
5+
slug: /operate/redis-at-scale/high-availability/exercise-2
6+
isEditable: false
7+
---
8+
9+
import useBaseUrl from '@docusaurus/useBaseUrl';
10+
11+
## Step 1
12+
If you still have the primary and replica instances we set up in the previous exercise (3.2) - great! We’ll reuse them to create our Sentinel setup. If not - refer back to the [instructions](/operate/redis-at-scale/high-availability/exercise-1) and go through them again.
13+
14+
When done, you will have a primary Redis instance with one replica.
15+
16+
## Step 2
17+
To initialise a Redis Sentinel, you need to provide a configuration file, so let’s go ahead and create one:
18+
19+
```bash
20+
$ touch sentinel1.conf
21+
```
22+
23+
Open the file and paste in the following settings:
24+
25+
```bash
26+
port 5000
27+
sentinel monitor myprimary 127.0.0.1 6379 2
28+
sentinel down-after-milliseconds myprimary 5000
29+
sentinel failover-timeout myprimary 60000
30+
sentinel auth-pass myprimary a_strong_password
31+
```
32+
33+
<strong>Breakdown of terms:</strong>
34+
35+
- `port` - The port on which the Sentinel should run
36+
- `sentinel monitor` - monitor the Primary on a specific IP address and port. Having the address of the Primary the Sentinels will be able to discover all the replicas on their own. The last argument on this line is the number of Sentinels needed for quorum. In our example - the number is 2.
37+
- `sentinel down-after-milliseconds` - how many milliseconds should an instance be unreachable so that it’s considered down
38+
- `sentinel failover-timeout` - if a Sentinel voted another Sentinel for the failover of a given master, it will wait this many milliseconds to try to failover the same master again.
39+
- `sentinel auth-pass` - In order for Sentinels to connect to Redis server instances when they are configured with `requirepass`, the Sentinel configuration must include the sentinel auth-pass directive.
40+
41+
## Step 3
42+
Make two more copies of this file - `sentinel2.conf` and `sentinel3.conf` and edit them so that the `PORT` configuration is set to 5001 and 5002, respectively.
43+
44+
## Step 4
45+
Let’s initialise the three Sentinels in three different terminal tabs:
46+
47+
```bash
48+
# Tab 1
49+
$ redis-server ./sentinel1.conf --sentinel
50+
```
51+
```bash
52+
# Tab 2
53+
$ redis-server ./sentinel2.conf --sentinel
54+
```
55+
```bash
56+
# Tab3
57+
$ redis-server ./sentinel3.conf --sentinel
58+
```
59+
60+
## Step 5
61+
If you connected to one of the Sentinels now you would be able to run many new commands that would give an error if run on a Redis instance. For example:
62+
63+
```bash
64+
# Provides information about the Primary
65+
SENTINEL master myprimary
66+
67+
# Gives you information about the replicas connected to the Primary
68+
SENTINEL replicas myprimary
69+
70+
# Provides information on the other Sentinels
71+
SENTINEL sentinels myprimary
72+
73+
# Provides the IP address of the current Primary
74+
SENTINEL get-master-addr-by-name myprimary
75+
```
76+
77+
## Step 6
78+
If we killed the primary Redis instance now by pressing Ctrl+C or by running the `redis-cli -p 6379 DEBUG sleep 30` command, we’ll be able to observe in the Sentinels’ logs that the failover process will start in about 5 seconds. If you run the command that returns the IP address of the Primary again you will see that the replica has been promoted to a Primary:
79+
80+
```bash
81+
redis> SENTINEL get-master-addr-by-name myprimary
82+
1) "127.0.0.1"
83+
2) "6380"
84+
```

‎docs/operate/redis-at-scale/high-availability/index-high-availability.mdx

+12
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
---
2+
id: index-high-availability
3+
title: Ensuring High Availability in Redis
4+
sidebar_label: High Availability
5+
slug: /operate/redis-at-scale/high-availability
6+
authors: [justin]
7+
isEditable: false
8+
---
9+
10+
import useBaseUrl from '@docusaurus/useBaseUrl';
11+
12+
Hello World!

‎docs/operate/redis-at-scale/high-availability/introduction/index-index-introduction.mdx

+11
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
---
2+
id: index-introduction
3+
title: 3.0 Introduction to High Availability
4+
sidebar_label: 3.0 Introduction to High Availability
5+
slug: /operate/redis-at-scale/high-availability/introduction
6+
isEditable: false
7+
---
8+
9+
import useBaseUrl from '@docusaurus/useBaseUrl';
10+
11+
<iframe width="586" height="330" src="https://www.youtube.com/embed/X6D2AQ_FCl0" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>

‎docs/operate/redis-at-scale/high-availability/understanding-sentinels/index-understanding-sentinels.mdx

+38
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
1+
---
2+
id: index-understanding-sentinels
3+
title: 3.3 Understanding Sentinels
4+
sidebar_label: 3.3 Understanding Sentinels
5+
slug: /operate/redis-at-scale/high-availability/understanding-sentinels
6+
isEditable: false
7+
---
8+
9+
import useBaseUrl from '@docusaurus/useBaseUrl';
10+
11+
In the beginning of this unit, we learned that we can’t have high availability without replication and automatic failover. We covered replication in the previous two chapters, and now we’ll explain Sentinel - a tool that provides the automatic failover.
12+
13+
Redis Sentinel is a distributed system consisting of multiple Redis instances started in sentinel mode. We call these instances <strong>Sentinels</strong>.
14+
15+
The group of Sentinels monitors a primary Redis instance and its replicas. If the sentinels detect that the primary instance has failed, the sentinel processes will look for the replica that has the latest data and will promote that replica to be the new primary. This way, the clients talking to the database will be able to reconnect to the new primary and continue functioning as usual, with minimal disruption to the users.
16+
17+
<br/>
18+
<img height="75%" width="75%" src="https://s3.us-east-2.amazonaws.com/assets-university.redislabs.com/ru301/3.4/image1.png" alt="Sentinel Quorum Diagram"/>
19+
<br/>
20+
21+
## Deciding that a primary instance is down
22+
23+
In order for the Sentinels to be able to decide that a primary instance is down we need to have enough Sentinels agree that the server is unreachable from their point of view.
24+
25+
Having a number of Sentinels agreeing that they need to take an action is called <strong>reaching a quorum</strong>. If the Sentinels can’t reach quorum, they cannot decide that the primary has failed. The exact number of Sentinels needed for quorum is configurable.
26+
27+
## Triggering a failover
28+
Once the Sentinels have decided that a primary instance is down, they need to elect and authorize a leader (a Sentinel instance) that will do the failover. A leader can only be chosen if the majority of the Sentinels agree on it.
29+
30+
In the final step, the leader will reconfigure the chosen replica to become a primary by sending the command `REPLICAOF NO ONE` and it will reconfigure the other replicas to follow the newly promoted primary.
31+
32+
## Sentinel and Client Libraries
33+
34+
If you have a system that uses Sentinel for high availability, then you need to have a client that supports Sentinel. Not all libraries have this feature, but most of the popular ones do, so make sure you add it to your list of requirements when choosing your library.
35+
36+
## Further Reading
37+
38+
For more information on Redis Sentinel, check out the [documentation](https://redis.io/topics/sentinel) on redis.io.

‎docs/operate/redis-at-scale/index-redis-at-scale.mdx

+42
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,42 @@
1+
---
2+
id: index-redis-at-scale
3+
title: Introduction to Running Redis at Scale
4+
sidebar_label: Overview
5+
slug: /operate/redis-at-scale
6+
authors: [justin, elena, kurt]
7+
isEditable: false
8+
---
9+
10+
import RedisCard from '@site/src/theme/RedisCard';
11+
12+
## Welcome
13+
<div class="text--center">
14+
<iframe width="560" height="315" src="https://www.youtube.com/embed/3H896s0rr8E" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>
15+
</div>
16+
17+
This course is broken up into units covering topics around scaling Redis for production deployment.
18+
19+
Scaling means more than just performance. We have tried to identify key topics that will help you have a performant, stable, and secure deployment of Redis. This course is divided into the following units:
20+
21+
- Talking to Redis: connection management and tuning of Redis.
22+
- Persistence/Durability: options persisting Redis data to disk.
23+
- High Availability: how to make sure Redis and your data is always there.
24+
- Scalability: scaling Redis for both higher throughput and capacity.
25+
- Observability: Visibility into your Redis deployment (metrics, etc.).
26+
27+
Our goal is to give you all the information you need to run Redis at scale, in whichever way is best for your organization. We hope you enjoy the course, and please don't hesitate to reach out on the course [Discord channel](https://discord.gg/ZnjZbDMDub) if you have any questions along the way.
28+
29+
## Prerequisites
30+
31+
- Access to a Linux-based system and familiarity with it
32+
- Redis server and redis-cli installed (examples and exercises assume redis-server is in the $PATH)
33+
docker and docker-compose installed
34+
- A git client and access to clone repos in Github. Some exercises will come from the following repository: https://github.com/redislabs-training/ru301
35+
36+
<strong>Note: This repo contains sample demonstrations of Redis running in various scaled configurations, and is not directly correlated with all of the exercises in this course. See the specific exercise instructions for usage.</strong>
37+
38+
## Assumptions
39+
40+
- Comfortable with Linux Bash shell exercises
41+
- Legacy terminology in Redis uses 'master' and 'slave' but in the course we will use 'primary' and 'replica'. You will still see the legacy terms in many commands, configurations, and field names.
42+
- We will use $ to indicate command line prompt and > to indicate a redis-cli prompt

0 commit comments

Comments
 (0)