Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 39 additions & 0 deletions content/embeds/rdi-db-reqs.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
* Redis Enterprise v6.4 or greater for the cluster.
* For production, 250MB RAM with one primary and one replica is recommended, but for the
quickstart or for development, 125MB and a single shard is sufficient.
* If you are deploying RDI for a production environment then secure this database with a password
and TLS.
* Provide the installation with the required RDI database details.
* Set the database's
[eviction policy]({{< relref "/operate/rs/databases/memory-performance/eviction-policy" >}}) to `noeviction`. Note that you can't set this using
[`rladmin`]({{< relref "/operate/rs/references/cli-utilities/rladmin" >}}),
so you must either do it using the admin UI or with the following
[REST API]({{< relref "/operate/rs/references/rest-api" >}})
command:

```bash
curl -v -k -d '{"eviction_policy": "noeviction"}' \
-u '<USERNAME>:<PASSWORD>' \
-H "Content-Type: application/json" \
-X PUT https://<CLUSTER_FQDN>:9443/v1/bdbs/<BDB_UID>
```
* Set the database's
[data persistence]({{< relref "/operate/rs/databases/configure/database-persistence" >}})
to AOF - fsync every 1 sec. Note that you can't set this using
[`rladmin`]({{< relref "/operate/rs/references/cli-utilities/rladmin" >}}),
so you must either do it using the admin UI or with the following
[REST API]({{< relref "/operate/rs/references/rest-api" >}})
commands:

```bash
curl -v -k -d '{"data_persistence":"aof"}' \
-u '<USERNAME>:<PASSWORD>' \
-H "Content-Type: application/json"
-X PUT https://<CLUSTER_FQDN>:9443/v1/bdbs/<BDB_UID>
curl -v -k -d '{"aof_policy":"appendfsync-every-sec"}' \
-u '<USERNAME>:<PASSWORD>' \
-H "Content-Type: application/json" \
-X PUT https://<CLUSTER_FQDN>:9443/v1/bdbs/<BDB_UID>
```
* **Ensure that the RDI database is not clustered.** RDI will not work correctly if the
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please ensure the database is not clustered & sharded (please add a GUI screenshot from RE console to make it clear)

RDI database is clustered, but it is OK for the target database to be clustered.
5 changes: 5 additions & 0 deletions content/embeds/rdi-k8s-reqs.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
RDI only supports versions of Kubernetes and OpenShift that have
not yet reached end-of-life (EOL). See the
[Kubernetes](https://kubernetes.io/releases/) and
[OpenShift](https://access.redhat.com/support/policy/updates/openshift)
lifecycle pages for the latest updates.
2 changes: 2 additions & 0 deletions content/embeds/rdi-os-reqs.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
* RHEL 8 or 9
* Ubuntu 20.04, 22.04, or 24.04
8 changes: 8 additions & 0 deletions content/embeds/rdi-vm-reqs.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
* **CPU**: A minimum of 4 CPU cores. You should consider adding
2-6 extra cores on top of this if your dataset is big and you want to ingest the
baseline snapshot as fast as possible.
* **RAM**: 2GB
* **Disk**: On top of the OS footprint,
RDI requires 20GB in the `/var` folder and 1GB in the `/opt` folder (to
store the log files). This allows space for upgrades.
* **Network interface**: 10GB or more.
Original file line number Diff line number Diff line change
Expand Up @@ -22,5 +22,9 @@ generally, these mechanisms are not switched on by default.
RDI's Debezium collector uses these mechanisms for change data capture (CDC),
so you must prepare your source database before you can use it with RDI.

RDI supports the following source databases:

{{< embed-md "rdi-supported-source-versions.md" >}}

The pages in this section give detailed instructions to get your source
database ready for Debezium to use:
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ linkTitle: Install on K8s
summary: Redis Data Integration keeps Redis in sync with the primary database in near
real time.
type: integration
weight: 2
weight: 20
---

This guide explains how to use the RDI [Helm chart](https://helm.sh/docs/topics/charts/)
Expand Down Expand Up @@ -58,46 +58,10 @@ Complete the following steps before running Helm:
### Create the RDI database

RDI uses a database on your Redis Enterprise cluster to store its state
information. *This requires Redis Enterprise v6.4 or greater*.

- Use the Redis console to create a database with 250MB RAM with one primary and one replica.
- If you are deploying RDI for a production environment then secure this database with a password
and [TLS](https://en.wikipedia.org/wiki/Transport_Layer_Security).
- Set the database's
[eviction policy]({{< relref "/operate/rs/databases/memory-performance/eviction-policy" >}}) to `noeviction`. Note that you can't set this using
[`rladmin`]({{< relref "/operate/rs/references/cli-utilities/rladmin" >}}),
so you must either do it using the admin UI or with the following
[REST API]({{< relref "/operate/rs/references/rest-api" >}})
command:

```bash
curl -v -k -d '{"eviction_policy": "noeviction"}' \
-u '<USERNAME>:<PASSWORD>' \
-H "Content-Type: application/json" \
-X PUT https://<CLUSTER_FQDN>:9443/v1/bdbs/<BDB_UID>
```

- Set the database's
[data persistence]({{< relref "/operate/rs/databases/configure/database-persistence" >}})
to AOF - fsync every 1 sec. Note that you can't set this using
[`rladmin`]({{< relref "/operate/rs/references/cli-utilities/rladmin" >}}),
so you must either do it using the admin UI or with the following
[REST API]({{< relref "/operate/rs/references/rest-api" >}})
commands:

```bash
curl -v -k -d '{"data_persistence":"aof"}' \
-u '<USERNAME>:<PASSWORD>' \
-H "Content-Type: application/json"
-X PUT https://<CLUSTER_FQDN>:9443/v1/bdbs/<BDB_UID>
curl -v -k -d '{"aof_policy":"appendfsync-every-sec"}' \
-u '<USERNAME>:<PASSWORD>' \
-H "Content-Type: application/json" \
-X PUT https://<CLUSTER_FQDN>:9443/v1/bdbs/<BDB_UID>
```

- **Ensure that the RDI database is not clustered.** RDI will not work correctly if the
RDI database is clustered, but it is OK for the target database to be clustered.
information. Use the Redis console to create the RDI database with the following
requirements:

{{< embed-md "rdi-db-reqs.md" >}}

You should then provide the details of this database in the [`values.yaml`](#the-valuesyaml-file)
file as described below.
Expand Down Expand Up @@ -128,6 +92,10 @@ To pull images from a local registry, you must provide the image pull secret and
- [Google Kubernetes Engine (GKE)](https://cloud.google.com/artifact-registry/docs/pull-cached-dockerhub-images)
- [Azure Kubernetes Service (AKS)](https://learn.microsoft.com/en-us/azure/aks/cluster-container-registry-integration?tabs=azure-cli)

## Supported versions of Kubernetes and OpenShift

{{< embed-md "rdi-k8s-reqs.md" >}}

## Install the RDI Helm chart

1. Scaffold the default `values.yaml` file from the chart into a local
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ linkTitle: Install on VMs
summary: Redis Data Integration keeps Redis in sync with the primary database in near
real time.
type: integration
weight: 1
weight: 10
---

This guide explains how to install Redis Data Integration (RDI) on one or more VMs and integrate it with
Expand All @@ -28,14 +28,7 @@ your source database. You can also
RDI is mainly CPU and network bound.
Each of the RDI VMs should have at least:

- **CPU**: A minimum of 4 CPU cores. You should consider adding
2-6 extra cores on top of this if your dataset is big and you want to ingest the
baseline snapshot as fast as possible.
- **RAM**: 2GB
- **Disk**: 25GB, which includes the OS footprint. In particular,
RDI requires 7GB in `/var` and 1GB in `/opt` folder (to
store the log files).
- **Network interface**: 10GB or more.
{{< embed-md "rdi-vm-reqs.md" >}}

## Install RDI on VMs

Expand Down Expand Up @@ -71,8 +64,7 @@ known to work correctly with RDI.

The supported OS versions for RDI are:

- RHEL 8 & 9
- Ubuntu 20.04, 22.04, and 24.04
{{< embed-md "rdi-os-reqs.md" >}}

You must run the RDI installer as a privileged user because it installs
[containerd](https://containerd.io/) and registers services. However, you don't
Expand Down Expand Up @@ -192,52 +184,14 @@ Follow the steps below for each of your VMs:
{{< /note >}}

RDI uses a database on your Redis Enterprise cluster to store its state
information. *This requires Redis Enterprise v6.4 or greater*.
information.

The installer gives you instructions to help you create secrets and create your pipeline.
It will ask you for cluster admin credentials during installation.

Use the Redis console to create the RDI database with the following requirements:

- 250MB RAM with one primary and one replica.
- If you are deploying RDI for a production environment then secure this database with a password
and TLS.
- Provide the installation with the required RDI database details.
- Set the database's
[eviction policy]({{< relref "/operate/rs/databases/memory-performance/eviction-policy" >}}) to `noeviction`. Note that you can't set this using
[`rladmin`]({{< relref "/operate/rs/references/cli-utilities/rladmin" >}}),
so you must either do it using the admin UI or with the following
[REST API]({{< relref "/operate/rs/references/rest-api" >}})
command:

```bash
curl -v -k -d '{"eviction_policy": "noeviction"}' \
-u '<USERNAME>:<PASSWORD>' \
-H "Content-Type: application/json" \
-X PUT https://<CLUSTER_FQDN>:9443/v1/bdbs/<BDB_UID>
```

- Set the database's
[data persistence]({{< relref "/operate/rs/databases/configure/database-persistence" >}})
to AOF - fsync every 1 sec. Note that you can't set this using
[`rladmin`]({{< relref "/operate/rs/references/cli-utilities/rladmin" >}}),
so you must either do it using the admin UI or with the following
[REST API]({{< relref "/operate/rs/references/rest-api" >}})
commands:

```bash
curl -v -k -d '{"data_persistence":"aof"}' \
-u '<USERNAME>:<PASSWORD>' \
-H "Content-Type: application/json"
-X PUT https://<CLUSTER_FQDN>:9443/v1/bdbs/<BDB_UID>
curl -v -k -d '{"aof_policy":"appendfsync-every-sec"}' \
-u '<USERNAME>:<PASSWORD>' \
-H "Content-Type: application/json" \
-X PUT https://<CLUSTER_FQDN>:9443/v1/bdbs/<BDB_UID>
```

- **Ensure that the RDI database is not clustered.** RDI will not work correctly if the
RDI database is clustered, but it is OK for the target database to be clustered.
{{< embed-md "rdi-db-reqs.md" >}}

{{< note >}}If you specify `localhost` as the address of the RDI database server during
installation then the connection will fail if the actual IP address changes for the local
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
---
Title: Requirements summary
alwaysopen: false
categories:
- docs
- integrate
- rs
- rdi
description: Requirements and recommendations for RDI installations.
group: di
hideListLinks: false
linkTitle: Requirements summary
summary: Redis Data Integration keeps Redis in sync with the primary database in near
real time.
type: integration
weight: 5
---

The sections below summarize the software and hardware requirements for
an RDI installation.

## Hardware requirements for VM installation

{{< embed-md "rdi-vm-reqs.md" >}}

## OS requirements for VM installation

{{< embed-md "rdi-os-reqs.md" >}}

## Kubernetes/OpenShift supported versions

{{< embed-md "rdi-k8s-reqs.md" >}}

## RDI database requirements

{{< embed-md "rdi-db-reqs.md" >}}

## Supported source databases

{{< embed-md "rdi-supported-source-versions.md" >}}
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ linkTitle: Upgrade
summary: Redis Data Integration keeps Redis in sync with the primary database in near
real time.
type: integration
weight: 3
weight: 30
---

## Upgrading a VM installation
Expand Down
42 changes: 2 additions & 40 deletions content/integrate/redis-data-integration/quick-start-guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,8 +18,7 @@ In this tutorial you will learn how to install RDI and set up a pipeline to inge
- [Redis Insight]({{< relref "/develop/tools/insight" >}})
to edit your pipeline
- A virtual machine (VM) with one of the following operating systems:
- Ubuntu 20.04, 22.04, or 24.04
- RHEL 8 or 9
{{< embed-md "rdi-os-reqs.md" >}}

## Overview

Expand Down Expand Up @@ -56,44 +55,7 @@ run the pipeline.

Use the Redis console to create the RDI database with the following requirements:

- A single-shard database with 125MB RAM is enough for the quickstart but double this
to 250MB if you want to add a replica.
- Provide the installation with the required RDI database details.
- Set the database's
[eviction policy]({{< relref "/operate/rs/databases/memory-performance/eviction-policy" >}}) to `noeviction`. Note that you can't set this using
[`rladmin`]({{< relref "/operate/rs/references/cli-utilities/rladmin" >}}),
so you must either do it using the admin UI or with the following
[REST API]({{< relref "/operate/rs/references/rest-api" >}})
command:

```bash
curl -v -k -d '{"eviction_policy": "noeviction"}' \
-u '<USERNAME>:<PASSWORD>' \
-H "Content-Type: application/json" \
-X PUT https://<CLUSTER_FQDN>:9443/v1/bdbs/<BDB_UID>
```

- Set the database's
[data persistence]({{< relref "/operate/rs/databases/configure/database-persistence" >}})
to AOF - fsync every 1 sec. Note that you can't set this using
[`rladmin`]({{< relref "/operate/rs/references/cli-utilities/rladmin" >}}),
so you must either do it using the admin UI or with the following
[REST API]({{< relref "/operate/rs/references/rest-api" >}})
commands:

```bash
curl -v -k -d '{"data_persistence":"aof"}' \
-u '<USERNAME>:<PASSWORD>' \
-H "Content-Type: application/json"
-X PUT https://<CLUSTER_FQDN>:9443/v1/bdbs/<BDB_UID>
curl -v -k -d '{"aof_policy":"appendfsync-every-sec"}' \
-u '<USERNAME>:<PASSWORD>' \
-H "Content-Type: application/json" \
-X PUT https://<CLUSTER_FQDN>:9443/v1/bdbs/<BDB_UID>
```

- **Ensure that the RDI database is not clustered.** RDI will not work correctly if the
RDI database is clustered, but it is OK for the target database to be clustered.
{{< embed-md "rdi-db-reqs.md" >}}

### Prepare the pipeline

Expand Down