Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
---
Title: Release notes
aliases: null
alwaysopen: false
categories:
- docs
- integrate
- rs
- rdi
description: null
group: di
hideListLinks: true
linkTitle: Release notes
summary: Redis Data Integration keeps Redis in sync with the primary database in near
real time.
type: integration
weight: 100
---

Here's what changed recently in Redis Data Integration (RDI):

{{< table-children columnNames="Version&nbsp;(Release&nbsp;date)&nbsp;,Major changes" columnSources="LinkTitle,Description" enableLinks="LinkTitle" >}}
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
---
Title: Redis Data Integration release notes 1.0 (June 2024)
alwaysopen: false
categories:
- docs
- operate
- rs
description: Changes to the processing mode. Simple installation. Silent installation. Pipeline orchestration. Logging. Monitoring. High availability mechanism.
linkTitle: 1.0 (June 2024)
toc: 'true'
weight: 1000
---

This is the first General Availability version of Redis Data Integration (RDI).

RDI’s mission is to help Redis customers sync Redis Enterprise with live data from their slow disk-based databases to:

- Meet the required speed and scale of read queries and provide an excellent and predictable user experience.
- Save resources and time when building pipelines and coding data transformations.
- Reduce the total cost of ownership by saving money on expensive database read replicas.

RDI keeps the Redis cache up to date with changes in the primary database, using a
[_Change Data Capture (CDC)_](https://en.wikipedia.org/wiki/Change_data_capture) mechanism.
It also lets you _transform_ the data from relational tables into convenient
and fast data structures that match your app's requirements. You specify the
transformations using a configuration system, so no coding is required.

## Headlines

- Changes to the processing mode: The preview versions of RDI processed data inside the Redis Enterprise database using the shard CPU. The GA version moves the processing of data outside the cluster. RDI is now deployed on VMs or on Kubernetes (K8s).
- Simple installation: RDI ships with all of its dependencies. A simple interactive installer provides a streamlined process that takes a few minutes.
- Silent installation: RDI can be installed by software using a script and an input file.
- Pipeline orchestration: The preview versions of RDI required you to manually install and configure the Debezium server. In this version, we add support for source database configuration to the pipeline configuration and orchestration of all pipeline components including the Debezium server (RDI Collector).
- Logging: All RDI component logs are now shipped to a central folder and get rotated by RDI's logging mechanism.
- Monitoring: RDI comes with two Prometheus exporters, one For Debezium Server and one for RDI's pipeline data processing.
- High availability mechanism: The preview versions of RDI used an external clustering dependency to provide active-passive deployment of the Debezium server. The GA version has a Redis-based built-in fail-over mechanism between an active VM and a passive VM. Kubernetes deployments rely on K8s probes that are included in RDI components.

## Limitations

- RDI can write data to a Redis Active-Active database. However, it doesn't support writing data to two or more Active-Active replicas. Writing data from RDI to several Active-Active replicas could easily harm data integrity as RDI is not synchronous with the source database commits.
- RDI write-behind (which is currently in preview) should not be used on the same data set that RDI ingest is writing to Redis. This would either cause an infinite loop or would harm the data integrity, since both ingest and write-behind are asynchronous, eventually-consistent processes.
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
---
Title: Redis Data Integration release notes 1.2 (June 2024)
alwaysopen: false
categories:
- docs
- operate
- rs
description: API server with a set of APIs to support Redis Insight with creating, testing, deploying & monitoring RDI pipelines. RDI Installer verifies all components are running at the end of installation.
linkTitle: 1.2 (June 2024)
toc: 'true'
weight: 999
---

> This minor release replaces the 1.0 release.

RDI’s mission is to help Redis customers sync Redis Enterprise with live data from their slow disk-based databases to:

- Meet the required speed and scale of read queries and provide an excellent and predictable user experience.
- Save resources and time when building pipelines and coding data transformations.
- Reduce the total cost of ownership by saving money on expensive database read replicas.

RDI keeps the Redis cache up to date with changes in the primary database, using a
[_Change Data Capture (CDC)_](https://en.wikipedia.org/wiki/Change_data_capture) mechanism.
It also lets you _transform_ the data from relational tables into convenient
and fast data structures that match your app's requirements. You specify the
transformations using a configuration system, so no coding is required.

## Headlines

- API server with a set of APIs to support Redis Insight with creating, testing, deploying & monitoring RDI pipelines.
- RDI Installer verifies all components are running at the end of installation.

## Fixed Bugs

- Support for source database TLS & mTLS was fixed. Certificate and file names used with the redis-di set-secret command can have any file names
- Mismatch between Reloader version provided and the one required.

## Limitations

- RDI can write data to a Redis Active-Active database. However, it doesn't support writing data to two or more Active-Active replicas. Writing data from RDI to several Active-Active replicas could easily harm data integrity as RDI is not synchronous with the source database commits.
- RDI write-behind (which is currently in preview) should not be used on the same data set that RDI ingest is writing to Redis. This would either cause an infinite loop or would harm the data integrity, since both ingest and write-behind are asynchronous, eventually-consistent processes.