Skip to content
Hot sync two Redis servers using dumps.
Go Shell
Branch: master
Clone or download
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
bin readme: edits Jul 5, 2019
cmd/rump gitignore: fix conflict with cmd Jul 5, 2019
docs remove dead link in README Jul 9, 2019
infra infra: update Rump version in Dockerfile.example Jul 9, 2019
pkg cleanup redis test Jul 9, 2019
.gitignore Update .gitignore Jul 8, 2019
LICENSE Add license. Nov 24, 2016
cloudbuild.yml infra: add ci setup Jul 5, 2019
docker-compose.yml infra: add ci setup Jul 5, 2019
go.mod version 0.9 beta Jul 5, 2019
go.sum formatting + typos + minor fixes Jul 5, 2019

README.md

Rump

Go Report Card GoDoc CI

Hot sync two Redis databases using dumps.

Why

There's no easy way to sync data from an AWS ElastiCache or GCP MemoryStore Redis cluster; the standard commands BGSAVE and SLAVEOF are blocked.

Rump is able to live sync Redis databases across cloud providers by only using SCAN, DUMP and RESTORE.

It's used at Sticker Mule to keep staging and development environments in sync with the production AWS/GCP Redis clusters.

Examples

# Sync local Redis DB 1 to DB 2.
$ rump -from redis://127.0.0.1:6379/1 -to redis://127.0.0.1:6379/2

# Sync ElastiCache cluster to local.
$ rump -from redis://production.cache.amazonaws.com:6379/1 -to redis://127.0.0.1:6379/1

# Sync protected ElastiCache via EC2 port forwarding.
$ ssh -L 6969:production.cache.amazonaws.com:6379 -N username@xxx.xxx.xxx.xxx &
$ rump -from redis://127.0.0.1:6969/1 -to redis://127.0.0.1:6379/1

# Dump GCP MemoryStore to file.
$ rump -from redis://10.0.20.2:6379/1 -to /backup/memorystore.rump

# Restore backup to ElastiCache.
$ rump -from /backup/memorystore.rump -to redis://production.cache.amazonaws.com:6379/1

# Sync with verbose mode disabled.
$ rump -from redis://127.0.0.1:6379/1 -to redis://127.0.0.1:6379/2 -silent

# Sync with TTLs.
$ rump -from redis://127.0.0.1:6379/1 -to redis://127.0.0.1:6379/2 -ttl

Features

  • Uses SCAN instead of KEYS to avoid DoS servers.
  • Doesn't use any temp file.
  • Can sync any key type.
  • Can optionally sync TTLs.
  • Uses buffered channels to optimize slow source servers.
  • Uses implicit pipelining to minimize network roundtrips.
  • Supports two-step sync: dump source to file, restore file to database.
  • Supports Redis URIs with auth.
  • Offers the same guarantees of the SCAN command.

Demo

asciicast

Development

# requirements: docker, docker-compose (dc)
dc up # watch/run Rump tests and a Redis container
dc run --rm rump sh # get shell on Rump container
dc run --rm redis sh; redis-cli -h redis # get Redis console

Install

Binaries can be found on the releases page.

curl -SL https://github.com/stickermule/rump/releases/download/1.0.0/rump-1.0.0-linux-amd64 -o rump \
  && chmod +x rump;
./rump

You can run rump in a container following the Dockerfile.example.

Mentions

Maintainers

nixtrace

Collaboration & License

You can’t perform that action at this time.