[Feature request] Replicate on DB or Key level

[ieslv]

Is your feature request related to a problem? Please describe.
In our setup we have different clusters where we can use Multi-master setup to replicate the data. In fact we need only one DB to be replicated but as we know it is not possible.
On the DB we are interested traffic is very low, but when we replicate all the data it generates a significant unneeded(for our just one DB) traffic for replication and also we should have same memory available on our cluster like on the source cluster, to be able to store the data we replicates. Sometimes we get OOM on the node just because it replicates all databases from the primary master which is high loaded.

Describe the solution you'd like
We should be able to specify DB list or DB/Key list to be replicated from the master

Describe alternatives you've considered
Right now we replicate all the data. As a workaround we considered to use different cluster with just one DB and it also involves additional load balancer.

[JohnSully]

This is interesting, I don't think filtering by key is a good idea, but perhaps by database is something we could look at.

[e271828-]

This is interesting, I don't think filtering by key is a good idea, but perhaps by database is something we could look at.

Big +1 to nominating DBs for replication. This greatly simplifies some use cases.

Example: N nodes, local cache per node for ephemeral requests; global cache for small subset of data promoted from local cache.

[yegors]

Salvatore did say this: https://groups.google.com/g/redis-db/c/vS5wX8X4Cjg/m/8ounBXitG4sJ

Ability to exclude keyspaces from being replicated would be a good idea, if it doesn't introduce a lot of moving parts.

[JohnSully]

I've done an initial implementation in the replication_filter branch. It works like this:

Add a line to your config file:
replication-db-disable "1 2 3 5-7"

The syntax is a string where you either explicitly list individual database numbers or ranges of databases which should not propagate. Note that this design may change the branch is for demonstration purposes and to see if we're on the right track with the implementation.

[e271828-]

I've done an initial implementation in the replication_filter branch. It works like this:

Add a line to your config file:
replication-db-disable "1 2 3 5-7"

The syntax is a string where you either explicitly list individual database numbers or ranges of databases which should not propagate. Note that this design may change the branch is for demonstration purposes and to see if we're on the right track with the implementation.

Looks reasonable. You might consider adding the inverse as well (replication-db-enable "1") since for many use cases there are more local than global dbs, and specifying explicitly is a good way to prevent accidental replication.

[JohnSully]

I’m thinking of adding a requirement that all nodes (with active rep) must have the same filter. With this design we’ll still receive queries for these databases and execute those. We just won’t transmit any of our own.

[e271828-]

I’m thinking of adding a requirement that all nodes (with active rep) must have the same filter. With this design we’ll still receive queries for these databases and execute those. We just won’t transmit any of our own.

Makes sense.