Add option to disable incremental sync in kvrocks2redis

[QQxiaoyuyu]

Search before asking

• I had searched in the issues and found no similar issues.

Version

2.8

Minimal reproduce step

Looking downwards

What did you expect to see?

Looking downwards

What did you see instead?

Looking downwards

Anything Else?

Currently, kvrocks2redis will synchronize incremental data by default, but if the increment is too large, many data files will be written, we can consider adding a configuration that does not write incremental data, and only does full synchronization.

Are you willing to submit a PR?

• I'm willing to submit a PR!

[zjregee]

I'd like to try this.

[git-hulk]

@zjregee Thank you!

[zjregee]

Hello, @git-hulk. I'm running into some issues while trying to resolve this issue and hope to get your advice.

kvrocks2redis currently updates the data into a new Redis step by step by writing an aof file for all the data in the kvrocks, which is not reasonable when the kvrocks contains a large amount of data. In order to solve this problem, we can first create an rdb file, synchronize a large amount of data at once through the rdb file, and continue to synchronize some unsynchronized data through the original incremental synchronization method.

But I have doubts about how to pass the rdb file to the new Redis. Redis does not seem to support receiving rdb file directly through a certain command. If the new Redis is synchronized through slaveof, this new master-slave relationship does not seem to meet this requirement.

How should I solve this problem? In order for the Redis to receive the rdb file, I seem to need to send the rdb file to the data directory related to the new Redis and restart the Redis. I am not sure if this method is reasonable.

[git-hulk]

@zjregee Sorry for not getting back to you sooner.

How should I solve this problem? In order for the Redis to receive the rdb file, I seem to need to send the rdb file to the data directory related to the new Redis and restart the Redis. I am not sure if this method is reasonable.

Redis only supports loading the RDB from the replication for now, and it should be not good to require users to restart the Redis server for syncing the RDB.

kvrocks2redis currently updates the data into a new Redis step by step by writing an aof file for all the data in the kvrocks, which is not reasonable when the kvrocks contains a large amount of data.

For this scenario, perhaps we can send the key value to the target node directly instead of writing AOF.