AOF file and RDB file as the source of Redis Mass insertion

[hwware]

The problem/use-case that the feature addresses

I found some aof file and rdb file can not import into Redis as Mass Insertion.

case 1: When I add aof-use-rdb-preamble yes in the Redis.conf and run Redis server sometime,
Redis will save 188242 key/value pair, then I stop the Redis server.

       Later, when I run the command:  redis-cli -h ipaddress -p portNumber --pipe   XXX.aof
       and want to import the 188242 key/value pair into Redis, I got following error:

        I am not sure if the error is due to the aof-use-rdb-preamble yes config parameter or because too many key/value pairs 
        in the aof file,

case 2: Now it looks like Redis do not supprt rdb file as the source of Mass insertion, if we could support this feature
in the future version?

Thanks

[oranagra]

Doing this would require for redis-cli to parse the RDB file and compose RESTORE commands.
I think there are plenty of tools that do that already.
Maybe it would be nice for redis-cli to do that too (although we probably won't do that very efficiently, i.e. pipeline and multiple clients, etc).
In order to do that we need to either build a basic RDB parser inside redis-cli, at least the global structure, not the keys / types themselves (not a wise idea), or better yet, change rdb.c so that it can be compiled as a library and serve other applications that want to parse RDB files.
We do have a plan do do that (librdb) in the future, but not in the next version.
@yoav-steinberg FYI.

[leonchen83]

hi @hwware
there are some external tools can do this job. first install redis-rdb-cli
step 1. split aof-use-rdb-preamble file to rdb file and aof file

rcut -s ./aof-use-rdb-preamble.aof -r ./dump.rdb -a ./appendonly.aof

step 2. convert rdb file to dump format

rct -f dump -s ./dump.rdb -o ./dump.aof -r

step 3. use redis-cli to do mass insertion

cat ./dump.aof | /redis/src/redis-cli -p 6379 --pipe
cat ./appendonly.aof | /redis/src/redis-cli -p 6379 --pipe

[hwware]

Thanks, I will take a look and try it.