Internal error in RDB reading function at rdb.c:1546 -> Duplicate keys detected

[Venorcis]

We were surprised today with the error in the title when trying to sync a new replica to its master. Upon this error, the whole replica even crashed (which I don't think is appropriate handling of this issue).

The exact same error appeared when running redis-check-rdb on the master's RDB file, only luckily in that output there was some additional info, i.e. the key of the database entry that caused the error and the fact that it happened whilst 'reading type 4 (hash-hashtable)'.

It turned out that for some unknown reason (IMO a Redis bug), indeed a key existed twice in a hash, as visible in this image:

Running 'HDEL key "key"' removed the first entry (with value "msg.save", the correct entry) and luckily allowed our sync to work again. However, under no circumstances can I now delete or edit the second entry; only in an 'HGETALL key' does it show up (i.e. HDEL and/or HGET do not work; HSET actually creates a new entry with key "key").

[itamarhaber]

Hello @Venorcis

Thanks for reporting this - which version of Redis are you experiencing this issue with?

P.S. epic issue number.

[Venorcis]

This is on latest (5.0.5).

P.S. didn't notice, awesome indeed!

[haochensust]

Is this problem solved? I also had this problem in version 5.0.11.