Thanks to @PhoneLi for reporting.
The code freed a reply object that was never created, resulting in a segfault every time randomkey returned a key that was deleted before we queried it for size.
When no encoding is possible, at least try to reallocate the sds string with one that does not waste memory (with free space at the end of the buffer) when the string is large enough.
We are sure that a string that is longer than 21 chars cannot be represented by a 64 bit signed integer, as -(2^64) is 21 chars: strlen(-18446744073709551616) => 21
The call to sdsMakeRoomFor() did not accounted for the amount of data already present in the query buffer, resulting into over-allocation.
This reverts commit 19de8e4.
Also a warning was suppressed by including unistd.h in redisassert.h (needed for _exit()).
By using redisassert.h version of assert() you get stack traces in the log instead of a process disappearing on assertions.
The previous hashing used the trivial algorithm of xoring the integers together. This is not optimal as it is very likely that different hash table setups will hash the same, for instance an hash table at the start of the rehashing process, and at the end, will have the same fingerprint. Now we hash N integers in a smarter way, by summing every integer to the previous hash, and taking the integer hashing again (see the code for further details). This way it is a lot less likely that we get a collision. Moreover this way of hashing explicitly protects from the same set of integers in a different order to hash to the same number. This commit is related to issue #1240.
Related to issue #1240.
This commit does mainly two things: 1) It fixes zunionInterGenericCommand() by removing mass-initialization of all the iterators used, so that we don't violate the unsafe iterator API of dictionaries. This fixes issue #1240. 2) Since the zui* APIs required the allocator to be initialized in the zsetopsrc structure in order to use non-iterator related APIs, this commit fixes this strict requirement by accessing objects directly via the op->subject->ptr pointer we have to the object.
dict.c allows the user to create unsafe iterators, that are iterators that will not touch the dictionary data structure in any way, preventing copy on write, but at the same time are limited in their usage. The limitation is that when itearting with an unsafe iterator, no call to other dictionary functions must be done inside the iteration loop, otherwise the dictionary may be incrementally rehashed resulting into missing elements in the set of the elements returned by the iterator. However after introducing this kind of iterators a number of bugs were found due to misuses of the API, and we are still finding bugs about this issue. The bugs are not trivial to track because the effect is just missing elements during the iteartion. This commit introduces auto-detection of the API misuse. The idea is that an unsafe iterator has a contract: from initialization to the release of the iterator the dictionary should not change. So we take a fingerprint of the dictionary state, xoring a few important dict properties when the unsafe iteartor is initialized. We later check when the iterator is released if the fingerprint is still the same. If it is not, we found a misuse of the iterator, as not allowed API calls changed the internal state of the dictionary. This code was checked against a real bug, issue #1240. This is what Redis prints (aborting) when a misuse is detected: Assertion failed: (iter->fingerprint == dictFingerprint(iter->d)), function dictReleaseIterator, file dict.c, line 587.
Before this commit redis-benchmark supported random argumetns in the form of :rand:000000000000. In every string of that form, the zeros were replaced with a random number of 12 digits at every command invocation. However this was far from perfect as did not allowed to generate simply random numbers as arguments, there was always the :rand: prefix. Now instead every argument in the form __rand_int__ is replaced with a 12 digits number. Note that "__rand_int__" is 12 characters itself. In order to implement the new semantic, it was needed to change a few thigns in the internals of redis-benchmark, as new clients are created cloning old clients, so without a stable prefix such as ":rand:" the old way of cloning the client was no longer able to understand, from the old command line, what was the position of the random strings to substitute. Now instead a client structure is passed as a reference for cloning, so that we can directly clone the offsets inside the command line.
This change was profiler-driven, but the actual effect is hard to measure in real-world redis benchmark runs.