Skip to content


Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

We’re showing branches in this repository, but you can also compare across forks.

base fork: antirez/redis
head fork: antirez/redis
This comparison is big! We're only showing the most recent 250 commits
Commits on Dec 11, 2014
Salvatore Sanfilippo Merge pull request #2094 from mattsta/nosort
Fix zero-ordering SORT when called against lists
Salvatore Sanfilippo SORT: Don't sort Set elements if not needed.
Related to #2094.
Matt Stancliff mattsta Add centralized IP/Peer formatting functions
This stops us from needing to manually check against ":" to
add brackets around IPv6 addresses everywhere.
Matt Stancliff mattsta Cleanup all IP formatting code
Instead of manually checking for strchr(n,':') everywhere,
we can use our new centralized IP formatting functions.
Matt Stancliff mattsta Add maxmemory_policy to INFO output
Also refactors getting human string values from
the defined value in `server.maxmemory_policy`
into a common function.
Matt Stancliff mattsta Add cached total system memory to INFO output
There is no standard cross-platform way of obtaining
system memory info, but I found a useful function
convering all common platforms.  I removed support
for uncommon Redis platforms (windows, AIX) and left
others intact.

For more info, see:

The system memory info is cached on startup, but some systems
may be able to change the amount of memory visible to Redis
at runtime if Redis is deployed in a VM or container.

Also see #1820
Matt Stancliff mattsta Sentinel: Improve INFO command behavior
  - Return empty string if asking for non-existing section (INFO foo)
  - Fix potential memory leak (caused by sdsempty() then returned if >2 args)
  - Clean up argument parsing
  - Allow "all" as valid section (same as "default" or zero args currently)
  - Move strcasecmp to end of evaluation chain in conditionals

Also, since we're C99, I moved some variable declarations to be closer
to where they are actually used (saves us from needing to free an empty info
if detect argument errors up front).

Closes #1915
Closes #1966
Matt Stancliff mattsta Bitops: Stop overallocating storage space on set
Previously the string was created empty then re-sized
to fit the offset, but sds resize causes the sds to
over-allocate by at least 1 MB (which is a lot when
you are operating at bit-level access).

This also improves the speed of initial sets by 2% to 6%
based on quick testing.

Patch logic provided by @oranagra

Fixes #1918
Salvatore Sanfilippo AnetFormatIP(): renamed, commented, now sticks to IP:port format.
A few code style changes + consistent format: not nice for humans but
better for parsers.
Salvatore Sanfilippo sdsformatip() removed.
Specialized single-use function. Not the best match for sds.c btw.
Also genClientPeerId() is no longer static: we need symbols.
Commits on Dec 12, 2014
clark.kang remove_warning in db.c 7ef1239
Salvatore Sanfilippo Merge pull request #2209 from charsyam/feature/remove_warning
remove_warning in db.c
Salvatore Sanfilippo Lua cmsgpack lib updated to latest version.
It fixes a bad bug that crashes the server in certain conditions
as shown in issue #2210.
Salvatore Sanfilippo Merge branch 'unstable' of into unstable f2858dc
Rhommel Lamas rhoml Add symlink to redis-sentinel during make install 7b72dbf
Commits on Dec 13, 2014
Salvatore Sanfilippo List of commands flagged as admin commands modified.
The old list did not made much sense... and the flag is currently not
used at all, so no side effects.
Salvatore Sanfilippo Don't log admin commands in MONITOR.
Otherwise there are security risks, especially when providing Redis as a
service, the user may "sniff" for admin commands renamed to an
unguessable string via rename-command in redis.conf.
Salvatore Sanfilippo CONTRIBUTING updated. c147cd8
Commits on Dec 14, 2014
Alon Diamant advance512 Added <count> parameter to SPOP:
spopCommand() now runs spopWithCountCommand() in case the <count> param is found.
Added intsetRandomMembers() to Intset: Copies N random members from the set into inputted 'values' array. Uses either the Knuth or Floyd sample algos depending on ratio count/size.
Added setTypeRandomElements() to SET type: Returns a number of random elements from a non empty set. This is a version of setTypeRandomElement() that is modified in order to return multiple entries, using dictGetRandomKeys() and intsetRandomMembers().
Added tests for SPOP with <count>: unit/type/set, unit/scripting, integration/aof
Cleaned up code a bit to match with required Redis coding style
Mihir Joshi mihirvj Stricter options for SET command
- As per Antirez's suggestion, this commit raises an error when mutually
exclusive options are provided. Duplicate options are allowed.
Commits on Dec 15, 2014
Salvatore Sanfilippo Make sure buffer is enough in clusterSendPing(). 90c7d8c
Commits on Dec 16, 2014
Salvatore Sanfilippo Fix comment in clusterHandleSlaveFailover(). 1aef29e
Commits on Dec 17, 2014
Rhommel Lamas rhoml Add redis-sentinel symlink to a relative path 24f4c01
Salvatore Sanfilippo Merge pull request #2221 from rhoml/rhoml/fix-redis-sentinel-ln
Fix redis-sentinel symlink to a relative path
Matt Stancliff mattsta Cluster: Notify user on accept error
If we woke up to accept a connection, but we can't
accept it, inform the user of the error going on
with their networking.

(The previous message was the same for success or error!)
Salvatore Sanfilippo Merge branch 'improve/INFO/add-system-memory-size' of git://…
…m/mattsta/redis into unstable
Salvatore Sanfilippo getMemorySize() moved into zmalloc.c with other low level mem utils.
See issue #2218.
Salvatore Sanfilippo Add info as requested by the original getMemorySize() license. 9a63543
Salvatore Sanfilippo Merge pull request #1582 from mattsta/add-error-mention-to-error-cond…

Cluster: Update accept error to mention error
Salvatore Sanfilippo Merge pull request #2215 from advance512/spopWithCount
SPOP optional count argument. (issue #1793, supersedes pull request #1803)
Commits on Dec 18, 2014
Alon Diamant advance512 Fix: case when SPOP with count>MAXINT, setTypeRandomElements() will g…
…et negative count argument due to signed/unsigned mismatch.

setTypeRandomElements() now returns unsigned long, and also uses unsigned long for anything related to count of members.
spopWithCountCommand() now uses unsigned long elements_returned instead of int, for values returned from setTypeRandomElements()
Salvatore Sanfilippo Merge pull request #2224 from advance512/spopWithCount
Fix: case when SPOP with count>MAXINT, setTypeRandomElements() will get ...
Commits on Dec 19, 2014
Salvatore Sanfilippo Fix adjustOpenFilesLimit() logging to match real state.
Fixes issue #2225.
Commits on Dec 20, 2014
Matt Stancliff mattsta Improve redis-trib replica assignment
This tiny bit of code has gone through so many revisions.  Hopefully
it's more correct now.

Fixes #2204
Commits on Dec 21, 2014
Alon Diamant advance512 Following @mattsta's friendly review:
  1. memory leak in t_set.c has been fixed
  2. end-of-line spaces has been removed (from all over the place)
  3. for loops have been ordered up to match existing Redis style (less weird)
  4. comments format has been fixed (added * in the beggining of every comment line)
Alon Diamant advance512 Fixed memory leaks in rdbSaveToSlavesSockets() 14b04c0
Commits on Dec 22, 2014
Salvatore Sanfilippo Merge pull request #2229 from advance512/spopWithCount
Memory leak fixes (+ code style fixes)
Commits on Dec 23, 2014
Matt Stancliff mattsta Add 'age' value to SENTINEL INFO-CACHE 32bba43
Salvatore Sanfilippo Merge pull request #2234 from mattsta/feature/sentinel-info-cache/age
Add 'age' value to SENTINEL INFO-CACHE
Salvatore Sanfilippo Merge pull request #2227 from mattsta/fix/trib/assignment/master-replica
Improve redis-trib replica assignment
Salvatore Sanfilippo INFO loading stats: three fixes.
1. Server unxtime may remain not updated while loading AOF, so ETA is
not updated correctly.

2. Number of processed byte was not initialized.

3. Possible division by zero condition (likely cause of issue #1932).
Salvatore Sanfilippo Merge branch 'unstable' of into unstable 9e718a1
Matt Stancliff mattsta Add addReplyBulkSds() function
Refactor a common pattern into one function so we don't
end up with copy/paste programming.
Matt Stancliff mattsta Add DEBUG JEMALLOC INFO
Uses jemalloc function malloc_stats_print() to return
stats about what jemalloc has allocated internally.
Matt Stancliff mattsta Remove ziplist compiler warnings
Only happen when compiled with the test define.
Matt Stancliff mattsta Allow all code tests to run using Redis args
Previously, many files had individual main() functions for testing,
but each required being compiled with their own testing flags.
That gets difficult when you have 8 different flags you need
to set just to run all tests (plus, some test files required
other files to be compiled aaginst them, and it seems some didn't
build at all without including the rest of Redis).

Now all individual test main() funcions are renamed to a test
function for the file itself and one global REDIS_TEST define enables
testing across the entire codebase.

Tests can now be run with:
  - `./redis-server test <test>`

  e.g. ./redis-server test ziplist

If REDIS_TEST is not defined, then no tests get included and no
tests are included in the final redis-server binary.
Matt Stancliff mattsta Add simple ll2string() tests 9b34367
Matt Stancliff mattsta Allow forcing non-jemalloc build d01d4dd
Matt Stancliff mattsta Fix how zipEntry returns values
zipEntry was returning a struct, but that caused some
problems with tests under 32 bit builds.

The tests run better if we operate on structs allocated in the
caller without worrying about copying on return.
Matt Stancliff mattsta Fix ziplistDeleteRange index parameter
It's valid to delete from negative offsets, so we *don't*
want unsigned arguments here.
Matt Stancliff mattsta Fix ziplist test for pop()
The previous test wasn't returning the new ziplist, so the test
was invalid.  Now the test works properly.

These problems were simultaenously discovered in #2154 and that
PR also had an additional fix we included here.
Matt Stancliff mattsta Cleanup ziplist valgrind warnings
Valgrind can't detect 'memset' initializes things, so let's
statically initialize them to remove some unnecessary warnings.
Matt Stancliff mattsta Fix three simple clang analyzer warnings d956d80
Commits on Jan 02, 2015
Matt Stancliff mattsta Add quicklist implementation
This replaces individual ziplist vs. linkedlist representations
for Redis list operations.

Big thanks for all the reviews and feedback from everybody in
Matt Stancliff mattsta Add ziplistMerge()
This started out as #2158 by sunheehnus, but I kept rewriting it
until I could understand things more easily and get a few more
correctness guarantees out of the readability flow.

The original commit created and returned a new ziplist with the contents of
both input ziplists, but I prefer to grow one of the input ziplists
and destroy the other one.

So, instead of malloc+copy as in #2158, the merge now reallocs one of
the existing ziplists and copies the other ziplist into the new space.

Also added merge test cases to ziplistTest()
Matt Stancliff mattsta Free ziplist test lists during tests
Freeing our test lists helps keep valgrind output clean
Matt Stancliff mattsta redis-benchmark: Add RPUSH and RPOP tests 60a9418
Matt Stancliff mattsta Add adaptive quicklist fill factor
Fill factor now has two options:
  - negative (1-5) for size-based ziplist filling
  - positive for length-based ziplist filling with implicit size cap.

Negative offsets define ziplist size limits of:
  -1: 4k
  -2: 8k
  -3: 16k
  -4: 32k
  -5: 64k

Positive offsets now automatically limit their max size to 8k.  Any
elements larger than 8k will be in individual nodes.

Positive ziplist fill factors will keep adding elements
to a ziplist until one of:
  - ziplist has FILL number of elements
    - or -
  - ziplist grows above our ziplist max size (currently 8k)

When using positive fill factors, if you insert a large
element (over 8k), that element will automatically allocate
an individual quicklist node with one element and no other elements will be
in the same ziplist inside that quicklist node.

When using negative fill factors, elements up to the size
limit can be added to one quicklist node.  If an element
is added larger than the max ziplist size, that element
will be allocated an individual ziplist in a new quicklist node.

Tests also updated to start testing at fill factor -5.
Matt Stancliff mattsta Add sdsnative()
Use the existing memory space for an SDS to convert it to a regular
character buffer so we don't need to allocate duplicate space just
to extract a usable buffer for native operations.
Matt Stancliff mattsta Convert RDB ziplist loading to sdsnative()
This saves us an unnecessary zmalloc, memcpy, and two frees.
Matt Stancliff mattsta Convert quicklist RDB to store ziplist nodes
Turns out it's a huge improvement during save/reload/migrate/restore
because, with compression enabled, we're compressing 4k or 8k
chunks of data consisting of multiple elements in one ziplist
instead of compressing series of smaller individual elements.
Matt Stancliff mattsta Increase test size for migrating large values
Previously, the old test ran 5,000 loops and used about 500k.

With quicklist, storing those same 5,000 loops takes up 24k, so the
"large value check" failed!

This increases the test to 20,000 loops which makes the object dump 96k.
Matt Stancliff mattsta Remove malloc failure checks
We trust zmalloc to kill the whole process on memory failure
Matt Stancliff mattsta Add quicklist info to DEBUG OBJECT
Added field 'ql_nodes' and 'ql_avg_per_node'.

ql_nodes is the number of quicklist nodes in the quicklist.
ql_avg_node is the average fill level in each quicklist node. (LLEN / QL_NODES)

Sample output:> DEBUG object b
Value at:0x7fa42bf2fed0 refcount:1 encoding:quicklist serializedlength:18489 lru:8983768 lru_seconds_idle:3 ql_nodes:430 ql_avg_per_node:511.73> llen b
(integer) 220044
Matt Stancliff mattsta Allow compression of interior quicklist nodes
Let user set how many nodes to *not* compress.

We can specify a compression "depth" of how many nodes
to leave uncompressed on each end of the quicklist.

Depth 0 = disable compression.
Depth 1 = only leave head/tail uncompressed.
  - (read as: "skip 1 node on each end of the list before compressing")
Depth 2 = leave head, head->next, tail->prev, tail uncompressed.
  - ("skip 2 nodes on each end of the list before compressing")
Depth 3 = Depth 2 + head->next->next + tail->prev->prev
  - ("skip 3 nodes...")

This also:
  - updates RDB storage to use native quicklist compression (if node is
    already compressed) instead of uncompressing, generating the RDB string,
    then re-compressing the quicklist node.
  - internalizes the "fill" parameter for the quicklist so we don't
    need to pass it to _every_ function.  Now it's just a property of
    the list.
  - allows a runtime-configurable compression option, so we can
    expose a compresion parameter in the configuration file if people
    want to trade slight request-per-second performance for up to 90%+
    memory savings in some situations.
  - updates the quicklist tests to do multiple passes: 200k+ tests now.
Matt Stancliff mattsta Cleanup quicklist style
Small fixes due to a new version of clang-format (it's less
crazy than the older version).
Matt Stancliff mattsta Add branch prediction hints to quicklist
Actually makes a noticeable difference.

Branch hints were selected based on profiler hotspots.
Matt Stancliff mattsta Config: Add quicklist, remove old list options
This removes:
  - list-max-ziplist-entries
  - list-max-ziplist-value

This adds:
  - list-max-ziplist-size
  - list-compress-depth

Also updates config file with new sections and updates
tests to use quicklist settings instead of old list settings.
Matt Stancliff mattsta Add more quicklist info to DEBUG OBJECT
Adds: ql_compressed (boolean, 1 if compression enabled for list, 0
Adds: ql_uncompressed_size (actual uncompressed size of all quicklistNodes)
Adds: ql_ziplist_max (quicklist max ziplist fill factor)

Compression ratio of the list is then ql_uncompressed_size / serializedlength

We report ql_uncompressed_size for all quicklists because serializedlength
is a _compressed_ representation anyway.

Sample output from a large list:> llen abc
(integer) 38370061> debug object abc
Value at:0x7ff97b51d140 refcount:1 encoding:quicklist serializedlength:19878335 lru:9718164 lru_seconds_idle:5 ql_nodes:21945 ql_avg_node:1748.46 ql_ziplist_max:-2 ql_compressed:0 ql_uncompressed_size:1643187761

The 1.36s result time is because rdbSavedObjectLen() is serializing the
object, not because of any new stats reporting.

If we run DEBUG OBJECT on a compressed list, DEBUG OBJECT takes almost *zero*
time because rdbSavedObjectLen() reuses already-compressed ziplists:> debug object abc
Value at:0x7fe5c5800040 refcount:1 encoding:quicklist serializedlength:19878335 lru:9718109 lru_seconds_idle:5 ql_nodes:21945 ql_avg_node:1748.46 ql_ziplist_max:-2 ql_compressed:1 ql_uncompressed_size:1643187761
Matt Stancliff mattsta Set optional 'static' for Quicklist+Redis
This also defines REDIS_STATIC='' for building everything
inside src/ and everything inside deps/lua/.
Matt Stancliff mattsta Upgrade LZF to 3.6 (2011) from 3.5 (2009) 5870e22
Commits on Jan 08, 2015
Salvatore Sanfilippo Merge pull request #2143 from mattsta/quicklist
Quicklist (linked list + ziplist)
Salvatore Sanfilippo RDB refactored to load plain strings from RDB. 68bc02c
Salvatore Sanfilippo Use RDB_LOAD_PLAIN to load quicklists and encoded types.
Before we needed to create a string object with an embedded SDS, adn
basically duplicate the SDS part into a plain zmalloc() allocation.
Salvatore Sanfilippo sdsnative() removed: New rdb.c API can load native strings. 32b1000
Salvatore Sanfilippo New RDB v7 opcode: RESIZEDB.
The new opcode is an hint about the size of the dataset (keys and number
of expires) we are going to load for a given Redis database inside the
RDB file. Since hash tables are resized accordingly ASAP, useless
rehashing is avoided, speeding up load times significantly, in the order
of ~ 20% or more for larger data sets.

Related issue: #1719
Salvatore Sanfilippo rdbLoad() refactoring to make it simpler to follow. 1a30e7d
Salvatore Sanfilippo RDB AUX fields support.
This commit introduces a new RDB data type called 'aux'. It is used in
order to insert inside an RDB file key-value pairs that may serve
different needs, without breaking backward compatibility when new
informations are embedded inside an RDB file. The contract between Redis
versions is to ignore unknown aux fields when encountered.

Aux fields can be used in order to:

1. Augment the RDB file with info like version of Redis that created the
RDB file, creation time, used memory while the RDB was created, and so
2. Add state about Redis inside the RDB file that we need to reload
later: replication offset, previos master run ID, in order to improve
failovers safety and allow partial resynchronization after a slave
3. Anything that we may want to add to RDB files without breaking the
ability of past versions of Redis to load the file.
Salvatore Sanfilippo A few more AUX info fields added to RDB. 4c0e892
Salvatore Sanfilippo Typo fixed: fiels -> fields in rdbSaveInfoAuxFields().
Thx to @badboy.
Salvatore Sanfilippo Advertise Redis Cluster as experimental in redis.conf. 4235766
Jungtaek Lim HeartSaVioR Explain make distclean which seems not well known 219ab66
Salvatore Sanfilippo Merge pull request #2262 from HeartSaVioR/explain-make-distclean
Explain make distclean which seems not well known
Salvatore Sanfilippo Merge branch 'unstable' of into unstable b45e16e
Salvatore Sanfilippo README section about make distclean reworded / extended. 622c69e
Matt Stancliff mattsta Fix redis-trib cluster create
Under certain conditions the node list wasn't being fully populated
and 'create' would fail trying to call methods on nil objects.
Matt Stancliff mattsta Remove end of line whitespace from redis-trib bf58f8b
Salvatore Sanfilippo Merge pull request #2265 from mattsta/fix/trib/create
Fix redis-trib creation failure
Commits on Jan 09, 2015
Salvatore Sanfilippo Prevent Lua scripts from violating Redis Cluster keyspace access rules.
Before this commit scripts were able to access / create keys outside the
set of hash slots served by the local node.
Salvatore Sanfilippo Cluster test: also write from Lua script in resharding test. f3fd58e
Salvatore Sanfilippo Check for __sun macro in solarisfixes.h, not in includers. 792a941
Salvatore Sanfilippo Add "-lrt" in Makefile for Solaris.
This fix is from @NanXiao, however I was not able to retain authorship
because the Pull Request original repository was removed.
Rebs Guarina rebx Create PID file even if in foreground
Previously, Redis only wrote the pid file if
it was daemonizing, but many times it's useful to have
the pid written out even if you're in the foreground.

Some background for this is:
I usually run redis via daemontools. That entails running
redis-server on the foreground. Given that, I'd also want
redis-server to create a pidfile so other processes (e.g. nagios)
can run checks for that.

Closes #463
Matt Stancliff mattsta Define default pidfile when creating pid
We want pidfile to be NULL on startup so we can detect if the user
set an explicit value versus only using the default value.

Closes #1967
Fixes #2076
Matt Stancliff mattsta Supervise redis processes only if configured
Adds configuration option 'supervised [no | upstart | systemd | auto]'

Also removed 'bzero' from the previous implementation because it's 2015.
(We could actually statically initialize those structs, but clang
throws an invalid warning when we try, so it looks bad even though it
isn't bad.)

Fixes #2264
Matt Stancliff mattsta Remove RDB AUX memory leaks eb7d67a
Matt Stancliff mattsta Improve consistency of INFO MEMORY fields
Adds used_memory_rss_human and used_memory_lua_human to match
all the other fields reporting human-readable memory too.
Matt Stancliff mattsta Add maxmemory limit to INFO MEMORY
Since we have the eviction policy, we should have the memory limit too.
Commits on Jan 12, 2015
Salvatore Sanfilippo Cluster: initialize mf_end.
Can't be initialized by resetManualFailover() since it's actual state
the function uses, so we need to initialize it at startup time. Not
really a bug in practical terms, but showed up into valgrind and is not
technically correct anyway.
Salvatore Sanfilippo Merge pull request #2266 from mattsta/improve/supervised/startup
Three fixes: explicit supervise, pidfile create, remove memory leaks.
Salvatore Sanfilippo Merge branch 'unstable' of into unstable 10007cb
Salvatore Sanfilippo Use REDIS_SUPERVISED_NONE instead of 0. 2c925b0
Salvatore Sanfilippo Revert "Use REDIS_SUPERVISED_NONE instead of 0."
This reverts commit 2c925b0.

Commits on Jan 13, 2015
Salvatore Sanfilippo Suppress valgrind error about write sending uninitialized data.
Valgrind checks that the buffers we transfer via syscalls are all
composed of bytes actually initialized. This is useful, it makes we able
to avoid leaking informations in non initialized parts fo messages
transferred to other hosts. This commit fixes one of such issues.
Salvatore Sanfilippo Cluster: clusterMsgDataGossip structure, explict padding + minor stuff.
Also explicitly set version to 0, add a protocol version define, improve
comments in the gossip structure.

Note that the structure layout is the same after the change, we are just
making the padding explicit with an additional not used 16 bits field.
So this commit is still able to talk with the previous versions of
cluster nodes.
Salvatore Sanfilippo Cluster: fetch my IP even if msg is not MEET for the first time.
In order to avoid that misconfigured cluster nodes at some time may
force an IP update on other nodes, it is required that nodes update
their own address only on MEET messages. However it does not make sense
to do this the first time a node is contacted and yet does not have an
IP, we just risk that myself->ip remains not assigned if there are
messages lost or cluster creation procedures that don't make sure
everybody is targeted by at least one incoming MEET message.

Also fix the logging of the IP switch avoiding the :-1 tail.
Commits on Jan 19, 2015
Matt Stancliff mattsta Improve RDB type correctness
It's possible large objects could be larger than 'int', so let's
upgrade all size counters to ssize_t.

This also fixes rdbSaveObject serialized bytes calculation.
Since entire serializations of data structures can be large,
so we don't want to limit their calculated size to a 32 bit signed max.

This commit increases object size calculation and
cascades the change back up to serializedlength printing.

Before:> debug object hihihi
... encoding:quicklist serializedlength:-2147483559 ...

After:> debug object hihihi
... encoding:quicklist serializedlength:2147483737 ...
Matt Stancliff mattsta Improve networking type correctness
read() and write() return ssize_t (signed long), not int.

For other offsets, we can use the unsigned size_t type instead
of a signed offset (since our replication offsets and buffer
positions are never negative).
Commits on Jan 20, 2015
Salvatore Sanfilippo Panic on recursive calls to luaRedisGenericCommand().
Related to issue #2302.
Salvatore Sanfilippo luaRedisGenericCommand() recursion: just return an error.
Instead of calling redisPanic() to abort the server.

Related to issue #2302.
Salvatore Sanfilippo luaRedisGenericCommand(): log error at WARNING level when re-entered.
Rationale is that when re-entering, it is likely due to Lua debugging
hooks. Returning an error will be ignored in most cases, going totally
unnoticed. With the log at least we leave a trace.

Related to issue #2302.
Commits on Jan 21, 2015
Salvatore Sanfilippo Fix gcc warning for lack of casting to char pointer. 92cfab4
Matt Stancliff mattsta Tell sentinel/cluster tests to allow valgrind b0146aa
Matt Stancliff mattsta Add --track-origins=yes to valgrind 491d57a
Salvatore Sanfilippo Cluster: set the slaves->slaveof filed to NULL when master is freed.
Related to issue #2289.
Salvatore Sanfilippo Cluster: node deletion cleanup / centralization. 2601e3e
Salvatore Sanfilippo Cluster/Sentinel test: pause on exceptions as well. a5bb0a0
Salvatore Sanfilippo Cluster/Sentinel test: also pause on abort_sentinel_test call. b3bf758
Salvatore Sanfilippo AOF rewrite: set iterator var to NULL when freed.
The cleanup code expects that if 'di' is not NULL, it is a valid
iterator that should be freed.

The result of this bug was a crash of the AOF rewriting process if an
error occurred after the DBs data are written and the iterator is no
longer valid.
Salvatore Sanfilippo Cluster test: wait for port to unbound in kill_instance.
Otherwise kill_instance + restart_instance in short succession will
still find the port busy and will fail.
Matt Stancliff mattsta Fix sending uninitialized bytes
Fixes valgrind error:
Syscall param write(buf) points to uninitialised byte(s)
   at 0x514C35D: ??? (syscall-template.S:81)
   by 0x456B81: clusterWriteHandler (cluster.c:1907)
   by 0x41D596: aeProcessEvents (ae.c:416)
   by 0x41D8EA: aeMain (ae.c:455)
   by 0x41A84B: main (redis.c:3832)
 Address 0x5f268e2 is 2,274 bytes inside a block of size 8,192 alloc'd
   at 0x4932D1: je_realloc (jemalloc.c:1297)
   by 0x428185: zrealloc (zmalloc.c:162)
   by 0x4269E0: sdsMakeRoomFor.part.0 (sds.c:142)
   by 0x426CD7: sdscatlen (sds.c:251)
   by 0x4579E7: clusterSendMessage (cluster.c:1995)
   by 0x45805A: clusterSendPing (cluster.c:2140)
   by 0x45BB03: clusterCron (cluster.c:2944)
   by 0x423344: serverCron (redis.c:1239)
   by 0x41D6CD: aeProcessEvents (ae.c:311)
   by 0x41D8EA: aeMain (ae.c:455)
   by 0x41A84B: main (redis.c:3832)
 Uninitialised value was created by a stack allocation
   at 0x457810: nodeUpdateAddressIfNeeded (cluster.c:1236)
Matt Stancliff mattsta Fix cluster reset memory leak
[maybe] Fixes valgrind errors:
32 bytes in 4 blocks are definitely lost in loss record 107 of 228
   at 0x80EA447: je_malloc (jemalloc.c:944)
   by 0x806E59C: zrealloc (zmalloc.c:125)
   by 0x80A9AFC: clusterSetMaster (cluster.c:801)
   by 0x80AEDC9: clusterCommand (cluster.c:3994)
   by 0x80682A5: call (redis.c:2049)
   by 0x8068A20: processCommand (redis.c:2309)
   by 0x8076497: processInputBuffer (networking.c:1143)
   by 0x8073BAF: readQueryFromClient (networking.c:1208)
   by 0x8060E98: aeProcessEvents (ae.c:412)
   by 0x806123B: aeMain (ae.c:455)
   by 0x806C3DB: main (redis.c:3832)

64 bytes in 8 blocks are definitely lost in loss record 143 of 228
   at 0x80EA447: je_malloc (jemalloc.c:944)
   by 0x806E59C: zrealloc (zmalloc.c:125)
   by 0x80AAB40: clusterProcessPacket (cluster.c:801)
   by 0x80A847F: clusterReadHandler (cluster.c:1975)
   by 0x30000FF: ???

80 bytes in 10 blocks are definitely lost in loss record 148 of 228
   at 0x80EA447: je_malloc (jemalloc.c:944)
   by 0x806E59C: zrealloc (zmalloc.c:125)
   by 0x80AAB40: clusterProcessPacket (cluster.c:801)
   by 0x80A847F: clusterReadHandler (cluster.c:1975)
   by 0x2FFFFFF: ???
Matt Stancliff mattsta Fix potential invalid read past end of array
If array has N elements, we can't read +1 if we are already at N.

Also, we need to move elements by their storage size in the array,
not just by individual bytes.
Matt Stancliff mattsta Fix cluster migrate memory leak
Fixes valgrind error:
48 bytes in 1 blocks are definitely lost in loss record 196 of 373
   at 0x4910D3: je_malloc (jemalloc.c:944)
   by 0x42807D: zmalloc (zmalloc.c:125)
   by 0x41FA0D: dictGetIterator (dict.c:543)
   by 0x41FA48: dictGetSafeIterator (dict.c:555)
   by 0x459B73: clusterHandleSlaveMigration (cluster.c:2776)
   by 0x45BF27: clusterCron (cluster.c:3123)
   by 0x423344: serverCron (redis.c:1239)
   by 0x41D6CD: aeProcessEvents (ae.c:311)
   by 0x41D8EA: aeMain (ae.c:455)
   by 0x41A84B: main (redis.c:3832)
Salvatore Sanfilippo Cluster test initialization: use transaction for reset + set-config-e…

Otherwise between the two commands other nodes may contact us making the
next SET-CONFIG-EPOCH call impossible.
Salvatore Sanfilippo Merge branch 'clusterfixes' into unstable af8d1b4
Salvatore Sanfilippo getRandomHexChars(): use /dev/urandom just to seed.
On Darwin /dev/urandom depletes terribly fast. This is not an issue
normally, but with Redis Cluster we generate a lot of unique IDs, for
example during nodes handshakes. Our IDs need just to be unique without
other strong crypto requirements, so this commit turns the function into
something that gets a 20 bytes seed from /dev/urandom, and produces the
rest of the output just using SHA1 in counter mode.
Commits on Jan 22, 2015
Salvatore Sanfilippo counter must be static in getRandomHexChars(). 9826038
Salvatore Sanfilippo The seed must be static in getRandomHexChars(). a330b6c
Salvatore Sanfilippo The seed must be static in getRandomHexChars(). e4d65e3
Salvatore Sanfilippo Cluster test: when valgrind is enabled, use a larger node-timeout.
Removes some percentage of timing related failures.
Salvatore Sanfilippo Merge branch 'clusterfixes' into unstable ca14440
Salvatore Sanfilippo Avoid duplicated instance execution code in Cluster test. e22d757
Commits on Jan 23, 2015
Salvatore Sanfilippo DEBUG structsize
Show sizes of a few important data structures in Redis. More missing.
Salvatore Sanfilippo dict.c: make chaining strategy more clear in dictAddRaw(). 8aaf507
Commits on Jan 24, 2015
Salvatore Sanfilippo Cluster: initialized not used fileds in gossip section.
Otherwise we risk sending not initialized data to other nodes, that may
contain anything. This was actually not possible only because the
initialization of the buffer where the cluster packets header is created
was larger than the 3 gossip sections we use, so the memory was already
all filled with zeroes by the memset().
Commits on Jan 25, 2015
mattcollier mattcollier Update redis-cli.c
Code was adding '\n'  (line 521) to the end of NIL values exlusively making csv output inconsistent.  Removed '\n'
Commits on Jan 28, 2015
Matt Stancliff mattsta Convert check-dump to Redis check-rdb mode
redis-check-dump is now named redis-check-rdb and it runs
as a mode of redis-server instead of an independent binary.

You can now use 'redis-server redis.conf --check-rdb' to check
the RDB defined in redis.conf.  Using argument --check-rdb
checks the RDB and exits.  We could potentially also allow
the server to continue starting if the RDB check succeeds.

This change also enables us to use RDB checking programatically
from inside Redis for certain failure conditions.
Matt Stancliff mattsta Remove code duplication from check-rdb
redis-check-rdb (previously redis-check-dump) had every RDB define
copy/pasted from rdb.h and some defines copied from redis.h.  Since
the initial copy, some constants had changed in Redis headers and
check-dump was using incorrect values.

Since check-rdb is now a mode of Redis, the old check-dump code
is cleaned up to:
  - replace all printf with redisLog (and remove \n from all strings)
  - remove all copy/pasted defines to use defines from rdb.h and redis.h
  - replace all malloc/free with zmalloc/zfree
  - remove unnecessary include headers
Matt Stancliff mattsta Improve RDB error-on-load handling
Previouly if we loaded a corrupt RDB, Redis printed an error report
with a big "REPORT ON GITHUB" message at the bottom.  But, we know
RDB load failures are corrupt data, not corrupt code.

Now when RDB failure is detected (duplicate keys or unknown data
types in the file), we run check-rdb against the RDB then exit.  The
automatic check-rdb hopefully gives the user instant feedback
about what is wrong instead of providing a mysterious stack
Salvatore Sanfilippo Merge pull request #1744 from mattsta/better-RDB-failure-error
Improve Loading RDB Failure Action
Salvatore Sanfilippo create-cluster script added.
Simple shell script to create / destroy Redis clusters for manual
Salvatore Sanfilippo Ignore redis-check-rdb after the name switch. 69583be
Commits on Jan 29, 2015
Salvatore Sanfilippo create-cluster script: sane default timeout. d59ad97
Salvatore Sanfilippo Cluster: use a number of gossip sections proportional to cluster size.
Otherwise it is impossible to receive the majority of failure reports in
the node_timeout*2 window in larger clusters.

Still with a 200 nodes cluster, 20 gossip sections are a very reasonable
amount of bytes to send.

A side effect of this change is also fater cluster nodes joins for large
clusters, because the cluster layout makes less time to propagate.
Salvatore Sanfilippo CLUSTER count-failure-reports command added. 1efacfe
Salvatore Sanfilippo Cluster: magical 10% of nodes explained in comments. e5a2206
Commits on Jan 30, 2015
Salvatore Sanfilippo Cluster: create-cluster script improved. 6b1c633
Salvatore Sanfilippo More correct wanted / maxiterations values in clusterSendPing(). 69b4f00
Salvatore Sanfilippo Cluster: some bias towwards FAIL/PFAIL nodes in gossip sections.
This improves PFAIL -> FAIL switch. Too late at this point in the RC
releases to add proper PFAIL/FAIL separate dictionary to do this in a
less randomized way. Tested in practice with experiments that this
helps. PFAIL -> FAIL average with 20 nodes and node-timeout set to 5
seconds takes 2.5 seconds without this commit, 1 second with this
Salvatore Sanfilippo Cluster: Tcl script to check avg pfail->fail time. 79fa67c
Salvatore Sanfilippo redis-check-rdb: initialize entry in case while is never entered. 6502947
Commits on Feb 02, 2015
Sun He sunheehnus sort.c: REDIS_LIST's dontsort optimization
also fix the situation "dontsort DESC" of a list
Salvatore Sanfilippo Merge pull request #2318 from mattcollier/patch-1
Fix redis-cli CSV NIL output
Commits on Feb 03, 2015
Salvatore Sanfilippo Ignore inside create-cluster script dir. 128c642
Salvatore Sanfilippo Norrow backtrace and setproctitle() to Linux+glibc.
Backtrace is a glibc extension, while setproctitle() implementation
depends on the memory layout and is partially libc dependent.
Salvatore Sanfilippo Handle redis-check-rdb as a standalone program.
This also makes it backward compatible in the usage, but for the command
name. However the old command name was less obvious so it is worth to
break it probably.

With the new setup the program main can perform argument parsing and
everything else useful for an RDB check regardless of the Redis server
Salvatore Sanfilippo Check RDB automatically in a few more cases. fad758b
Salvatore Sanfilippo Merge branch 'unstable' of git:// into set-pr 5101000
Salvatore Sanfilippo More obvious indentation in setCommand(). 1dbd8e9
Salvatore Sanfilippo Merge pull request #2346 from sunheehnus/sort
Little optimization & bug fix in sort.c
Salvatore Sanfilippo Hopefully better sort.c optimization comments.
Related to #2346.
Salvatore Sanfilippo Suppress sign warning in sort.c.
Related to #2346.
Commits on Feb 04, 2015
Salvatore Sanfilippo Replication: put server.master client creation into separated function. c5dd686
Salvatore Sanfilippo Merge branch 'unstable' of into unstable 78dd2b3
Salvatore Sanfilippo Fix RDB corruption test after server behavior change. 2ac7b5a
Chris Lamb lamby Support "1G" etc. units in CONFIG SET maxmemory
Signed-off-by: Chris Lamb <>
Commits on Feb 07, 2015
Salvatore Sanfilippo Initial implementation of redis-cli --latency-dist. 2860cf4
Salvatore Sanfilippo Add missing latency-dest legend symbol. 414df14
Mariano Pérez Rodríguez mariano-perez-rodriguez Fix for #2371
Fixing #2371 as per @mattsta's suggestion
Salvatore Sanfilippo redis-cli latency dist: add new top HL. ace1acc
Salvatore Sanfilippo redis-cli --latency-dist now uses a color palette.
Still not happy with the result but low grays are hard to see in certain
monitors with a non perfect gamma.
Commits on Feb 08, 2015
Salvatore Sanfilippo redis-cli --latecy-dist reverted to gray scale.
So far not able to find a color palette within the 256 colors which is
not confusing. However I believe it is a possible task, so will try
better later.
Salvatore Sanfilippo redis-cli: interactive reconnection for latency modes.
--stat mode already used to reconnect automatically if the server is no
longer available. This is useful since this is an interactive mode used
for debugging, however the same applies to --latency and --latency-dist
modes, so now both use the reconnecting command execution as well.

The reconnection code was modified to use basic VT100 escape sequences
in order to play better with different kinds of output on the screen
when the reconnection happens, and to hide the reconnection attempt
output when finally the reconnection happens.
Commits on Feb 09, 2015
Salvatore Sanfilippo redis-cli --lru-test implemented (cache workload simulator). bd128f7
Commits on Feb 10, 2015
Salvatore Sanfilippo Merge pull request #2372 from mariano-perez-rodriguez/patch-2
Salvatore Sanfilippo HAVE_SYNC_FILE_RANGE should be protected by ifdef __linux__.
Related to issue #2372.
Salvatore Sanfilippo Added regression test for issue #2371. e1fce55
Salvatore Sanfilippo Faster memory efficiency test.
This test on Linux was extremely slow, since in Tcl we can't enable
easily tcp-nodelay, so the busy loop used to take *a lot* with bigger
writes. Fixed using pipelining.
Commits on Feb 11, 2015
Salvatore Sanfilippo Less blocking dictGetRandomKeys().
Related to issue #2306.
Salvatore Sanfilippo dict.c: prevent useless resize to same size.
Related to issue #2306.
Salvatore Sanfilippo dict.c: put a bound to max work dictRehash() call can do.
Related to issue #2306.
Salvatore Sanfilippo dict.c: dictGetRandomKeys() visit pattern optimization.
We use the invariant that the original table ht[0] is never populated up
to the index before the current rehashing index.

Related to issue #2306.
Salvatore Sanfilippo dict.c: dictGetRandomKeys() optimization for big->small table case.
Related to issue #2306.
Salvatore Sanfilippo dict.c: don't try buckets that are empty for sure in dictGetRandomKey().
This is very similar to the optimization applied to dictGetRandomKeys,
but applied to the single key variant.

Related to issue #2306.
Sun He sunheehnus dict.c/dictRehash: check again to update 2385630
Salvatore Sanfilippo dict.c: avoid code repetition in dictRehash().
Avoid code repetition introduced with PR #2367, also fixes the return
value to always return 0 if there is nothing more to rehash.
Salvatore Sanfilippo dict.c: add dictGetSomeKeys(), specialized for eviction. 5792a21
Salvatore Sanfilippo Remove optional single-key path from evictionPoolPopulate(). 8696874
Salvatore Sanfilippo redis-cli --stat: show LOAD when loading. 05841a6
Salvatore Sanfilippo dict.c Rehashing visualization code snippet added to utils. edda00b
Salvatore Sanfilippo dict.c: reset emptylen when bucket is not empty.
Fixed by @oranagra, thank you.
Salvatore Sanfilippo Separate latency monitoring of eviction loop and eviction DELs. 585d1a6
Salvatore Sanfilippo SPOP with count: initial fixes to the implementation.
Severan problems are addressed but still a few missing.
Since replication of this command was more complex than others since it
needs to replicate multiple SREM commands, an old API able to do this
was reused (it was taken inside the implementation since it was pretty
obvious soon or later that would be useful). The API was improved a bit
so that now a command may opt-out for the standard command replication
when the server.dirty counter is incremented, in order to "manually"
replicate what it wants.
Salvatore Sanfilippo Change alsoPropagate() behavior to make it more usable.
Now the API automatically creates its argv copy and increment ref count
of passed objects.
Salvatore Sanfilippo alsoPropagate: handle REDIS_CALL_PROPAGATE and AOF loading. 55003f7
Salvatore Sanfilippo SPOP: reimplemented for speed and better distribution.
The old version of SPOP with "count" argument used an API call of dict.c
which was actually designed for a different goal, and was not capable of
good distribution. We follow a different three-cases approach optimized
for different ratiion between sets and requested number of elements.

The implementation is simpler and allowed the removal of a large amount
of code.
Salvatore Sanfilippo SPOP with count: fix replication for code path #3. a37d0f8
Salvatore Sanfilippo SPOP explicit tests for the three different code paths. dffbbb5
Salvatore Sanfilippo SPOP replication tests. a1d9ec0
Commits on Feb 12, 2015
Masahiko Sawada MasahikoSawada Unify to uppercase the headline 2b3eba0
Salvatore Sanfilippo Better memtoll() error checking.
Related to PR #2357.
Salvatore Sanfilippo Merge pull request #2357 from lamby/config-set-maxmemory-units
Support "1G" etc. units in CONFIG SET maxmemory
Salvatore Sanfilippo Extend memory unit support in CONFIG SET.
Related to PR #2357.
Salvatore Sanfilippo Merge pull request #2390 from sawaman/fix-typo
Unify to uppercase the headline
Commits on Feb 13, 2015
Salvatore Sanfilippo redis-cli --latency-dist, hopefully better palette.
Less grays: more readable palette since usually we have a non linear
distribution of percentages and very near gray tones are hard to take
apart. Final part of the palette is gradient from yellow to red. The red
part is hardly reached because of usual distribution of latencies, but
shows up mainly when latencies are very high because of the logarithmic
scale, this is coherent to what people expect: red = bad.
Salvatore Sanfilippo redis-cli --latency-dist: one gray more, and --mono support. f638f04
Salvatore Sanfilippo README -> cac9a90
Salvatore Sanfilippo Fix README minor grammar issues. 097b307
Salvatore Sanfilippo A few more README markdown conversions. c414db2
Jan-Erik Rediger badboy Use code tags where necessary, escape special markers 2576864
Salvatore Sanfilippo README: give idea about what Redis is. f916a58
Salvatore Sanfilippo Fix links into README. 78c44ca
Salvatore Sanfilippo Fix README links again.
Apparently no refernece-style links supported in Github markdown.
Jan-Erik Rediger badboy Link files in README 07a8965
Salvatore Sanfilippo Merge pull request #2393 from badboy/readme-fix
Readme fix
Commits on Feb 14, 2015
Salvatore Sanfilippo Version bumped to 3.1.999 (3.2 alpha versions). 41a1fb8
Salvatore Sanfilippo Merge branch 'unstable' of into unstable cfe5eaa
Commits on Feb 17, 2015
emperor1523 emperor1523 Fix quicklist Pop() result
Closes #2398
Matt Stancliff mattsta Fix quicklist tests for Pop()
Now the tests actually compare return values instead of just
verifying _something_ got returned.
Commits on Feb 18, 2015
Sisir Koppaka sisirkoppaka rehashing.c: Fix compile error originating from SPOP rewrite acb933a
Commits on Feb 19, 2015
Salvatore Sanfilippo Example redis.conf binds just to
It's hard to pick a good approach here. A few arguments:

1) There are many exposed instances on the internet.
2) Changing the default when "bind" is not given is very dangerous,
   after an upgrade the server changes a fundamental behavior.
3) Usually Redis, when used in a proper way, will be protected *and*
   accessed often from other computers, so this new default is likely
   not what most people want.
4) However if users end with this default, they are using the example
   redis.conf: likely they are reading what is inside, and they'll see
   the warning.
Salvatore Sanfilippo Give obvious instructions on how to bind all interfaces. 0aa5acc
Commits on Feb 21, 2015
landmime landmime added a new hvstrlen command
the hvstrlen command returns the length of a hash field value
Commits on Feb 23, 2015
Salvatore Sanfilippo Change RENAME behavior when src and dst keys are the same.
Fixes issue #2392.
Commits on Feb 24, 2015
Salvatore Sanfilippo Merge pull request #2401 from mattsta/fix/quicklist/pop
Fix quicklist Pop()
Salvatore Sanfilippo Merge pull request #2273 from mattsta/improve/consistency/INFO/memory
Improve consistency of INFO MEMORY
Salvatore Sanfilippo Merge pull request #1966 from mattsta/fix-sentinel-info
Sentinel: Improve INFO command behavior
Salvatore Sanfilippo Merge pull request #2301 from mattsta/fix/lengths
Improve type correctness
Matt Stancliff mattsta Fix types broken by previous type cleanup
Revert some size_t back to off_t
Set reply_bytes needs to 64 bits everywhere
Revert bufpos to int since it's a max of 16k into buf[]
Commits on Feb 25, 2015
Salvatore Sanfilippo Merge pull request #2054 from mattsta/fix-set-sentinel-quorum
Sentinel: Add initial quorum bounds check
Salvatore Sanfilippo Merge pull request #2050 from mattsta/bitops-no-overalloc
Bitops: Stop overallocating storage space on set
Salvatore Sanfilippo Test: split basic unit into string, incr, keyspace units. 954151f
Salvatore Sanfilippo Cast sentlen to int before comparison wit bufpos.
This is safe since bufpos is small, inside the range of the local
client buffer.
Salvatore Sanfilippo Test: fixes a few tests after basic unit refactoring. 74354ce
Salvatore Sanfilippo Merge pull request #2405 from sisirkoppaka/unstable
rehashing.c: Fix compile error originating from SPOP rewrite
Salvatore Sanfilippo utils/hashtable/rehashing.c test updated to use new API. cc0d339
Commits on Feb 26, 2015
tektommy tektommy Add last_dbid to migrateCachedSocket to avoid redundant SELECT
Avoid redundant SELECT calls when continuously migrating keys to
the same dbid within a target Redis instance.
Salvatore Sanfilippo Improvements to PR #2425
1. Remove useless "cs" initialization.
2. Add a "select" var to capture a condition checked multiple times.
3. Avoid duplication of the same if (!copy) conditional.
4. Don't increment dirty if copy is given (no deletion is performed),
   otherwise we propagate MIGRATE when not needed.
Commits on Feb 27, 2015
Salvatore Sanfilippo server.current_client fix and minor refactoring.
Thanks to @codeslinger (Toby DiPasquale) for identifying the issue.

Related to issue #2409.
Salvatore Sanfilippo Utils: added function to get radix 10 string length of signed integer. c955078
Salvatore Sanfilippo Hash: API to get value string len by field name. d8f8b05
Salvatore Sanfilippo Merge pull request #2415 from landmime/unstable
added a new hvstrlen command
Salvatore Sanfilippo Merge branch 'unstable' of into unstable 8855b81
Salvatore Sanfilippo Hash: HSTRLEN (was HVSTRLEN) improved.
1. HVSTRLEN -> HSTRLEN. It's unlikely one needs the length of the key,
   not clear how the API would work (by value does not make sense) and
   there will be better names anyway.
2. Default is to return 0 when field is missing.
3. Default is to return 0 when key is missing.
4. The implementation was slower than needed, and produced unnecessary COW.

Related issue #2415.
Salvatore Sanfilippo Hash: HSTRLEN crash fixed when getting len of int-encoded value 0ace1e6
Salvatore Sanfilippo Test: HSTRLEN stress test of corner cases.
Main point here is to correctly report LLONG_MIN length, since to take
the absolute value we need care in sdigits10().
Salvatore Sanfilippo Utils: Include stdint.h and fix signess in sdigits10(). 0e5e8ca
Salvatore Sanfilippo String: use new sdigits10() API in stringObjectLen().
Should be much faster, and regardless, the code is more obvious now
compared to generating a string just to get the return value of the
ll2stirng() function.
Salvatore Sanfilippo Migrate: replace conditional with pre-computed value. c77081a