modified aeDeleteFileEvent() function #1450

Closed
wants to merge 631 commits into
from

Projects

None yet
@liexusong
Contributor

modified aeDeleteFileEvent() function:

void aeDeleteFileEvent(aeEventLoop *eventLoop, int fd, int mask)
{
    if (fd >= eventLoop->setsize) return;
    aeFileEvent *fe = &eventLoop->events[fd];

    if (fe->mask == AE_NONE) return;

    aeApiDelEvent(eventLoop, fd, mask);

    fe->mask = fe->mask & (~mask);
    if (fd == eventLoop->maxfd && fe->mask == AE_NONE) {
        /* Update the max fd */
        int j;

        for (j = eventLoop->maxfd-1; j >= 0; j--)
            if (eventLoop->events[j].mask != AE_NONE) break;
        eventLoop->maxfd = j;
    }
}
and others added some commits Oct 25, 2012
@antirez Fix compilation on Linux kernels or glibc versions lacking sync_file_…
…range().

This fixes issue #667.

Many thanks to Didier Spezia for the fix.
2a49aa1
@openbaas @antirez openbaas patch config.h for sync_file_range 72f42cf
@antirez Marginally more robust glibc version test for sync_file_range detection. 6bd1050
@antirez Ctrl+w support in linenoise. 6c7e957
@antirez help.h update (adds bitop, bitcount, evalsha...) 38272f9
@antirez No longer used macro rdbIsOpcode() removed. 537235b
@antirez Invert two sides of if expression in SET to avoid a lookup.
Because of the short circuit behavior of && inverting the two sides of
the if expression avoids an hash table lookup if the non-EX variant of
SET is called.

Thanks to Weibin Yao (@yaoweibin on github) for spotting this.
9fab63b
@yamt @antirez yamt fix a typo in a comment 428b706
@antirez 32 bit build fixed on Linux.
It failed because of the way jemalloc was compiled (without passing the
right flags to make, but just to configure). Now the same set of flags
are also passed to the make command, fixing the issue.

This fixes issue #744
655e683
@antirez Unix socket clients properly displayed in MONITOR and CLIENT LIST.
This also fixes issue #745.
d775b35
@cofyc @antirez cofyc fix typo in comments (redis.c, networking.c) fecc879
@antirez More robust handling of AOF rewrite child.
After the wait3() syscall we used to do something like that:

    if (pid == server.rdb_child_pid) {
        backgroundSaveDoneHandler(exitcode,bysignal);
    } else {
        ....
    }

So the AOF rewrite was handled in the else branch without actually
checking if the pid really matches. This commit makes the check explicit
and logs at WARNING level if the pid returned by wait3() does not match
neither the RDB or AOF rewrite child.
64be5e3
@run @antirez run fix a typo in redis.h line 595 comment 2b28ef0
@antirez Type mismatch errors are now prefixed with WRONGTYPE.
So instead to reply with a generic error like:

-ERR ... wrong kind of value ...

now it replies with:

-WRONGTYPE ... wrong kind of value ...

This makes this particular error easy to check without resorting to
(fragile) pattern matching of the error string (however the error string
used to be consistent already).

Client libraries should return a specific exeption type for this error.

Most of the commit is about fixing unit tests.
46c5d39
@antirez BSD license added to every C source and header file. 8ddb23b
@antirez Make clear that contributing code to the Redis project means to relea…
…se it under the terms of the BSD license.
cda5c0b
@antirez Copyright date fixed in COPYING file. 388cc20
@antirez TTL API change: TTL returns -2 for non existing keys.
The previous behavior was to return -1 if:

1) Existing key but without an expire set.
2) Non existing key.

Now the second case is handled in a different, and TTL will return -2
if the key does not exist at all.

PTTL follows the same behavior as well.
50c41de
@antirez zmalloc: kill unused __size parameter in update_zmalloc_stat_alloc() …
…macro.
1a706c7
@antirez zmalloc_get_private_dirty() function added (Linux only).
For non Linux systmes it just returns 0.

This function is useful to estimate copy-on-write because of childs
saving stuff on disk.
1d8af60
@antirez Children creating AOF or RDB files now report memory used by COW.
Finally Redis is able to report the amount of memory used by
copy-on-write while saving an RDB or writing an AOF file in background.

Note that this information is currently only logged (at NOTICE level)
and not shown in INFO because this is less trivial (but surely doable
with some minor form of interprocess communication).

The reason we can't capture this information on the parent before we
call wait3() is that the Linux kernel will release the child memory
ASAP, and only retain the minimal state for the process that is useful
to report the child termination to the parent.

The COW size is obtained by summing all the Private_Dirty fields found
in the "smap" file inside the proc filesystem for the process.

All this is Linux specific and is not available on other systems.
c342b07
@antirez Use more fine grained HAVE macros instead of HAVE_PROCFS. dd90541
@antirez Fast memory test on Redis crash. d4b5c48
@antirez Make bio.c threads killable ASAP if needed.
We use this new bio.c feature in order to stop our I/O threads if there
is a memory test to do on crash. In this case we don't want anything
else than the main thread to run, otherwise the other threads may mess
with the heap and the memory test will report a false positive.
dff1ec4
@antirez Safer handling of MULTI/EXEC on errors.
After the transcation starts with a MULIT, the previous behavior was to
return an error on problems such as maxmemory limit reached. But still
to execute the transaction with the subset of queued commands on EXEC.

While it is true that the client was able to check for errors
distinguish QUEUED by an error reply, MULTI/EXEC in most client
implementations uses pipelining for speed, so all the commands and EXEC
are sent without caring about replies.

With this change:

1) EXEC fails if at least one command was not queued because of an
error. The EXECABORT error is used.
2) A generic error is always reported on EXEC.
3) The client DISCARDs the MULTI state after a failed EXEC, otherwise
pipelining multiple transactions would be basically impossible:
After a failed EXEC the next transaction would be simply queued as
the tail of the previous transaction.
60f9dac
@antirez Test: MULTI/EXEC tests moved into multi.tcl. 53600e3
@antirez Test: make sure EXEC fails after previous transaction errors. a527235
@antirez Test: MULTI state is cleared after EXECABORT error. ca9321a
@antirez Fix integer overflow in zunionInterGenericCommand().
This fixes issue #761.
cc85ed4
@antirez EVALSHA is now case insensitive.
EVALSHA used to crash if the SHA1 was not lowercase (Issue #783).
Fixed using a case insensitive dictionary type for the sha -> script
map used for replication of scripts.
d0570c9
@openbaas @antirez openbaas remove compile warning bioKillThreads ed48550
@arsenm @antirez arsenm It's a watchdog, not a watchdong. ccb500c
@openbaas @antirez openbaas Remove unnecessary condition in _dictExpandIfNeeded (dict.c) 1f0d0b0
@antirez Jemalloc updated to version 3.2.0. 053f1f3
@antirez On crash memory test rewrote so that it actaully works.
1) We no longer test location by location, otherwise the CPU write cache
completely makes our business useless.
2) We still need a memory test that operates in steps from the first to
the last location in order to never hit the cache, but that is still
able to retain the memory content.

This was tested using a Linux box containing a bad memory module with a
zingle bit error (always zero).

So the final solution does has an error propagation step that is:

1) Invert bits at every location.
2) Swap adiacent locations.
3) Swap adiacent locations again.
4) Invert bits at every location.
5) Swap adiacent locations.
6) Swap adiacent locations again.

Before and after these steps, and after step 4, a CRC64 checksum is computed.
If the three CRC64 checksums don't match, a memory error was detected.
cd99c14
@antirez Introduced the Build ID in INFO and --version output.
The idea is to be able to identify a build in a unique way, so for
instance after a bug report we can recognize that the build is the one
of a popular Linux distribution and perform the debugging in the same
environment.
3b71404
@antirez Make an EXEC test more latency proof. e34c14d
@antirez redis-benchmark: seed the PRNG with time() at startup. 2572bb1
@antirez SDIFF is now able to select between two algorithms for speed.
SDIFF used an algorithm that was O(N) where N is the total number
of elements of all the sets involved in the operation.

The algorithm worked like that:

ALGORITHM 1:

1) For the first set, add all the members to an auxiliary set.
2) For all the other sets, remove all the members of the set from the
auxiliary set.

So it is an O(N) algorithm where N is the total number of elements in
all the sets involved in the diff operation.

Cristobal Viedma suggested to modify the algorithm to the following:

ALGORITHM 2:

1) Iterate all the elements of the first set.
2) For every element, check if the element also exists in all the other
remaining sets.
3) Add the element to the auxiliary set only if it does not exist in any
of the other sets.

The complexity of this algorithm on the worst case is O(N*M) where N is
the size of the first set and M the total number of sets involved in the
operation.

However when there are elements in common, with this algorithm we stop
the computation for a given element as long as we find a duplicated
element into another set.

I (antirez) added an additional step to algorithm 2 to make it faster,
that is to sort the set to subtract from the biggest to the
smallest, so that it is more likely to find a duplicate in a larger sets
that are checked before the smaller ones.

WHAT IS BETTER?

None of course, for instance if the first set is much larger than the
other sets the second algorithm does a lot more work compared to the
first algorithm.

Similarly if the first set is much smaller than the other sets, the
original algorithm will less work.

So this commit makes Redis able to guess the number of operations
required by each algorithm, and select the best at runtime according
to the input received.

However, since the second algorithm has better constant times and can do
less work if there are duplicated elements, an advantage is given to the
second algorithm.
ccc974d
@antirez SDIFF fuzz test added. c1eda78
@antirez Client should not block multiple times on the same key.
Sending a command like:

BLPOP foo foo foo foo 0

Resulted into a crash before this commit since the client ended being
inserted in the waiting list for this key multiple times.
This resulted into the function handleClientsBlockedOnLists() to fail
because we have code like that:

    if (de) {
        list *clients = dictGetVal(de);
        int numclients = listLength(clients);

        while(numclients--) {
            listNode *clientnode = listFirst(clients);

            /* server clients here... */
        }
    }

The code to serve clients used to remove the served client from the
waiting list, so if a client is blocking multiple times, eventually the
call to listFirst() will return NULL or worse will access random memory
since the list may no longer exist as it is removed by the function
unblockClientWaitingData() if there are no more clients waiting for this
list.

To avoid making the rest of the implementation more complex, this commit
modifies blockForKeys() so that a client will be put just a single time
into the waiting list for a given key.

Since it is Saturday, I hope this fixes issue #801.
64e69c0
@antirez Test: regression for issue #801. fbf3e33
@antirez Blocking POP: use a dictionary to store keys clinet side.
To store the keys we block for during a blocking pop operation, in the
case the client is blocked for more data to arrive, we used a simple
linear array of redis objects, in the blockingState structure:

    robj **keys;
    int count;

However in order to fix issue #801 we also use a dictionary in order to
avoid to end in the blocked clients queue for the same key multiple
times with the same client.

The dictionary was only temporary, just to avoid duplicates, but since
we create / destroy it there is no point in doing this duplicated work,
so this commit simply use a dictionary as the main structure to store
the keys we are blocked for. So instead of the previous fields we now
just have:

    dict *keys;

This simplifies the code and reduces the work done by the server during
a blocking POP operation.
07a9f85
@antirez Test: fixed osx "leaks" support in test.
Due to changes in recent releases of osx leaks utility, the osx leak
detection no longer worked. Now it is fixed in a way that should be
backward compatible.
e397352
@antirez Memory leak fixed: release client's bpop->keys dictionary.
Refactoring performed after issue #801 resolution (see commit
2f87cf8) introduced a memory leak that
is fixed by this commit.

I simply forgot to free the new allocated dictionary in the client
structure trusting the output of "make test" on OSX.

However due to changes in the "leaks" utility the test was no longer
testing memory leaks. This problem was also fixed.

Fortunately the CI test running at ci.redis.io spotted the bug in the
valgrind run.

The leak never ended into a stable release.
124cb6d
@bmcmanus @antirez bmcmanus Issue 804 Add Default-Start and Default-Stop LSB tags for RedHat star…
…tup and update-rc.d compatability.
17e31eb
@antirez Fix config.h endianess detection to work on Linux / PPC64.
Config.h performs endianess detection including OS-specific headers to
define the endianess macros, or when this is not possible, checking the
processor type via ifdefs.

Sometimes when the OS-specific macro is included, only __BYTE_ORDER is
defined, while BYTE_ORDER remains undefined. There is code at the end of
config.h endianess detection in order to define the macros without the
underscore, but it was not working correctly.

This commit fixes endianess detection fixing Redis on Linux / PPC64 and
possibly other systems.
bbee226
@antirez Patrick TJ McPhee Define _XOPEN_SOURCE appropriately on NetBSD. 5207c03
@antirez serverCron() frequency is now a runtime parameter (was REDIS_HZ).
REDIS_HZ is the frequency our serverCron() function is called with.
A more frequent call to this function results into less latency when the
server is trying to handle very expansive background operations like
mass expires of a lot of keys at the same time.

Redis 2.4 used to have an HZ of 10. This was good enough with almost
every setup, but the incremental key expiration algorithm was working a
bit better under *extreme* pressure when HZ was set to 100 for Redis
2.6.

However for most users a latency spike of 30 milliseconds when million
of keys are expiring at the same time is acceptable, on the other hand a
default HZ of 100 in Redis 2.6 was causing idle instances to use some
CPU time compared to Redis 2.4. The CPU usage was in the order of 0.3%
for an idle instance, however this is a shame as more energy is consumed
by the server, if not important resources.

This commit introduces HZ as a runtime parameter, that can be queried by
INFO or CONFIG GET, and can be modified with CONFIG SET. At the same
time the default frequency is set back to 10.

In this way we default to a sane value of 10, but allows users to
easily switch to values up to 500 for near real-time applications if
needed and if they are willing to pay this small CPU usage penalty.
a6d117b
@antirez Added missing license and copyright in deps/hiredis. 6fa2610
@antirez CONTRIBUTING updated with request to add BSD license. 61fb0aa
@antirez Fix overflow in mstime() in redis-cli and benchmark.
The problem does not exist in the Redis server implementation of mstime()
but is only limited to redis-cli and redis-benchmark.

Thix fixes issue #839.
858b557
@antirez ae.c: set errno when error is not a failing syscall.
In this way the caller is able to perform better error checking or to
use strerror() without the risk of meaningless error messages being
displayed.
6570d2f
@antirez Better error reporting when fd event creation fails. cefff3a
@antirez Multiple fixes for EVAL (issue #872).
1) The event handler was no restored after a timeout condition if the
   command was eventually executed with success.
2) The command was not converted to EVAL in case of errors in the middle
   of the execution.
3) Terrible duplication of code without any apparent reason.
414e066
@antirez Test: added regression for issue #872. e2e36cf
@antirez Comment in the call() function clarified a bit. 8a70007
@antirez Makefile.dep updated. 64ee9c8
@antirez Undo slave-master handshake when SLAVEOF sets a new slave.
Issue #828 shows how Redis was not correctly undoing a non-blocking
connection attempt with the previous master when the master was set to a
new address using the SLAVEOF command.

This was also a result of lack of refactoring, so now there is a
function to cancel the non blocking handshake with the master.
The new function is now used when SLAVEOF NO ONE is called or when
SLAVEOF is used to set the master to a different address.
aa9497f
@antirez CLIENT GETNAME and CLIENT SETNAME introduced.
Sometimes it is much simpler to debug complex Redis installations if it
is possible to assign clients a name that is displayed in the CLIENT
LIST output.

This is the case, for example, for "leaked" connections. The ability to
provide a name to the client makes it quite trivial to understand what
is the part of the code implementing the client not releasing the
resources appropriately.

Behavior:

    CLIENT SETNAME: set a name for the client, or remove the current
                    name if an empty name is set.
    CLIENT GETNAME: get the current name, or a nil.
    CLIENT LIST: now displays the client name if any.

Thanks to Mark Gravell for pushing this idea forward.
786bd39
@antirez Typo fixed, ASCI -> ASCII. f31d10b
@antirez Tests for CLIENT GETNAME/SETNAME. 7897e0a
@antirez redis-cli: save an RDB dump from remote server to local file. c48584b
@nparry @antirez nparry redis-cli --rdb fails if server sends a ping
Redis pings slaves in "pre-synchronization stage" with newlines. (See
https://github.com/antirez/redis/blob/2.6.9/src/replication.c#L814)
However, redis-cli does not expect this - it sees the newline as the end
of the bulk length line, and ends up returning 0 as bulk the length.
This manifests as the following when running redis-cli:

    $ ./src/redis-cli --rdb some_file
    SYNC sent to master, writing 0 bytes to 'some_file'
    Transfer finished with success.

With this commit, we just ignore leading newlines while reading the bulk
length line.

To reproduce the problem, load enough data into Redis so that the
preparation of the RDB snapshot takes long enough for a ping to occur
while redis-cli is waiting for the data.
f2bc198
@bitterb @antirez bitterb Fix an error reply for CLIENT command aadcda9
@badboy @antirez badboy Always exit if connection fails.
This avoids unnecessary core dumps. Fixes antirez/redis#894
a8f9cec
@openbaas @antirez openbaas redis-cli prompt bug fix ff1e4d2
@guiquanz @antirez guiquanz Fixed many typos.
Conflicts fixed, mainly because 2.8 has no cluster support / files:
	00-RELEASENOTES
	src/cluster.c
	src/crc16.c
	src/redis-trib.rb
	src/redis.h
1caf093
@antirez Slowlog: don't log EXEC but just the executed commands.
The Redis Slow Log always used to log the slow commands executed inside
a MULTI/EXEC block. However also EXEC was logged at the end, which is
perfectly useless.

Now EXEC is no longer logged and a test was added to test this behavior.

This fixes issue #759.
d766907
@antirez Clear server.shutdown_asap on failed shutdown.
When a SIGTERM is received Redis schedules a shutdown. However if it
fails to perform the shutdown it must be clear the shutdown_asap flag
otehrwise it will try again and again possibly making the server
unusable.
1e20c93
@antirez Whitelist SIGUSR1 to avoid auto-triggering errors.
This commit fixes issue #875 that was caused by the following events:

1) There is an active child doing BGSAVE.
2) flushall is called (or any other condition that makes Redis killing
the saving child process).
3) An error is sensed by Redis as the child exited with an error (killed
by a singal), that stops accepting write commands until a BGSAVE happens
to be executed with success.

Whitelisting SIGUSR1 and making sure Redis always uses this signal in
order to kill its own children fixes the issue.
39f0a33
@antirez Additionally two typos fixed thanks to @jodal 635c532
@antirez Not every __sun has backtrace().
I don't know how to test for Open Solaris that has support for
backtrace() so for now removing the #ifdef that breaks compilation under
other Solaris flavors.
f6f43d7
@bilalhusain @antirez bilalhusain s/adiacent/adjacent/
fixed typo in a comment (step 2 memcheck)
2f1318a
@antirez Fixed a bug in memtest progress bar, that had no actual effects.
This closes issue #859, thanks to @erbenmo.
850117a
@antirez UNSUBSCRIBE and PUNSUBSCRIBE: always provide a reply.
UNSUBSCRIBE and PUNSUBSCRIBE commands are designed to mass-unsubscribe
the client respectively all the channels and patters if called without
arguments.

However when these functions are called without arguments, but there are
no channels or patters we are subscribed to, the old behavior was to
don't reply at all.

This behavior is broken, as every command should always reply.
Also it is possible that we are no longer subscribed to a channels but we
are subscribed to patters or the other way around, and the client should
be notified with the correct number of subscriptions.

Also it is not pretty that sometimes we did not receive a reply at all
in a redis-cli session from these commands, blocking redis-cli trying
to read the reply.

This fixes issue #714.
3ff75e5
@antirez redis-cli --bigkeys output is now simpler to understand. 767a53a
@antirez Lua struct library updated to version 0.2.
There was a bug in the previous version of this library that caused a
crash under the circumstances described in issue #901.

The newer version of the library appears to be fixed (I tested it
manually with valgrind and everything seems fine now).

For more information about this library please visit this web site:

    http://www.inf.puc-rio.br/~roberto/struct/
153766e
@antirez Keyspace events notification API. 212edbc
@antirez Two fixes to initial keyspace notifications API. 6e64525
@antirez Fixed over-80-cols comment in db.c dd2ce74
@antirez Initial test events for the new keyspace notification API. fdfb59b
@antirez notifyKeyspaceEvent(): release channel names using the right pointers. a64383c
@antirez Fix decrRefCount() prototype from void to robj pointer.
decrRefCount used to get its argument as a void* pointer in order to be
used as destructor where a 'void free_object(void*)' prototype is
expected. However this made simpler to introduce bugs by freeing the
wrong pointer. This commit fixes the argument type and introduces a new
wrapper called decrRefCountVoid() that can be used when the void*
argument is needed.
2825f21
@antirez Keyspace events added for more commands. 4ff138f
@antirez Enable keyspace events notification when testing. 81af92b
@antirez decrRefCount -> decrRefCountVoid in list constructor. 6de052c
@antirez Keyspace events: it is now possible to select subclasses of events.
When keyspace events are enabled, the overhead is not sever but
noticeable, so this commit introduces the ability to select subclasses
of events in order to avoid to generate events the user is not
interested in.

The events can be selected using redis.conf or CONFIG SET / GET.
b9bc4f9
@antirez Keyspace notifications: fixed a leak and a bug introduced in the late…
…st commit.
9db8fbc
@antirez Send 'expired' events when a key expires by lookup. d3cf778
@antirez Tests for keyspace notifications. 664de41
@antirez Generate del events when S*STORE commands delete the destination key. ca8e7d4
@antirez Z*STORE event fixed: generate del only if resulting sorted set is empty. 8024de7
@ltnwgl @antirez ltnwgl Fix a bug in srandmemberWithCountCommand()
In CASE 2, the call sunionDiffGenericCommand will involve the string "srandmember"
> sadd foo one
(integer 1)
> sadd srandmember two
(integer 2)
> srandmember foo 3
1)"one"
2)"two"
57e411c
@davidcelis @antirez davidcelis Fix a few typos and improve grammar of redis.conf
Make several edits to the example redis.conf configuration file for
improved flow and grammar.

Signed-off-by: David Celis <me@davidcel.is>
44d0eac
@antirez Rock Li retval doesn't initalized
If each if conditions are all fail, variable retval will under uninitlized
2543fa6
@jbergstroem @antirez jbergstroem Check available tcl versions e92edd3
@jbergstroem @antirez jbergstroem Enforce tcl 8.5 or newer 1124cda
@jbergstroem @antirez jbergstroem Use `info nameofexectuable` to find current executable d2a8bca
@antirez Test: No clients timeout while testing. 14daa01
and others added some commits Mar 11, 2013
@antirez In databasesCron() never test more DBs than we have. c5afbb6
@antirez activeExpireCycle() smarter with many DBs and under expire pressure.
activeExpireCycle() tries to test just a few DBs per iteration so that
it scales if there are many configured DBs in the Redis instance.
However this commit makes it a bit smarter when one a few of those DBs
are under expiration pressure and there are many many keys to expire.

What we do is to remember if in the last iteration had to return because
we ran out of time. In that case the next iteration we'll test all the
configured DBs so that we are sure we'll test again the DB under
pressure.

Before of this commit after some mass-expire in a given DB the function
tested just a few of the next DBs, possibly empty, a few per iteration,
so it took a long time for the function to reach again the DB under
pressure. This resulted in a lot of memory being used by already expired
keys and never accessed by clients.
8e7d962
@antirez redis-cli --bigkeys: don't crash with empty DBs. 378fcaa
@antirez Set default for stop_writes_on_bgsave_err in initServerConfig().
It was placed for error in initServer() that's called after the
configuation is already loaded, causing issue #1000.
b2e42a9
@djanowski @antirez djanowski Abort when opening the RDB file results in an error other than ENOENT.
This fixes cases where the RDB file does exist but can't be accessed for
any reason. For instance, when the Redis process doesn't have enough
permissions on the file.
edb3c10
@antirez rdbLoad(): rework code to save vertical space. 64b91c1
@antirez Test: check that Redis starts empty without an RDB file. 7b6fb96
@antirez Test: more RDB loading checks.
A test for issue #1001 is included.
67a7455
@antirez Test: make sure broken RDB checksum is detected. ba8367e
@antirez Replication: master_link_down_since_seconds initial value should be h…
…uge.

server.repl_down_since used to be initialized to the current time at
startup. This is wrong since the replication never started. Clients
testing this filed to check if data is uptodate should never believe
data is recent if we never ever connected to our master.
58708fa
@antirez redis-cli --stat, stolen from redis-tools.
Redis-tools is a connection of tools no longer mantained that was
intented as a way to economically make sense of Redis in the pre-vmware
sponsorship era. However there was a nice redis-stat utility, this
commit imports one of the functionalities of this tool here in redis-cli
as it seems to be pretty useful.

Usage: redis-cli --stat

The output is similar to vmstat in the format, but with Redis specific
stuff of course.

From the point of view of the monitored instance, only INFO is used in
order to grab data.
ec18d4b
@antirez Test: obuf-limits test false positive removed.
Fixes #621.
6b7562e
@NanXiao @antirez NanXiao Update config.c
Fix bug in configGetCommand function: get correct masterauth value.
382420e
@antirez Allow SELECT while loading the DB.
Fixes issue #1024.
150f7e4
@antirez Transactions: use the propagate() API to propagate MULTI.
The behavior is the same, but the code is now cleaner and uses the
proper interface instead of dealing directly with AOF/replication
functions.
d91dfaf
@antirez Transactions: use discardTransaction() in EXEC implementation. 82516e8
@antirez Transactions: propagate MULTI/EXEC only when needed.
MULTI/EXEC is now propagated to the AOF / Slaves only once we encounter
the first command that is not a read-only one inside the transaction.

The old behavior was to always propagate an empty MULTI/EXEC block when
the transaction was composed just of read only commands, or even
completely empty. This created two problems:

1) It's a bandwidth waste in the replication link and a space waste
   inside the AOF file.

2) We used to always increment server.dirty to force the propagation of
   the EXEC command, resulting into triggering RDB saves more often
   than needed.

Note: even read-only commands may also trigger writes that will be
propagated, when we access a key that is found expired and Redis will
synthesize a DEL operation. However there is no need for this to stay
inside the transaction itself, but only to be ordered.

So for instance something like:

    MULTI
    GET foo
    SET key zap
    EXEC

May be propagated into:

    DEL foo
    MULTI
    SET key zap
    EXEC

While the DEL is outside the transaction, the commands are delivered in
the right order and it is not possible for other commands to be inserted
between DEL and MULTI.
2618101
@antirez Flag PUBLISH as read-only in the command table. 10c6195
@antirez TTL / PTTL commands: two bugs fixed.
This commit fixes two corner cases for the TTL command.

1) When the key was already logically expired (expire time older
than current time) the command returned -1 instead of -2.

2) When the key was existing and the expire was found to be exactly 0
(the key was just about to expire), the command reported -1 (that is, no
expire) instead of a TTL of zero (that is, about to expire).
a452b54
@antirez Test: new functions to capture and analyze the replication stream. 7009ff9
@antirez Test: Restore DB back to 9 after testing MULTI/EXEC with DB 5. 61b1d6d
@antirez Test: test replication of MULTI/EXEC. c6a9a20
@antirez DEBUG set-active-expire added.
We need the ability to disable the activeExpireCycle() (active
expired key collection) call for testing purposes.
10d8e6a
@antirez Test: verify that lazy-expire works. 978185b
@antirez Better DEBUG error message when num of arguments is wrong. 4de4080
@antirez Test: regression test for issue #1026. be83292
@antirez EXPIRE should not resurrect keys. Issue #1026. e6e0704
@antirez Extended SET command implemented (issue #931). 24aa617
@antirez Test: Extended SET tests. 0c447d3
@openbaas @antirez openbaas Support for case unsensitive SET options. cf8484b
@jbergstroem @antirez jbergstroem Slightly refactor CFLAGS/LDFLAGS/LIBS
This way, we can avoid -rdynamic and -pthread warnings on darwin.
0c8690d
@jbergstroem @antirez jbergstroem Spaces to tabs 49eef77
@jbergstroem @antirez jbergstroem Inherit CC for Lua ca0b1c2
@jbergstroem @antirez jbergstroem make check is a common naming convention for tests c72459b
@jbergstroem @antirez jbergstroem Remove extra spaces 31844fe
@jbergstroem @antirez jbergstroem Only pass -rdynamic as linker option 2094eb3
@jbergstroem @antirez jbergstroem Silence mkdir output 034d941
@jbergstroem @antirez jbergstroem use `install` as default installer (except on SunOS) 0bc7dbd
@antirez Throttle BGSAVE attempt on saving error.
When a BGSAVE fails, Redis used to flood itself trying to BGSAVE at
every next cron call, that is either 10 or 100 times per second
depending on configuration and server version.

This commit does not allow a new automatic BGSAVE attempt to be
performed before a few seconds delay (currently 5).

This avoids both the auto-flood problem and filling the disk with
logs at a serious rate.

The five seconds limit, considering a log entry of 200 bytes, will use
less than 4 MB of disk space per day that is reasonable, the sysadmin
should notice before of catastrofic events especially since by default
Redis will stop serving write queries after the first failed BGSAVE.

This fixes issue #849
d6b0c18
@antirez Make rio.c comment 80-columns friendly. d83d72f
@antirez Test: remove useless statements and comments from test default config. 8d81e68
@antirez redis-cli: --latency-history mode implemented. 0d7b10e
@antirez redis-cli: raise error on bad command line switch.
Previously redis-cli never tried to raise an error when an unrecognized
switch was encountered, as everything after the initial options is to be
transmitted to the server.

However this is too liberal, as there are no commands starting with "-".
So the new behavior is to produce an error if there is an unrecognized
switch starting with "-". This should not break past redis-cli usages
but should prevent broken options to be silently discarded.

As far the first token not starting with "-" is encountered, all the
rest is considered to be part of the command, so you cna still use
strings starting with "-" as values, like in:

    redis-cli --port 6380 set foo --my-value
42ba5d3
@antirez Redis/Jemalloc Gitignore were too aggressive.
Redis gitignore was too aggressive since simply broken.

Jemalloc gitignore was too agressive because it is conceived to just
keep the files that allow to generate all the rest in development
environments (so for instance the "configure" file is excluded).
0e3baa5
@antirez More explicit panic message on out of memory. 3a86582
@antirez Test: split conceptually unrelated comments in RDB test. a500c12
@antirez Sentinel: turn old master into a slave when it comes back. d0c9a2a
@antirez Test: fix RDB test checking file permissions.
When the test is executed using the root account, setting the permission
to 222 does not work as expected, as root can read files with 222
permission.

Now we skip the test if root is detected.

This fixes issue #1034 and the duplicated #1040 issue.

Thanks to Jan-Erik Rediger (@badboy on Github) for finding a way to reproduce the issue.
e426ea5
@antirez rio.c: added ability to fdatasync() from time to time while writing. 5276996
@antirez AOF: sync data on disk every 32MB when rewriting.
This prevents the kernel from putting too much stuff in the output
buffers, doing too heavy I/O all at once. So the goal of this commit is
to split the disk pressure due to the AOF rewrite process into smaller
spikes.

Please see issue #1019 for more information.
9ca306d
@antirez Config option to turn AOF rewrite incremental fsync on/off. b06f13e
@antirez Sentinel: always redirect on master->slave transition.
Sentinel redirected to the master if the instance changed runid or it
was the first time we got INFO, and a role change was detected from
master to slave.

While this is a good idea in case of slave->master, since otherwise we
could detect a failover without good reasons just after a reboot with a
slave with a wrong configuration, in the case of master->slave
transition is much better to always perform the redirection for the
following reasons:

1) A Sentinel may go down for some time. When it is back online there is
no other way to understand there was a failover.
2) Pointing clients to a slave seems to be always the wrong thing to do.
3) There is no good rationale about handling things differently once an
instance is rebooted (runid change) in that case.
d2ff5ed
@antirez Lua updated to version 5.1.5. 2b427a8
@antirez Sentinel: only demote old master into slave under certain conditions.
We used to always turn a master into a slave if the DEMOTE flag was set,
as this was a resurrecting master instance.

However the following race condition is possible for a Sentinel that
got partitioned or internal issues (tilt mode), and was not able to
refresh the state in the meantime:

1) Sentinel X is running, master is instance "A".
3) "A" fails, sentinels will promote slave "B" as master.
2) Sentinel X goes down because of a network partition.
4) "A" returns available, Sentinels will demote it as a slave.
5) "B" fails, other Sentinels will promote slave "A" as master.
6) At this point Sentinel X comes back.

When "X" comes back he thinks that:

"B" is the master.
"A" is the slave to demote.

We want to avoid that Sentinel "X" will demote "A" into a slave.
We also want that Sentinel "X" will detect that the conditions changed
and will reconfigure itself to monitor the right master.

There are two main ways for the Sentinel to reconfigure itself after
this event:

1) If "B" is reachable and already configured as a slave by other
sentinels, "X" will perform a redirection to "A".
2) If there are not the conditions to demote "A", the fact that "A"
reports to be a master will trigger a failover detection in "X", that
will end into a reconfiguraiton to monitor "A".

However if the Sentinel was not reachable, its state may not be updated,
so in case it titled, or was partiitoned from the master instance of the
slave to demote, the new implementation waits some time (enough to
guarantee we can detect the new INFO, and new DOWN conditions).

If after some time still there are not the right condiitons to demote
the instance, the DEMOTE flag is cleared.
7084532
@antirez Sentinel: more sensible delay in master demote after tilt. 4028a77
@antirez Sentinel: changes to tilt mode.
Tilt mode was too aggressive (not processing INFO output), this
resulted in a few problems:

1) Redirections were not followed when in tilt mode. This opened a
   window to misinform clients about the current master when a Sentinel
   was in tilt mode and a fail over happened during the time it was not
   able to update the state.

2) It was possible for a Sentinel exiting tilt mode to detect a false
   fail over start, if a slave rebooted with a wrong configuration
   about at the same time. This used to happen since in tilt mode we
   lose the information that the runid changed (reboot).

   Now instead the Sentinel in tilt mode will still remove the instance
   from the list of slaves if it changes state AND runid at the same
   time.

Both are edge conditions but the changes should overall improve the
reliability of Sentinel.
e7bcec8
@openbaas @antirez openbaas Fix AOF bug: expire could be removed from key on AOF rewrite.
There was a race condition in the AOF rewrite code that, with bad enough
timing, could cause a volatile key just about to expire to be turned
into a non-volatile key. The bug was never reported to cause actualy
issues, but was found analytically by an user in the Redis mailing list:

https://groups.google.com/forum/?fromgroups=#!topic/redis-db/Kvh2FAGK4Uk

This commit fixes issue #1079.
a94db79
@soveran @antirez soveran Reset masterauth if an empty string is configured. b60465e
@antirez CONFIG SET server.masterauth aesthetic change.
This is just to make the code exactly like the above instance used for
requirepass. No actual change nor the original code violated the Redis
coding style.
b05d251
@antirez Added more info about 32 bit builds into README. 85671d0
@july2993 @antirez july2993 in 32bit machine, popcount don't work with a input string length up t…
…o 512 MB,

bitcount commant may return negtive integer with string length more than 256 MB
3d93dff
@antirez Test: various issues with the replication-4.tcl test fixed.
The test actually worked, but vars for master and slave were inverted
and sometimes used incorrectly.
2865bb7
@antirez Test: check that replication partial sync works if we break the link.
The test checks both successful syncs and unsuccessful ones by changing
the backlog size.
c847c73
@antirez Version bumped to 2.7.1 cb55ad9
@antirez Test: more PSYNC tests (backlog TTL). 41babac
@antirez redis-cli: help.h updated. d8dd70c
@antirez Obtain absoute path of configuration file, expose it in INFO. 7a5d3d9
@antirez CONFIG REWRITE: Initial support code and design. 973f793
@antirez CONFIG REWRITE: support for save and syslog-facility. 91ef10e
@antirez CONFIG REWRITE: support for dir and slaveof. f62851a
@antirez CONFIG REWRITE: support for client-output-buffer-limit. cb8433f
@antirez CONFIG REWRITE: strip multiple empty lines. 8cf0b66
@antirez CONFIG REWRITE: remove orphaned lines. a461376
@antirez CONFIG REWRITE: actually rewrite the config file, atomically. 8feb3d2
@antirez CONFIG REWRITE: Use sane perms when creating redis.conf from scratch. 417253a
@antirez Added a define for most configuration defaults.
Also the logfile option was modified to always have an explicit value
and to log to stdout when an empty string is used as log file.

Previously there was special handling of the string "stdout" that set
the logfile to NULL, this always required some special handling.
180cfaa
@antirez Test default config modified to set logfile to empty string.
Required because of recent changes in the way logfile is set to standard
output.
e8cc959
@antirez CONFIG REWRITE: repl-disable-tcp-nodelay is a boolean option. c95a609
@antirez CONFIG REWRITE: fixed typo in AOF fsync policy. 2a29cf4
@antirez CONFIG REWRITE: "active-rehashing" -> "activerehashing". b4ce029
@antirez CONFIG REWRITE: correctly escape the notify-keyspace-events option. 901ceab
@antirez CONFIG REWRITE: when rewriting amount of bytes use GB, MB, KB if poss…
…ible.
2173f30
@antirez CONFIG REWRITE: bindaddr -> bind. 7d338fd
@antirez CONFIG REWRITE: backlog size is a bytes option. 604efb0
@antirez Use memtoll() when parsing the backlog-size option. 414d5db
@antirez CONFIG REWRITE: remove cluster stuff for 2.8 back port. 059018c
@antirez Don't stop reading redis.conf if line has no args.
Should be "continue" and was "return".

This fixes issue #1110
0b398b6
@antirez
Owner
antirez commented Dec 12, 2013

Hello @liexusong, I see that you moved the aeApiDelEvent() call before the loop to update the max fd. I'm not sure about why this is needed, please could you clarify? Thank you.

@liexusong
Contributor

Because the aeApiDelEvent() function using following code in the ae_epoll.c file:

int mask = eventLoop->events[fd].mask & (~delmask);

but eventLoop->events[fd].mask was changed by following code in the aeDeleteFileEvent() function:

fe->mask = fe->mask & (~mask);

so moved the aeApiDelEvent() call before the loop to update the max fd can avoid fe->mask = fe->mask & (~mask); affect.

@antirez
Owner
antirez commented Dec 12, 2013

Ok got it now, thanks. Are you aware of any practical effect of this bug?

@liexusong
Contributor

No practical effect but more stable.

@antirez
Owner
antirez commented Dec 12, 2013

Ok thank you again, I'll incorporate your change, have a nice day!

@mattsta mattsta added a commit to mattsta/redis that referenced this pull request Aug 2, 2014
@liexusong @mattsta liexusong + mattsta Remove redundant event loop fd processing b7e8e08
@mattsta mattsta added a commit to mattsta/redis that referenced this pull request Aug 2, 2014
@liexusong @mattsta liexusong + mattsta Remove redundant event loop fd processing 57de6c3
@mattsta mattsta referenced this pull request Aug 2, 2014
Closed

ALL simple issue fixes #1906

@mattsta mattsta added a commit to mattsta/redis that referenced this pull request Aug 6, 2014
@liexusong @mattsta liexusong + mattsta Remove redundant event loop fd processing 2fbd2fa
@antirez antirez added a commit that referenced this pull request Aug 8, 2014
@liexusong @antirez liexusong + Remove redundant event loop fd processing
Reason:
#1450 (comment)

Closes #1450
a8d3e93
@antirez antirez closed this in a8d3e93 Aug 25, 2014
@antirez antirez added a commit that referenced this pull request Aug 26, 2014
@liexusong @antirez liexusong + Remove redundant event loop fd processing
Reason:
#1450 (comment)

Closes #1450
6ef09f6
@antirez antirez added a commit that referenced this pull request Aug 27, 2014
@liexusong @antirez liexusong + Remove redundant event loop fd processing
Reason:
#1450 (comment)

Closes #1450
d1df50a
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment