Merge 6.2 RC2

00-RELEASENOTES

@@ -1,8 +1,84 @@
+================================================================================
+Redis 6.2 RC2   Released Tue Jan 12 16:17:20 IST 2021
+================================================================================
+
+Upgrade urgency LOW: This is the second Release Candidate of Redis 6.2.
+
+IMPORTANT: If you're running Redis on ARM64 or a big-endian system, upgrade may
+have significant implications. Please be sure to read the notes below.
+
+Here is a comprehensive list of changes in this release compared to 6.2 RC1,
+each one includes the PR number that added it, so you can get more details
+at https://github.com/redis/redis/pull/<number>
+
+New commands / args:
+* Add the REV, BYLEX and BYSCORE arguments to ZRANGE, and the ZRANGESTORE command (#7844)
+* Add the XAUTOCLAIM command (#7973)
+* Add the MINID trimming strategy and the LIMIT argument to XADD and XTRIM (#8169)
+* Add the ANY argument to GEOSEARCH and GEORADIUS (#8259)
+* Add the CH, NX, XX arguments to GEOADD (#8227)
+* Add the COUNT argument to LPOP and RPOP (#8179)
+* Add the WRITE argument to CLIENT PAUSE for pausing write commands exclusively (#8170)
+* Change the proto-ver argument of HELLO to optional (#7377)
+* Add the CLIENT TRACKINGINFO subcommand (#7309)
+
+Command behavior changes:
+* CLIENT TRACKING yields an error when given overlapping BCAST prefixes (#8176)
+* SWAPDB invalidates WATCHed keys (#8239)
+* SORT command behaves differently when used on a writable replica (#8283)
+
+Other behavior changes:
+* Avoid propagating MULTI/EXEC for read-only transactions (#8216)
+* Remove the read-only flag from TIME, ECHO, ROLE, LASTSAVE (#8216)
+* Fix the command flags of PFDEBUG (#8222)
+* Tracking clients will no longer receive unnecessary key invalidation messages after FLUSHDB (#8039)
+* Sentinel: Fix missing updates to the config file after SENTINEL SET command (#8229)
+
+Bug fixes with compatibility implications (bugs introduced in Redis 6.0):
+* Fix RDB CRC64 checksum on big-endian systems (#8270)
+  If you're using big-endian please consider the compatibility implications with
+  RESTORE, replication and persistence.
+* Fix wrong order of key/value in Lua's map response (#8266)
+  If your scripts use redis.setresp() or return a map (new in Redis 6.0), please
+  consider the implications.
+
+Bug fixes that are only applicable to previous releases of Redis 6.2:
+* Resolve rare assertions in active defragmentation while loading (#8284, #8281)
+
+Bug fixes:
+* Fix the selection of a random element from large hash tables (#8133)
+* Fix an issue where a forked process deletes the parent's pidfile (#8231)
+* Fix crashes when enabling io-threads-do-reads (#8230)
+* Fix a crash in redis-cli after executing cluster backup (#8267)
+* Fix redis-benchmark to use an IP address for the first cluster node (#8154)
+* Fix saving of strings larger than 2GB into RDB files (#8306)
+
+Additional improvements:
+* Improve replication handshake time (#8214)
+* Release client tracking table memory asynchronously in cases where the DB is also freed asynchronously (#8039)
+* Avoid wasteful transient memory allocation in certain cases (#8286, #5954)
+* Handle binary string values by the 'requirepass' and 'masterauth' configs (#8200)
+
+Platform and deployment-related changes:
+* Install redis-check-rdb and redis-check-aof as symlinks to redis-server (#5745)
+* Add a check for an ARM64 Linux kernel bug (#8224)
+  Due to the potential severity of this issue, Redis will refuse to run on
+  affected platforms by default.
+
+Info fields and introspection changes:
+* Add the errorstats section to the INFO command (#8217)
+* Add the failed_calls and rejected_calls fields INFO's commandstats section (#8217)
+* Report child copy-on-write metrics continuously (#8264)
+
+Module API changes:
+* Add the RedisModule_SendChildCOWInfo API (#8264)
+* Add the may-replicate command flag (#8170)
+
 ================================================================================
 Redis 6.2 RC1   Released Mon Dec 14 11:50:00 IST 2020
 ================================================================================
 
-Upgrade urgency LOW: This is the first RC of Redis 6.2.
+Upgrade urgency LOW: This is the first Release Candidate of Redis 6.2.
 
 Introduction to the Redis 6.2 release
 =====================================
@@ -29,7 +105,7 @@ New commands / args:
 * Add SMISMEMBER command that checks multiple members (#7615)
 * Add ZMSCORE command that returns an array of scores (#7593)
 * Add LMOVE and BLMOVE commands that pop and push arbitrarily (#6929)
-* Add RESET command that resets client connection state(#7982)
+* Add RESET command that resets client connection state (#7982)
 * Add COPY command that copies keys (#7953)
 * Add ZDIFF and ZDIFFSTORE commands (#7961)
 * Add ZINTER and ZUNION commands (#7794)
@@ -146,7 +222,7 @@ Platform / toolchain support related improvements:
 
 New configuration options:
 * Enable configuring OpenSSL using the standard openssl.cnf (#8143)
-* oom-score-adj-values can config can now take absolute values (besides relative ones) (#8046)
+* oom-score-adj-values config can now take absolute values (besides relative ones) (#8046)
 * TLS: Add different client cert support. (#8076)
 * Note that a few other changes listed above added their config options.
 
@@ -177,31 +253,42 @@ We'll follow up with more RC releases, until the code looks production ready
 and we don't get reports of serious issues for a while.
 
 A special thank you for the amount of work put into this release by:
+- Oran Agra
+- Yossi Gottlieb
+- Itamar Haber
+- Guy Benoish
+- Filipe Oliveira
+- Madelyn Olson
 - Wang Yuan
 - Felipe Machado
-- Filipe Oliveira
-- Tatsuya Arisawa
 - Yang Bodong
+- Tatsuya Arisawa
+- Jonah H. Harris
 - Jim Brunner
+- Yaacov Hazan
 - Wen Hui
-- Guy Benoish
 - Chen Yang
 - Nitai Caro
 - Meir Shpilraien
-- Madelyn Olson
 - maohuazhu
 - Valentino Geron
+- Qu Chen
+- sundb
+- George Prekas
+- Zhao Zhao
 - Tyson Andre
 - Michael Grunder
 - alexronke-channeladvisor
-- sundb
-- WuYunlong
+- Wu Yunlong
+- Wei Kukey
 - Yoav Steinberg
 - Uri Shachar
+- Greg Femec
 - Nykolas Laurentino de Lima
+- xhe
+- zhenwei pi
 - David CARLIER
 
-
 Migrating from 6.0 to 6.2
 =========================
 

deps/hiredis/async_private.h

@@ -51,7 +51,7 @@
 #define _EL_CLEANUP(ctx) do { \
         if ((ctx)->ev.cleanup) (ctx)->ev.cleanup((ctx)->ev.data); \
         ctx->ev.cleanup = NULL; \
-    } while(0);
+    } while(0)
 
 static inline void refreshTimeout(redisAsyncContext *ctx) {
     #define REDIS_TIMER_ISSET(tvp) \

redis.conf

@@ -196,9 +196,12 @@ tcp-keepalive 300
 #
 # tls-cluster yes
 
-# Explicitly specify TLS versions to support. Allowed values are case insensitive
-# and include "TLSv1", "TLSv1.1", "TLSv1.2", "TLSv1.3" (OpenSSL >= 1.1.1) or
-# any combination. To enable only TLSv1.2 and TLSv1.3, use:
+# By default, only TLSv1.2 and TLSv1.3 are enabled and it is highly recommended
+# that older formally deprecated versions are kept disabled to reduce the attack surface.
+# You can explicitly specify TLS versions to support.
+# Allowed values are case insensitive and include "TLSv1", "TLSv1.1", "TLSv1.2",
+# "TLSv1.3" (OpenSSL >= 1.1.1) or any combination.
+# To enable only TLSv1.2 and TLSv1.3, use:
 #
 # tls-protocols "TLSv1.2 TLSv1.3"
 
@@ -688,7 +691,7 @@ replica-priority 100
 
 # Redis implements server assisted support for client side caching of values.
 # This is implemented using an invalidation table that remembers, using
-# 16 millions of slots, what clients may have certain subsets of keys. In turn
+# a radix key indexed by key name, what clients have which keys. In turn
 # this is used in order to send invalidation messages to clients. Please
 # check this page to understand more about the feature:
 #
@@ -1973,3 +1976,10 @@ jemalloc-bg-thread yes
 #
 # Set bgsave child process to cpu affinity 1,10,11
 # bgsave_cpulist 1,10-11
+
+# In some cases redis will emit warnings and even refuse to start if it detects
+# that the system is in bad state, it is possible to suppress these warnings
+# by setting the following config which takes a space delimited list of warnings
+# to suppress
+#
+# ignore-warnings ARM64-COW-BUG

src/Makefile

@@ -115,8 +115,18 @@ else
 ifeq ($(uname_S),Darwin)
 	# Darwin
 	FINAL_LIBS+= -ldl
+	# Homebrew's OpenSSL is not linked to /usr/local to avoid
+	# conflicts with the system's LibreSSL installation so it
+	# must be referenced explicitly during build.
+ifeq ($(uname_M),arm64)
+	# Homebrew arm64 uses /opt/homebrew as HOMEBREW_PREFIX
+	OPENSSL_CFLAGS=-I/opt/homebrew/opt/openssl/include
+	OPENSSL_LDFLAGS=-L/opt/homebrew/opt/openssl/lib
+else
+	# Homebrew x86/ppc uses /usr/local as HOMEBREW_PREFIX
 	OPENSSL_CFLAGS=-I/usr/local/opt/openssl/include
 	OPENSSL_LDFLAGS=-L/usr/local/opt/openssl/lib
+endif
 else
 ifeq ($(uname_S),AIX)
         # AIX
@@ -260,11 +270,11 @@ endif
 
 REDIS_SERVER_NAME=redis-server$(PROG_SUFFIX)
 REDIS_SENTINEL_NAME=redis-sentinel$(PROG_SUFFIX)
-REDIS_SERVER_OBJ=adlist.o quicklist.o ae.o anet.o dict.o server.o sds.o zmalloc.o lzf_c.o lzf_d.o pqsort.o zipmap.o sha1.o ziplist.o release.o networking.o util.o object.o db.o replication.o rdb.o t_string.o t_list.o t_set.o t_zset.o t_hash.o config.o aof.o pubsub.o multi.o debug.o sort.o intset.o syncio.o cluster.o crc16.o endianconv.o slowlog.o scripting.o bio.o rio.o rand.o memtest.o crcspeed.o crc64.o bitops.o sentinel.o notify.o setproctitle.o blocked.o hyperloglog.o latency.o sparkline.o redis-check-rdb.o redis-check-aof.o geo.o lazyfree.o module.o evict.o expire.o geohash.o geohash_helper.o childinfo.o defrag.o siphash.o rax.o t_stream.o listpack.o localtime.o lolwut.o lolwut5.o lolwut6.o acl.o gopher.o tracking.o connection.o tls.o sha256.o timeout.o setcpuaffinity.o monotonic.o
+REDIS_SERVER_OBJ=adlist.o quicklist.o ae.o anet.o dict.o server.o sds.o zmalloc.o lzf_c.o lzf_d.o pqsort.o zipmap.o sha1.o ziplist.o release.o networking.o util.o object.o db.o replication.o rdb.o t_string.o t_list.o t_set.o t_zset.o t_hash.o config.o aof.o pubsub.o multi.o debug.o sort.o intset.o syncio.o cluster.o crc16.o endianconv.o slowlog.o scripting.o bio.o rio.o rand.o memtest.o crcspeed.o crc64.o bitops.o sentinel.o notify.o setproctitle.o blocked.o hyperloglog.o latency.o sparkline.o redis-check-rdb.o redis-check-aof.o geo.o lazyfree.o module.o evict.o expire.o geohash.o geohash_helper.o childinfo.o defrag.o siphash.o rax.o t_stream.o listpack.o localtime.o lolwut.o lolwut5.o lolwut6.o acl.o gopher.o tracking.o connection.o tls.o sha256.o timeout.o setcpuaffinity.o monotonic.o mt19937-64.o
 REDIS_CLI_NAME=redis-cli$(PROG_SUFFIX)
-REDIS_CLI_OBJ=anet.o adlist.o dict.o redis-cli.o zmalloc.o release.o ae.o crcspeed.o crc64.o siphash.o crc16.o monotonic.o cli_common.o
+REDIS_CLI_OBJ=anet.o adlist.o dict.o redis-cli.o zmalloc.o release.o ae.o crcspeed.o crc64.o siphash.o crc16.o monotonic.o cli_common.o mt19937-64.o
 REDIS_BENCHMARK_NAME=redis-benchmark$(PROG_SUFFIX)
-REDIS_BENCHMARK_OBJ=ae.o anet.o redis-benchmark.o adlist.o dict.o zmalloc.o release.o crcspeed.o crc64.o siphash.o crc16.o monotonic.o cli_common.o
+REDIS_BENCHMARK_OBJ=ae.o anet.o redis-benchmark.o adlist.o dict.o zmalloc.o release.o crcspeed.o crc64.o siphash.o crc16.o monotonic.o cli_common.o mt19937-64.o
 REDIS_CHECK_RDB_NAME=redis-check-rdb$(PROG_SUFFIX)
 REDIS_CHECK_AOF_NAME=redis-check-aof$(PROG_SUFFIX)
 
@@ -336,7 +346,7 @@ $(REDIS_CLI_NAME): $(REDIS_CLI_OBJ)
 $(REDIS_BENCHMARK_NAME): $(REDIS_BENCHMARK_OBJ)
 	$(REDIS_LD) -o $@ $^ ../deps/hiredis/libhiredis.a ../deps/hdr_histogram/hdr_histogram.o $(FINAL_LIBS)
 
-dict-benchmark: dict.c zmalloc.c sds.c siphash.c
+dict-benchmark: dict.c zmalloc.c sds.c siphash.c mt19937-64.c
 	$(REDIS_CC) $(FINAL_CFLAGS) $^ -D DICT_BENCHMARK_MAIN -o $@ $(FINAL_LIBS)
 
 DEP = $(REDIS_SERVER_OBJ:%.o=%.d) $(REDIS_CLI_OBJ:%.o=%.d) $(REDIS_BENCHMARK_OBJ:%.o=%.d)
@@ -405,8 +415,8 @@ install: all
 	$(REDIS_INSTALL) $(REDIS_SERVER_NAME) $(INSTALL_BIN)
 	$(REDIS_INSTALL) $(REDIS_BENCHMARK_NAME) $(INSTALL_BIN)
 	$(REDIS_INSTALL) $(REDIS_CLI_NAME) $(INSTALL_BIN)
-	$(REDIS_INSTALL) $(REDIS_CHECK_RDB_NAME) $(INSTALL_BIN)
-	$(REDIS_INSTALL) $(REDIS_CHECK_AOF_NAME) $(INSTALL_BIN)
+	@ln -sf $(REDIS_SERVER_NAME) $(INSTALL_BIN)/$(REDIS_CHECK_RDB_NAME)
+	@ln -sf $(REDIS_SERVER_NAME) $(INSTALL_BIN)/$(REDIS_CHECK_AOF_NAME)
 	@ln -sf $(REDIS_SERVER_NAME) $(INSTALL_BIN)/$(REDIS_SENTINEL_NAME)
 
 uninstall:

src/acl.c

@@ -174,15 +174,15 @@ sds ACLHashPassword(unsigned char *cleartext, size_t len) {
     return sdsnewlen(hex,HASH_PASSWORD_LEN);
 }
 
-/* Given a hash and the hash length, returns C_OK if it is a valid password 
+/* Given a hash and the hash length, returns C_OK if it is a valid password
  * hash, or C_ERR otherwise. */
 int ACLCheckPasswordHash(unsigned char *hash, int hashlen) {
     if (hashlen != HASH_PASSWORD_LEN) {
-        return C_ERR;      
+        return C_ERR;
     }
- 
+
     /* Password hashes can only be characters that represent
-     * hexadecimal values, which are numbers and lowercase 
+     * hexadecimal values, which are numbers and lowercase
      * characters 'a' through 'f'. */
     for(int i = 0; i < HASH_PASSWORD_LEN; i++) {
         char c = hash[i];
@@ -2184,18 +2184,30 @@ void aclCommand(client *c) {
         }
     } else if (c->argc == 2 && !strcasecmp(sub,"help")) {
         const char *help[] = {
-"LOAD                             -- Reload users from the ACL file.",
-"SAVE                             -- Save the current config to the ACL file.",
-"LIST                             -- Show user details in config file format.",
-"USERS                            -- List all the registered usernames.",
-"SETUSER <username> [attribs ...] -- Create or modify a user.",
-"GETUSER <username>               -- Get the user details.",
-"DELUSER <username> [...]         -- Delete a list of users.",
-"CAT                              -- List available categories.",
-"CAT <category>                   -- List commands inside category.",
-"GENPASS [<bits>]                 -- Generate a secure user password.",
-"WHOAMI                           -- Return the current connection username.",
-"LOG [<count> | RESET]            -- Show the ACL log entries.",
+"CAT [<category>]",
+"    List all commands that belong to <category>, or all command categories",
+"    when no category is specified.",
+"DELUSER <username> [<username> ...]",
+"    Delete a list of users.",
+"GETUSER <username>",
+"    Get the user's details.",
+"GENPASS [<bits>]",
+"    Generate a secure 256-bit user password. The optional `bits` argument can",
+"    be used to specify a different size.",
+"LIST",
+"    Show users details in config file format.",
+"LOAD",
+"    Reload users from the ACL file.",
+"LOG [<count> | RESET]",
+"    Show the ACL log entries.",
+"SAVE",
+"    Save the current config to the ACL file.",
+"SETUSER <username> <attribute> [<attribute> ...]",
+"    Create or modify a user with the specified attributes.",
+"USERS",
+"    List all the registered usernames.",
+"WHOAMI",
+"    Return the current connection username.",
 NULL
         };
         addReplyHelp(c,help);
@@ -2224,7 +2236,7 @@ void addReplyCommandCategories(client *c, struct redisCommand *cmd) {
 void authCommand(client *c) {
     /* Only two or three argument forms are allowed. */
     if (c->argc > 3) {
-        addReply(c,shared.syntaxerr);
+        addReplyErrorObject(c,shared.syntaxerr);
         return;
     }