Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Loading…

False "too big bulk count string" for big HMSET #673

Open
piotr-piatkowski opened this Issue · 3 comments

3 participants

Piotr Piatkowski Salvatore Sanfilippo Bruno Renié
Piotr Piatkowski

There's a bug in processMultibulkBuffer in networking.c (v.2.4.17) - if you get buffer ending exactly before next bulk string count, like here:

*7
$5
HMSET
$1
x
$10
abcdefghij
[more data to follow in next call]

...then in the "while(c->multibulklen)" loop redis will read data for third arg, and then try to read bulk count for the fourth one having no data in buffer, and instead of returning from method it will signal "too big bulk count string".

This happens randomly on some specific data with big hash with big fields, I have data which triggers this error very often, but cannot provide it as it contains confidential data (it was taken from production server). I tried to generate such data manually using perl script, but with no effect - although data looked very similar, it didn't trigger an error.

Nevertheless, I saw many posts in the internet with similar problem so I think it happens quite often - and there's very simple fix I did:

diff --git a/src/networking.c b/src/networking.c
index 500df0d..fb9c05e 100644
--- a/src/networking.c
+++ b/src/networking.c
@@ -812,7 +812,7 @@ int processMultibulkBuffer(redisClient *c) {
     }

     redisAssert(c->multibulklen > 0);
-    while(c->multibulklen) {
+    while(c->multibulklen && sdslen(c->querybuf)-pos > 2) {
         /* Read bulk length if unknown */
         if (c->bulklen == -1) {
             newline = strchr(c->querybuf+pos,'\r');

So if you have an idea how to replicate this problem you can add some test case for your test suite, and either way you could include this fix in next stable release.

I checked current unstable branch, and it looks it was fixed somehow - in any case I wasn't able to replicate an error.

Salvatore Sanfilippo
Owner

Hello, this is going to be my priority tomorrow morning, thanks.

Bruno Renié

What's the status of this? For the record I was having this exact issue (Protocol error on specific large HMSETs) with redis-py/rq and found out I was still running redis 2.4. After upgrading to 2.6 the issue completely disappeared.

Piotr Piatkowski

For me it also disappeared in 2.6 - so it looks as fixed, only this issue wasn't closed.

B. W. Lewis bwlewis referenced this issue in bwlewis/rredis
Closed

storing large objects #6

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.