rredis seems to be ok for small sized, data, but when I try to store a large object the connection seems to die:
> redisSet("test", rnorm(1e8))
Error in doTryCatch(return(expr), name, parentenv, handler) :
ERR Protocol error: invalid bulk length
In addition: Warning messages:
1: In writeBin(v, con) : problem writing to connection
2: In writeBin(.raw("\r\n"), con) : problem writing to connection
Any idea what is going on?
I am having the same problem as jeroenooms. Did you ever manage to find a solution to this or is it still unsolved?
Hi! Redis values are limited to 512MB, see for example:
Any R object exceeding this length will result in an invalid bulk length error.
There was a bug in Redis affecting HMSET that could result in this kind of error happening randomly. it was fixed months ago. Here is a reference: antirez/redis#673
The upshot is, if you need to store large objects in redis, you'll have to compile a custom redis server.
Thank you. You say that to store objects larger than 512MB I need to compile a custom redis server, is this something I can change in the redis.conf file? Can you point me in the direction of what I need to change please?
This is actually an open bug in Redis. Unfortunately, it's not a configuration option. And it appears that even changing the source code of Redis to support this is trickier than I thought. See the following bug thread for a discussion on this topic:
So it looks like 512MB will be the limit for a while yet.