Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Already on GitHub? Sign in to your account

PHP compressed value can't process #173

Open
ghost opened this Issue Jan 23, 2014 · 4 comments

Comments

Projects
None yet
3 participants
@ghost

ghost commented Jan 23, 2014

Hi,

We have faced an issue when the key is stored via php with compression it can't process in memcached module.

php code: Memcache::set ( "SampleKey" , "Sample value" , MEMCACHE_COMPRESSED , 2000);

undefined:1
x����N� �
^
SyntaxError: Unexpected token x
at Object.parse (native)
at Socket.value (/usr/local/lib/node_modules/memcached/lib/memcached.js:497:26)
at Client.rawDataReceived (/usr/local/lib/node_modules/memcached/lib/memcached.js:708:51)
at Client.BufferBuffer (/usr/local/lib/node_modules/memcached/lib/memcached.js:654:12)
at Socket.bowlofcurry (/usr/local/lib/node_modules/memcached/lib/utils.js:108:15)
at Socket.EventEmitter.emit (events.js:95:17)
at Socket. (stream_readable.js:746:14)
at Socket.EventEmitter.emit (events.js:92:17)
at emitReadable
(_stream_readable.js:408:10)
at emitReadable (_stream_readable.js:404:5

Owner

3rd-Eden commented Jan 23, 2014

The module doesn't support compressed values as it has no logic to unzip the compressed information.

@ghost

ghost commented Jan 23, 2014

Is there any option to compress.decompress the data using node memcached module instead of storing raw data ?

Contributor

rcoup commented Feb 15, 2015

@kazzalib unfortunately, each memcached client tends to use the type flags in the protocol differently, so what the PHP library interprets as a zipped string node-memcached might interpret as an integer and python's memcached library might interpret as a double and so on. String is the only type that is consistent in my experience.

So, one option is to pre-gzip your data (might need to base64 it too) in PHP and store it as a string to memcached? Then you'll get a string back in node and can un-gzip it.

avin45h commented Aug 27, 2015

I was able to fix a similar issue. We were facing same kind of php compressed data. Have a look at the diff file below, (It may sound like a nasty fix with a lot of side effects, but works like a charm in our case). We were unable to change original php code due to backward compatibility limitations with existing php code base. Works good for both case, compressed and non compressed.

diff --git a/node_modules/memcached/lib/memcached.js b/node_modules/memcached/lib/memcached.js
index d699f2b..141fa94 100644
--- a/node_modules/memcached/lib/memcached.js
+++ b/node_modules/memcached/lib/memcached.js
@@ -4,7 +4,8 @@
  * Node's native modules
  */
 var Stream = require('net').Stream
-  , Socket = require('net').Socket;
+  , Socket = require('net').Socket
+  , zlib = require('zlib');

 /**
  * External or custom modules
@@ -176,7 +177,7 @@ Client.config = {
       S.streamID = sid++;
       S.setTimeout(memcached.timeout);
       S.setNoDelay(true);
-      S.setEncoding('utf8');
+      S.setEncoding('binary');
       S.metaData = [];
       S.responseBuffer = "";
       S.bufferArray = [];
@@ -728,7 +729,12 @@ Client.config = {

         // fetch the response content
         if (tokenSet[0] === 'VALUE') {
-          dataSet = Utils.unescapeValue(S.bufferArray.shift());
+          var value = S.bufferArray.shift();
+          try{
+            value = zlib.inflateSync(new Buffer(value, "binary")).toString()
+          }catch(e){
+          }
+          dataSet = Utils.unescapeValue(value);
         }

         resultSet = privates.parsers[tokenSet[0]].call(S, tokenSet, dataSet || token, err, queue, this);

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment