Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

uncaught error: 348 trailing bytes #17

Closed
zacharynevin opened this issue Jul 13, 2014 · 33 comments
Closed

uncaught error: 348 trailing bytes #17

zacharynevin opened this issue Jul 13, 2014 · 33 comments

Comments

@zacharynevin
Copy link

I am using socket.io-emitter to broadcast an event to a set of channels with a for loop:

In the file, I have:

// in the file
var io = require('socket.io-emitter')({
  host: 'localhost',
  port: 6379
});

module.exports = {
    exampleFunction: function(req, res, next) {
      var channels = req.param('channels'),
            data = req.param('data');

      for (var i=0; i<channels.length; i++) {
        io.to(channels[i]).emit('example event', data)
      }
    }

}

In app.js, I have socket.io-redis:

io.adapter(socketio_redis({ 
  host: 'localhost', 
  port: 6379,
  pubClient: redis.createClient(6379, '127.0.0.1'),
  subClient: redis.createClient(6379, '127.0.0.1')
}))

When I try to run exampleFunction, I get the following uncaught error in my console:

Error: 348 trailing bytes
    at Object.decode (C:\Users\Zachary\Documents\GitHub\thegraduate_backend\node
_modules\socket.io-redis\node_modules\msgpack-js\msgpack.js:200:47)
    at Redis.onmessage (C:\Users\Zachary\Documents\GitHub\thegraduate_backend\no
de_modules\socket.io-redis\index.js:93:24)
    at RedisClient.EventEmitter.emit (events.js:106:17)
    at RedisClient.return_reply (C:\Users\Zachary\Documents\GitHub\thegraduate_b
ackend\node_modules\redis\index.js:672:22)
    at ReplyParser.<anonymous> (C:\Users\Zachary\Documents\GitHub\thegraduate_ba
ckend\node_modules\redis\index.js:309:14)
    at ReplyParser.EventEmitter.emit (events.js:95:17)
    at ReplyParser.send_reply (C:\Users\Zachary\Documents\GitHub\thegraduate_bac
kend\node_modules\redis\lib\parser\javascript.js:300:10)
    at ReplyParser.execute (C:\Users\Zachary\Documents\GitHub\thegraduate_backen
d\node_modules\redis\lib\parser\javascript.js:211:22)
    at RedisClient.on_data (C:\Users\Zachary\Documents\GitHub\thegraduate_backen
d\node_modules\redis\index.js:534:27)
    at Socket.<anonymous> (C:\Users\Zachary\Documents\GitHub\thegraduate_backend
\node_modules\redis\index.js:91:14)

When I change data to be something smaller, like:

var data = { testing: true }

I get the same error, but to a smaller number of trailing bytes. e.g. 95 trailing bytes.

@alexprice1
Copy link

So we have narrowed this down. https://github.com/automattic/socket.io-emitter#emitterclient-opts says to set return_buffers to true on your redis client. This should fix this issue.

We need to add support to socket.io-redis to see if the incoming/outcoming data is binary before sending it to msgpack.

@zacharynevin
Copy link
Author

It worked! Thank you!

dy-dx added a commit to SizzlingStats/sizzlingstats.com that referenced this issue Oct 23, 2014
vodolaz095 added a commit to vodolaz095/hunt that referenced this issue Oct 28, 2014
@hems
Copy link

hems commented Jun 28, 2015

maybe on the README we could "enforce" a bit more that parameter!

i also missed the detect_buffers: true on the "Custom Client" section

@nnmduc
Copy link

nnmduc commented Jul 21, 2015

I have same problem.
I use ruby socket emitter to broadcast an event:

emitter = SocketIO::Emitter.new(key: 'socket.io.dev', redis: Redis.new(url: 'redis://localhost:6379'))
data = {id: 100, username: 'peternguyen', content: Lorem Ipsum is simply dummy text of the printing and typesetting industry. Lorem Ipsum has been the industry's standard dummy text ever since the 1500s'}
emitter.emit('comment', data)

In app.js

var io = require('socket.io')(5001);
var request = require('request');
var redis = require('redis');
var redis_adapter = require('socket.io-redis');

var pub = redis.createClient(6379, 'localhost', {return_buffers: true, auth_pass: ''});
var sub = redis.createClient(6379, 'localhost', {return_buffers: true, auth_pass: ''});

io.adapter(redis_adapter({
    key: 'socket.io.dev',
    pubClient: pub,
    subClient: sub
}));

I get the following in my console:

events.js:72
        throw er; // Unhandled 'error' event
              ^
Error: -2340102607 trailing bytes
    at Object.decode (/home/peter/node_modules/socket.io-redis/node_modules/msgpack-js/msgpack.js:200:47)
    at Redis.onmessage (/home/peter/node_modules/socket.io-redis/index.js:94:24)
    at RedisClient.emit (events.js:106:17)
    at RedisClient.return_reply (/home/peter/node_modules/redis/index.js:696:22)
    at ReplyParser.<anonymous> (/home/peter/node_modules/redis/index.js:321:14)
    at ReplyParser.emit (events.js:95:17)
    at ReplyParser.send_reply (/home/peter/node_modules/redis/lib/parser/javascript.js:300:10)
    at ReplyParser.execute (/home/peter/node_modules/redis/lib/parser/javascript.js:211:22)
    at RedisClient.on_data (/home/peter/node_modules/redis/index.js:547:27)
    at Socket.<anonymous> (/home/peter/node_modules/redis/index.js:102:14)

@hems
Copy link

hems commented Jul 21, 2015

peter chexk the return_buffers option, that should sort your issue.

@Thesephi
Copy link

@peternguyen1991 please refer here: #61 (commits)

Your problem was that Ruby's SocketIO::Emitter packs the messages using the new msgpack-js v5, while the official socket.io-redis client only supports traditional msgpack-js. You could either clone from @burnssun's brilliant fork or manually edit your local socket.io-redis node module to reflect the necessary changes.

@aschmid
Copy link

aschmid commented Sep 22, 2015

i saw the trailing bytes issue the first time on my app today.
when i set the return buffers option i get this error

TypeError: Object socket.io#VZCddR has no method 'split'

  at Redis.onmessage (/path/to/my/app/node_modules/socket.io-redis/index.js:92:26)

is there a permanent fix available for all these issues?

@aschmid
Copy link

aschmid commented Sep 22, 2015

ok found the issue.
node redis v2.0.0 (released yesterday) has issues with the latest version of socket.io-redis causing the trailing bytes exeption.
when setting return_buffers to true unfortunately it throws an exception like in my previous comment.

@stephengardner
Copy link

@aschmid so is the solution to simply rollback one of these modules versions, or did you find another way?

@aschmid
Copy link

aschmid commented Sep 27, 2015

@stephengardner i didn't have time to look into a 'proper' solution yet so my fix was to use v1.0.0 of https://www.npmjs.com/package/redis.

@PerMoeller
Copy link

In my setup, node 4.1.1, socket-io 0.1.4, node_redis 2.0.1 the problem is that index.js line 92 expect "channel" to be a string, but it's a Buffer.

@Mokbii
Copy link

Mokbii commented Oct 2, 2015

set up redis ver. 1.0.0... is trailing bytes error fixed..

i think, that issue is node_redis 2.0.1 bug.

modify your file(pakage.json) redis version " 1.* " in dependencies

@zacharynevin
Copy link
Author

I am still getting this issue even when I set my redis version to 1.0.0

@zacharynevin
Copy link
Author

I used the same package versions as PerMoeller and got his exact issue:

channel.split("#") <!-- cannot read split of undefined

vodolaz095 added a commit to vodolaz095/hunt that referenced this issue Oct 17, 2015
@vodolaz095
Copy link

I'm still having this error with redis client of 2.2.3 and socket.io of 1.3.7, socket.io-redis:0.1.4, even with the detect_buffers and return_buffers set to true https://github.com/vodolaz095/hunt/blob/master/lib/datastore/redisClient.js#L44
.

/home/projects/vodolaz095/hunt/node_modules/socket.io-redis/node_modules/msgpack-js/msgpack.js:200
  if (decoder.offset !== buffer.length) throw new Error((buffer.length - decoder.offset) + " trailing bytes");
                                        ^

Error: 75 trailing bytes
    at Object.decode (/home/projects/vodolaz095/hunt/node_modules/socket.io-redis/node_modules/msgpack-js/msgpack.js:200:47)
    at Redis.onmessage (/home/projects/vodolaz095/hunt/node_modules/socket.io-redis/index.js:94:24)
    at emitThree (events.js:97:13)
    at RedisClient.emit (events.js:175:7)
    at RedisClient.return_reply (/home/projects/vodolaz095/hunt/node_modules/redis/index.js:622:22)
    at /home/projects/vodolaz095/hunt/node_modules/redis/index.js:310:18
    at doNTCallback0 (node.js:417:9)
    at process._tickDomainCallback (node.js:387:13)

npm ERR! Linux 4.1.8-100.fc21.x86_64
npm ERR! argv "/usr/bin/node" "/bin/npm" "start"
npm ERR! node v4.2.1
npm ERR! npm  v2.14.7
npm ERR! code ELIFECYCLE
npm ERR! hunt@0.6.0 start: `node examples/index.js`
npm ERR! Exit status 1
npm ERR! 
npm ERR! Failed at the hunt@0.6.0 start script 'node examples/index.js'.
npm ERR! This is most likely a problem with the hunt package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR!     node examples/index.js
npm ERR! You can get their info via:
npm ERR!     npm owner ls hunt
npm ERR! There is likely additional logging output above.

npm ERR! Please include the following file with any support request:
npm ERR!     /home/projects/vodolaz095/hunt/npm-debug.log

And it works ok with redis of 1.0.0.

@Caligone
Copy link

Same issue here with redis v2.2.5, socket.io v1.3.7 and socket.io-redis v0.1.4.

// Server
redis.createClient(redisOptions.port, redisOptions.host, {
      detect_buffers: true,
      return_buffers: true,
      auth_pass: redisOptions.password
    });
// Client
socket.emit('echo', {content: 'A'});

And I'm getting this error:

/app/node_modules/socket.io-redis/node_modules/msgpack-js/msgpack.js:200
  if (decoder.offset !== buffer.length) throw new Error((buffer.length - decoder.offset) + " trailing bytes");
                                        ^

Error: 86 trailing bytes
    at Object.decode (/app/node_modules/socket.io-redis/node_modules/msgpack-js/msgpack.js:200:47)
    at Redis.onmessage (/app/node_modules/socket.io-redis/index.js:94:24)
    at emitThree (events.js:97:13)
    at RedisClient.emit (events.js:175:7)
    at RedisClient.return_reply (/app/node_modules/redis/index.js:620:22)
    at /app/node_modules/redis/index.js:309:18
    at doNTCallback0 (node.js:417:9)
    at process._tickCallback (node.js:346:13)

Hope it helps

@austinh
Copy link

austinh commented Nov 12, 2015

I have this issue with the following versions

node: v4.2.1
socket.io: 1.37
socket.io-redis: 0.14
redis: 1.0 (also tried 2.0 and 2.3)

I am emitting an event with the socket.io-php-emitter with msgpack and redis

events.js:141
      throw er; // Unhandled 'error' event
      ^

Error: -842817745 trailing bytes
    at Object.decode (/var/www/website1/lib/vendor/node/node_modules/socket.io-redis/node_modules/msgpack-js/msgpack.js:200:47)
    at Redis.onmessage (/var/www/website1/lib/vendor/node/node_modules/socket.io-redis/index.js:94:24)
    at emitThree (events.js:97:13)
    at RedisClient.emit (events.js:175:7)
    at RedisClient.return_reply (/var/www/website1/lib/vendor/node/node_modules/socket.io-redis/node_modules/redis/index.js:672:22)
    at HiredisReplyParser.<anonymous> (/var/www/website1/lib/vendor/node/node_modules/socket.io-redis/node_modules/redis/index.js:309:14)
    at emitOne (events.js:77:13)
    at HiredisReplyParser.emit (events.js:169:7)
    at HiredisReplyParser.execute (/var/www/website1/lib/vendor/node/node_modules/socket.io-redis/node_modules/redis/lib/parser/hiredis.js:43:18)
    at RedisClient.on_data (/var/www/website1/lib/vendor/node/node_modules/socket.io-redis/node_modules/redis/index.js:534:27)

@austinh
Copy link

austinh commented Nov 13, 2015

Update: Switching dependenices from msgpack-js to msgpack-js-v5 fixed this problem for me. It looks like php's msgpack module has been updated to the spec, but the version used by msgpack has not.

@laukaichung
Copy link

Does this bug have anything to do with redis version after 3.0.6?
I have tried the suggestions in this post and downgraded all modules but the problem isn't fixed

@StevenRoan
Copy link

After several experiments, my service works fine with

    "socket.io": "1.3.7",
    "socket.io-redis": "0.1.4",

Just for reference

@terion-name
Copy link

Same error here. Nothing fixes :(
Nor return_buffers, nor version suggested by @StevenRoan

@xyzlast
Copy link

xyzlast commented Feb 25, 2016

I've same problem.

but the version suggested by @StevenRoan is works fine.

I've test it using socket.io-reids version is 1.0.0.

socket.io: 1.4.7
socket.io-redis: 1.0.0

case 1. detect_buffer: true, return_buffers: true

NoError. But Subscription is not working.

case 2. detect_buffer: true, return_buffers: false

message Pack Error.

if (decoder.offset !== buffer.length) throw new Error((buffer.length - decoder.offset) + " trailing bytes");

case 3. detect_buffer: false, return_buffers: true

Same case 2.

case 3. detect_buffer: false, return_buffers: false

Same case 2.

@xyzlast
Copy link

xyzlast commented Mar 2, 2016

Resolved it. It's my mistake.;;

detect_buffer, return_buffer options are not same in Pub and Sub. @StevenRoan, @terion-name check your options in Pub/Sub Clients.

here is my code that is working.

    "socket.io": "^1.4.5",
    "socket.io-redis": "^1.0.0",
function create(options) {
  if (config.redis.password) {
    options.auth_pass = config.redis.password;
  }
  const client = require('redis').createClient(config.redis.port, config.redis.host, options);
  const promise = require('bluebird');
  promise.promisifyAll(client);
  return client;
}

function createClient() {
  const options = {
    detect_buffers: true,
    return_buffers: false
  };
  return create(options);
}

function createPub() {
  const options = {
    detect_buffers: true,
    return_buffers: false
  };
  return create(options);
}

function createSub() {
  const options = {
    return_buffers: true
  };
  return create(options);
}

@FabioAntunes
Copy link

I'm also having the same issue, and it's very easy to replicate.

index.js:

var io = require('socket.io')(3000);
var redisIO = require('socket.io-redis');
var redis = require('redis');
var config = {
  redis: {
    host: '10.0.1.231',
    port: 6379
  },
  room: 'client1'
};

var pub = redis.createClient(config.redis.port, config.redis.host, {return_buffers: true});
var sub = redis.createClient(config.redis.port, config.redis.host, {return_buffers: true});

io.adapter(redisIO({
  host: config.redis.host,
  port: config.redis.port,
  pubClient: pub,
  subClient: sub
}));

io.on('connection', function(socket) {
  console.log('connected');
  socket.on('getMessages', function(){
    console.log('published');
    pub.publish('coolMessages', 'yolo message');
  });
});

sub.subscribe('coolMessages');

sub.on('message', function(channel, data){
  console.log('message!'); // this doesn't run
});

package.json:

{
  "name": "test_dm",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "author": "",
  "license": "MIT",
  "dependencies": {
    "redis": "^2.3.0",
    "socket.io": "^1.4.5",
    "socket.io-redis": "^1.0.0"
  }
}

just creat these two files, do a npm install.

Then for the client:
html

<!doctype html>
<html lang="en">
<head>
  <meta charset="UTF-8">
  <title>Socket.IO Chat Example</title>
  <link rel="stylesheet" href="style.css">
</head>
<body>
  <ul class="pages">
    <li class="chat page">
      <div class="chatArea">
        <ul class="messages"></ul>
      </div>
      <input class="inputMessage" placeholder="Type here..."/>
    </li>
    <li class="login page">
      <div class="form">
        <h3 class="title">What's your nickname?</h3>
        <input class="usernameInput" type="text" maxlength="14" />
      </div>
    </li>
  </ul>

  <script src="https://code.jquery.com/jquery-1.10.2.min.js"></script>
  <script src="bower_components/socket.io-client/socket.io.js"></script>
  <script>
    $(function() {
      var socket = io('http://localhost:3000');

      socket.on('connect', function () {
        console.log(socket.id);
      });

      socket.emit('getMessages')
    });
  </script>
</body>
</html>

bower.json

{
  "name": "socket-test",
  "description": "",
  "main": "",
  "moduleType": [],
  "license": "MIT",
  "homepage": "",
  "ignore": [
    "**/.*",
    "node_modules",
    "bower_components",
    "test",
    "tests"
  ],
  "dependencies": {
    "socket.io-client": "^1.4.5"
  }
}

Just create also these two files and to a bower install.

@agix
Copy link

agix commented Mar 18, 2016

Hi !
Actually the bug exists if you use the same redis client in socket.io-redis and in a "classic" usage.
Just create two more redisClient to pub and sub.

var ioPub = redis.createClient(redisPort, redisHost, {detect_buffers: true, return_buffers: false});
var ioSub = redis.createClient(redisPort, redisHost, {return_buffers: true});

var pub = redis.createClient(redisPort, redisHost, {detect_buffers: true, return_buffers: false});
var sub = redis.createClient(redisPort, redisHost, {detect_buffers: true, return_buffers: false});

io.adapter(redisIO({
  host: config.redis.host,
  port: config.redis.port,
  pubClient: ioPub,
  subClient: ioSub
}));

io.on('connection', function(socket) {
  console.log('connected');
  socket.on('getMessages', function(){
    console.log('published');
    pub.publish('coolMessages', 'yolo message');
  });
});

sub.subscribe('coolMessages');

sub.on('message', function(channel, data){
  console.log('message!'); // this doesn't run
});

@Audio
Copy link

Audio commented Mar 23, 2016

I had this issue with redis@2.4.2, but it seem to be okay after upgrading to v2.5.0

https://github.com/NodeRedis/node_redis/releases

@ucola
Copy link

ucola commented May 13, 2016

I have the same error with version
redis@2.6.0-2
socket.io@1.4.6

/home/*/npm_modules/node_modules/socket.io-redis/node_modules/msgpack-js/msgpack.js:200
if (decoder.offset !== buffer.length) throw new Error((buffer.length - decoder.offset) + " trailing bytes");

var redis = require('/home/' + _servername + '/npm_modules/node_modules/redis').createClient;
var adapter = require('/home/' + _servername + '/npm_modules/node_modules/socket.io-redis');
var pub = redis(_redisport, _redishost, { auth_pass: _redispass });
var sub = redis(_redisport, _redishost, { detect_buffers: true, auth_pass: _redispass });
io.adapter(adapter({ pubClient: pub, subClient: sub }));

did anybody have a solution for this issue?
with redis@1.0.0 and socket.io@1.3.4 it works fine

@austinkelleher
Copy link

+1 I am experiencing this issue as well

@garfieldhamilton
Copy link

I was able to install the v5 version of msgpack-js-v5,
npm i msgpack-js-v5 --save

then change the requires statement in node_modules/socket.io-redis/index.js (around line 8) from
var msgpack = require('msgpack-js');
to
var msgpack = require('msgpack-js-v5');

and that worked in my setup.

@ucola
Copy link

ucola commented Jul 19, 2016

@garfieldhamilton can you tel us the version of redis and socket.io that you use?

@CostaRico
Copy link

+1

@mjasnikovs
Copy link

@garfieldhamilton msgpack-js-v5 is not passing tests, use msgpack5

@blalop
Copy link

blalop commented Jun 10, 2021

For those still experimenting this issue (I truly hope no one is still using these versions) the following versions made the fix for me:

        "redis": "1.0.0",
        "socket.io": "3.0.0",
        "socket.io-redis": "4.0.1",

No additional configuration required.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests