Bad decoding for languages other than English

[Cyperwu]

Hi!

In line 767, var buf = client.inbound.toString('binary', 0, MAX_CONTROL_LINE_SIZE);
encoding error will occur when using other languages.

---

I tried change 'binary' to 'utf8' in line 767 and add a function getting binary length from utf8 string:

function byteLength(str) {
  // returns the byte length of an utf8 string
  var s = str.length;
  for (var i=str.length-1; i>=0; i--) {
    var code = str.charCodeAt(i);
    if (code > 0x7f && code <= 0x7ff) s++;
    else if (code > 0x7ff && code <= 0xffff) s+=2;
    if (code >= 0xDC00 && code <= 0xDFFF) i--; //trail surrogate
  }
  return s;
}

Then change line 936 var psize = m[0].length; to var psize = byteLength(m[0]), my problem seems solved.

[aricart]

@Cyperwu Thanks for the report. Could you be so kind as to send us a sample input (subject) and payload that we can take a look at? Thanks!

[aricart]

@Cyperwu I am guessing here that the subject is not ASCII, and likely it should be. While the payload can be anything, the NATS protocol must be ASCII. Can you confirm this is the case?

[Cyperwu]

@aricart No the subject is not ASCII. But the NATS documentation didn't specify any encoding is a "must use" in subjects.

Oh, Sorry. I've seen it.

[Cyperwu]

@aricart tried to directly subscribe to a non-alphanumeric subject and received no message.
But in fact if we use wildcard to match the rest part of the subject, even non-alphanumeric subjects are valid.

PUB mainChannel.噢.额 3 44\r\n
Hello\r\n\r\n

SUB mainChannel.> 3\r\n\r\n will still get me messages

[isobit]

Perhaps an issue should be filed with gnatsd requesting support for UTF-8 topic names? If the NATS protocol requires that topics are ASCII-encoded then it seems like this is a non-issue for client libs.

[wallyqs]

A reproducible example which makes it fail might be nice, tried out of curiosity and there does not seem to be an issue receiving the message when using non-ASCII characters in the protocol line either.

asyncio-nats (master) $ python examples/nats-sub ">"
Connected to NATS at 127.0.0.1:4222...
Received a message on 'hello.日本語 ': hi

node-nats (master) $ node examples/node-sub ">" 
Listening on [>]
Received "hi"

ruby-nats (master) $ ruby bin/nats-pub "hello.日本語" hi
Published [hello.日本語] : 'hi'

[Cyperwu]

@isobit I agree.

[Cyperwu]

@wallyqs What if you try node examples/node-sub "hello.日本語 " ?

[wallyqs]

@Cyperwu that works for me locally too...

node examples/node-sub "hello.日本語 "
Listening on [hello.日本語 ]
Received "中文"

node examples/node-pub "hello.日本語 " 中文
Published [hello.日本語 ] : "中文"

[Cyperwu]

@wallyqs In my case it is Received "中文" from "hello.æ�¥æ�¬èª"

[aricart]

I also get:

> ~/go/src/git...e-nats/examples]$ node-sub "hello.日本語 "
Listening on [hello.日本語 ]
Received "中文"

@Cyperwu what OS are you running in?

[Cyperwu]

That's odd. My machine is MacOS X El Capitan 10.11.5. @aricart

[aricart]

v2 doesn't deal with any encoding messages are simply byte arrays.