Under Ruby 1.9, received messages are encoded as US-ASCII

[ibc]

ws.onmessage do |msg|
  puts msg.encoding
end

It produces "US-ASCII" even when using Ruby 1.9. It would be much better to produce a UTF-8 string (as WebSocket protocol mandates text messages under UTF-8).

[ibc]

Even worse:

ws.onmessage do |msg|
  ws.send "iñaki"
end

And it just sends "iñak". And for enxt message it produces an error in the JS side:

One or more reserved bits are on: reserved1 = 1, reserved2 = 1, reserved3 = 0

probably due to the remaining byte.

[ibc]

IMHO the problem is here:

https://github.com/igrigorik/em-websocket/blob/master/lib/em-websocket/framing07.rb#L121

length = application_data.size

Under Ruby 1.9, "iñaki".size returns 5, but using UTF-8 there are 6 bytes. Use String#bytesize:

length = (application_data.respond_to?(:bytesize) ? application_data.bytesize : application_data.size)

This solves the issue.

[mloughran]

Thanks for reporting this - responding to each part separately

onmessage returns US-ASCII string:

• This happened for drafts > 76, and I've just pushed a fix for it

sending non-ascii data caused js error:

• I can't replicate this. Can you let me know which browser version you're seeing this in?

application_data.size hunch:

• This is fine because the data gets converted to BINARY encoding in Connection#send, and therefore size returns the same as bytesize. However I agree that bytesize is probably clearer, but I'm loath to change it because size it used right through the framing code...

[ibc]

The problem with the non-ascii data is that (before current fix I hope) the frame size said 10 but the payload size was 11 (for example) due to the usage of String#size (which in Ruby 1.9 should be String#bytesize). So the JS client keeps a buffered byte and when receives a new frame from the server it parses the data incorrectly (obvious as it has a garbage first byte).

Anyhow indeed I see the line https://github.com/igrigorik/em-websocket/blob/master/lib/em-websocket/connection.rb#L131 which should fix the issue. However it didn't work for me until I used String#bytesize.

Have you tested my example code using Ruby 1.9?

ws.onmessage do |msg|
  ws.send "iñaki"
end

[ibc]

No idea, I cannot reproduce the issue again (I mean the last one using ws.send).

[mloughran]

How frustrating... I'll leave this issue closed for now, but if we see this again we'll reopen. Cheers

[refaelos]

still getting US-ASCII on onmessage ...

how can i get UTF-8 ?!

[mloughran]

@refaelos are you using the released version of em-websocket or the code from master? This fix hasn't made it into a released version yet - I will try to get round to that soon.

[refaelos]

how do i use it in my Gemfile ?

when i just put "gem 'em-websocket'" i get the code without the fix.

[mloughran]

Just take a look at the bundler docs for including git repos

On 9 Hyd 2011, at 16:46, refaelosreply@reply.github.com wrote:

how do i use it in my Gemfile ?

when i just put "gem 'em-websocket'" i get the code without the fix.

Reply to this email directly or view it on GitHub:
#57 (comment)

[refaelos]

10x

[smpallen99]

I'm still seeing this issue after upgrading to 0.3.2. Any suggestions?

[mloughran]

@smpallen99 would be great if you could elaborate - as far as I know this should all be fixed. Which ruby version & websocket draft / browser in particular?

[smpallen99]

I'm using ruby-1.9.2-p290.

I get the following message in chrome:
One or more reserved bits are on: reserved1 = 1, reserved2 = 0, reserved3 = 0

I get the following message in firefox 6.0.2:
[WebSocket] bad response: HTTP/1.1 400 Bad Request
__log(a="%5BWebSocket%5D%20bad%2...1%20400%20Bad%20Request")flashf....min.js (line 53)
0);return!0},WebSocket.__log=function(..._SOCKET_DISABLE_AUTO_INITIALIZATION||

The strange this is I have two PCs that are almost identical, one has the issue and the other does not. I also have another user that has been using my site for a couple months now, but just started experiencing the problem a couple days ago.

[mloughran]

@smpallen99 so your issue is not related to received message encoding? Could you open a new issue in that case?

[smpallen99]

Issue #62 created #62