New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Multibyte unicode characters #9

Closed
StefansM opened this Issue Sep 12, 2010 · 7 comments

Comments

Projects
None yet
2 participants
@StefansM

StefansM commented Sep 12, 2010

Hi all,

If I write a string containing multibyte characters to the IXP fs, I get the following error:

irb(Rumai):002:0> Node.new("/rbar/0-mpd").write("\u{2019}")
Errno::ECONNRESET: Connection reset by peer
        from /usr/lib/ruby/gems/1.9.1/gems/rumai-3.3.1/lib/rumai/ixp/message.rb:32:in `read'
        from /usr/lib/ruby/gems/1.9.1/gems/rumai-3.3.1/lib/rumai/ixp/message.rb:32:in `read_9p'
        from /usr/lib/ruby/gems/1.9.1/gems/rumai-3.3.1/lib/rumai/ixp/message.rb:417:in `from_9p'
        from /usr/lib/ruby/gems/1.9.1/gems/rumai-3.3.1/lib/rumai/ixp/transport.rb:140:in `block (2 levels) in recv'
        from :10:in `synchronize'
        from /usr/lib/ruby/gems/1.9.1/gems/rumai-3.3.1/lib/rumai/ixp/transport.rb:122:in `block in recv'
        from /usr/lib/ruby/gems/1.9.1/gems/rumai-3.3.1/lib/rumai/ixp/transport.rb:121:in `loop'
        from /usr/lib/ruby/gems/1.9.1/gems/rumai-3.3.1/lib/rumai/ixp/transport.rb:121:in `recv'
        from /usr/lib/ruby/gems/1.9.1/gems/rumai-3.3.1/lib/rumai/ixp/transport.rb:176:in `talk'
        from /usr/lib/ruby/gems/1.9.1/gems/rumai-3.3.1/lib/rumai/ixp/transport.rb:471:in `clunk'
        from /usr/lib/ruby/gems/1.9.1/gems/rumai-3.3.1/lib/rumai/ixp/transport.rb:265:in `close'
        from /usr/lib/ruby/gems/1.9.1/gems/rumai-3.3.1/lib/rumai/ixp/transport.rb:222:in `open'
        from /usr/lib/ruby/gems/1.9.1/gems/rumai-3.3.1/lib/rumai/ixp/transport.rb:376:in `write'
        from /usr/lib/ruby/gems/1.9.1/gems/rumai-3.3.1/lib/rumai/fs.rb:116:in `write'
        from (irb):2
        from /usr/lib/ruby/gems/1.9.1/gems/rumai-3.3.1/bin/rumai:16:in `'
        from /usr/bin/rumai:19:in `load'
        from /usr/bin/rumai:19:in `'irb(Rumai):003:0> 

I'm using ruby 1.9.2. My current workaround is to not use unicode characters :)

@sunaku

This comment has been minimized.

Show comment
Hide comment
@sunaku

sunaku Sep 12, 2010

Owner

I agree, and wmiir does not seem to have this problem:

wmiir xwrite /rbar/0-mpd "label ’"
Owner

sunaku commented Sep 12, 2010

I agree, and wmiir does not seem to have this problem:

wmiir xwrite /rbar/0-mpd "label ’"
@ghost

This comment has been minimized.

Show comment
Hide comment
@ghost

ghost Nov 5, 2010

Hi,

I can confirm this issue. After updating rbar field with an utf-8 character it crashes with the above message in wmiirc.log and also tags switching stops working. I cann't use localised date info or MPD titles/artist names with localised chars.

I suspect it's caused by hardcoded sizes for characters (only ASCII is handled) for communication stream in IXP Rumai module.

Is there a chance this bug/misfeature will be fixed soon ?

ghost commented Nov 5, 2010

Hi,

I can confirm this issue. After updating rbar field with an utf-8 character it crashes with the above message in wmiirc.log and also tags switching stops working. I cann't use localised date info or MPD titles/artist names with localised chars.

I suspect it's caused by hardcoded sizes for characters (only ASCII is handled) for communication stream in IXP Rumai module.

Is there a chance this bug/misfeature will be fixed soon ?

@sunaku

This comment has been minimized.

Show comment
Hide comment
@sunaku

sunaku Nov 5, 2010

Owner

I'm pretty busy these days; feel free to contribute a patch. :)

Owner

sunaku commented Nov 5, 2010

I'm pretty busy these days; feel free to contribute a patch. :)

@sunaku

This comment has been minimized.

Show comment
Hide comment
@sunaku

sunaku Feb 23, 2011

Owner

I am using ruby 1.9.2p180 and my MPD status applet is able to write Unicode text to the bar label:

>> fs.rbar['0-mpd'].read
"colors #dcdcdc #343434 #444444\nlabel (mpd) \xC3\x98ystein Sev\xC3\xA5g: Consolation\n"

Notice that the UTF-8 characters are expressed as two bytes instead of a single \u{...} compound. Of course, this isn't a solution for the original issue, but it is somewhat puzzling.

>> puts "\xC3\x98ystein Sev\xC3\xA5g"
Øystein Sevåg
nil

>> puts "\xC3\x98ystein Sev\xC3\xA5g".encoding
UTF-8
nil
Owner

sunaku commented Feb 23, 2011

I am using ruby 1.9.2p180 and my MPD status applet is able to write Unicode text to the bar label:

>> fs.rbar['0-mpd'].read
"colors #dcdcdc #343434 #444444\nlabel (mpd) \xC3\x98ystein Sev\xC3\xA5g: Consolation\n"

Notice that the UTF-8 characters are expressed as two bytes instead of a single \u{...} compound. Of course, this isn't a solution for the original issue, but it is somewhat puzzling.

>> puts "\xC3\x98ystein Sev\xC3\xA5g"
Øystein Sevåg
nil

>> puts "\xC3\x98ystein Sev\xC3\xA5g".encoding
UTF-8
nil
@sunaku

This comment has been minimized.

Show comment
Hide comment
@sunaku

sunaku Feb 25, 2011

Owner

The problem was how Ruby 1.9 counts the length of a string: it counts characters, not bytes. I've got an ad-hoc solution working locally but my existing unit tests are failing. I'll push an update once I fix the failing tests.

Owner

sunaku commented Feb 25, 2011

The problem was how Ruby 1.9 counts the length of a string: it counts characters, not bytes. I've got an ad-hoc solution working locally but my existing unit tests are failing. I'll push an update once I fix the failing tests.

@sunaku

This comment has been minimized.

Show comment
Hide comment
@sunaku

sunaku Feb 25, 2011

Owner

Commit e7a9757 fixes this issue.

Owner

sunaku commented Feb 25, 2011

Commit e7a9757 fixes this issue.

@ghost

This comment has been minimized.

Show comment
Hide comment
@ghost

ghost Mar 5, 2011

Finally fixed. Grreeaatt !
Thank goes to sunaku.

ghost commented Mar 5, 2011

Finally fixed. Grreeaatt !
Thank goes to sunaku.

This issue was closed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment