Skip to content
This repository

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Can not work with multibyte UTF-8 strings #8

Closed
ytaka opened this Issue December 29, 2010 · 4 comments

2 participants

Takayuki Yamaguchi Martin Vidner
Takayuki Yamaguchi

String#length and String#size of Ruby 1.9 does not equal to
byte size of multibyte strings.
So, when we use multibyte strings of utf-8,
ruby-dbus can not work properly in ruby 1.9.

I created the following patch for ruby 1.9 and 1.8.7.
Since ruby 1.8.6 does not support String#bytesize,
it can not work for ruby 1.8.6.

I wonder if, verifying that this modification does not have bugs,
it is merged.

diff --git a/lib/dbus/auth.rb b/lib/dbus/auth.rb
index d14cc5c..bce9742 100644
--- a/lib/dbus/auth.rb
+++ b/lib/dbus/auth.rb
@@ -62,7 +62,7 @@ module DBus
       # name of cookie file, id of cookie in file, servers random challenge  
       context, id, s_challenge = data.split(' ')
       # Random client challenge        
-      c_challenge = Array.new(s_challenge.length/2).map{|obj|obj=rand(255).to_s}.join
+      c_challenge = Array.new(s_challenge.bytesize/2).map{|obj|obj=rand(255).to_s}.join
       # Search cookie file for id
       path = File.join(ENV['HOME'], '.dbus-keyrings', context)
       puts "DEBUG: path: #{path.inspect}" if $debug
@@ -161,7 +161,7 @@ module DBus
       while left > 0
         buf = @socket.read( left > 1 ? 1 : left )
         break if buf.nil?
-        left -= buf.size
+        left -= buf.bytesize
         data += buf
         break if data.include? crlf #crlf means line finished, the TCP socket keeps on listening, so we break 
       end
diff --git a/lib/dbus/introspect.rb b/lib/dbus/introspect.rb
index 9320a73..653da11 100644
--- a/lib/dbus/introspect.rb
+++ b/lib/dbus/introspect.rb
@@ -47,7 +47,7 @@ module DBus
 
     # Validates a service _name_.
     def validate_name(name)
-      raise InvalidIntrospectionData if name.size > 255
+      raise InvalidIntrospectionData if name.bytesize > 255
       raise InvalidIntrospectionData if name =~ /^\./ or name =~ /\.$/
       raise InvalidIntrospectionData if name =~ /\.\./
       raise InvalidIntrospectionData if not name =~ /\./
@@ -107,7 +107,7 @@ module DBus
 
     # Validates element _name_.
     def validate_name(name)
-      if (not name =~ MethodSignalRE) or (name.size > 255)
+      if (not name =~ MethodSignalRE) or (name.bytesize > 255)
         raise InvalidMethodName, name
       end
     end
diff --git a/lib/dbus/marshall.rb b/lib/dbus/marshall.rb
index 6f0c01c..532d29b 100644
--- a/lib/dbus/marshall.rb
+++ b/lib/dbus/marshall.rb
@@ -53,7 +53,7 @@ module DBus
     # Return an array of unmarshalled objects
     def unmarshall(signature, len = nil)
       if len != nil
-        if @buffy.size  @buffy.size
+        raise IncompleteBufferException if @idx > @buffy.bytesize
       else
         raise "Unsupported alignment #{a}"
       end
@@ -86,7 +86,7 @@ module DBus
 
     # Retrieve the next _nbytes_ number of bytes from the buffer.
     def get(nbytes)
-      raise IncompleteBufferException if @idx + nbytes > @buffy.size
+      raise IncompleteBufferException if @idx + nbytes > @buffy.bytesize
       ret = @buffy.slice(@idx, nbytes)
       @idx += nbytes
       ret
@@ -96,8 +96,8 @@ module DBus
     def get_nul_terminated
       raise IncompleteBufferException if not @buffy[@idx..-1] =~ /^([^\0]*)\0/
       str = $1
-      raise IncompleteBufferException if @idx + str.size + 1 > @buffy.size
-      @idx += str.size + 1
+      raise IncompleteBufferException if @idx + str.bytesize + 1 > @buffy.bytesize
+      @idx += str.bytesize + 1
       str
     end
 
@@ -107,7 +107,7 @@ module DBus
       align(4)
       str_sz = get(4).unpack(@uint32)[0]
       ret = @buffy.slice(@idx, str_sz)
-      raise IncompleteBufferException if @idx + str_sz + 1 > @buffy.size
+      raise IncompleteBufferException if @idx + str_sz + 1 > @buffy.bytesize
       @idx += str_sz
       if @buffy[@idx].ord != 0
         raise InvalidPacketException, "String is not nul-terminated"
@@ -122,7 +122,7 @@ module DBus
     def get_signature
       str_sz = get(1).unpack('C')[0]
       ret = @buffy.slice(@idx, str_sz)
-      raise IncompleteBufferException if @idx + str_sz + 1 >= @buffy.size
+      raise IncompleteBufferException if @idx + str_sz + 1 >= @buffy.bytesize
       @idx += str_sz
       if @buffy[@idx].ord != 0
         raise InvalidPacketException, "Type is not nul-terminated"
@@ -193,7 +193,7 @@ module DBus
         raise InvalidPacketException if array_sz > 67108864
 
         align(signature.child.alignment)
-        raise IncompleteBufferException if @idx + array_sz > @buffy.size
+        raise IncompleteBufferException if @idx + array_sz > @buffy.bytesize
 
         packet = Array.new
         start_idx = @idx
@@ -267,18 +267,18 @@ module DBus
 
     # Align the buffer with NULL (\0) bytes on a byte length of _a_.
     def align(a)
-      @packet = @packet.ljust(num_align(@offset + @packet.length, a) - @offset, 0.chr)
+      @packet = @packet.ljust(num_align(@offset + @packet.bytesize, a) - @offset, 0.chr)
     end
 
     # Append the the string _str_ itself to the packet.
     def append_string(str)
       align(4)
-      @packet += [str.length].pack("L") + str + "\0"
+      @packet += [str.bytesize].pack("L") + str + "\0"
     end
 
     # Append the the signature _signature_ itself to the packet.
     def append_signature(str)
-      @packet += str.length.chr + str + "\0"
+      @packet += str.bytesize.chr + str + "\0"
     end
 
     # Append the array type _type_ to the packet and allow for appending
@@ -286,12 +286,12 @@ module DBus
     def array(type)
       # Thanks to Peter Rullmann for this line
       align(4)
-      sizeidx = @packet.size
+      sizeidx = @packet.bytesize
       @packet += "ABCD"
       align(type.alignment)
-      contentidx = @packet.size
+      contentidx = @packet.bytesize
       yield
-      sz = @packet.size - contentidx
+      sz = @packet.bytesize - contentidx
       raise InvalidPacketException if sz > 67108864
       @packet[sizeidx...sizeidx + 4] = [sz].pack("L")
     end
@@ -372,7 +372,7 @@ module DBus
 
         append_signature(vartype.to_s)
         align(vartype.alignment)
-        sub = PacketMarshaller.new(@offset + @packet.length)
+        sub = PacketMarshaller.new(@offset + @packet.bytesize)
         sub.append(vartype, vardata)
         @packet += sub.packet
       when Type::ARRAY
diff --git a/lib/dbus/message.rb b/lib/dbus/message.rb
index 7affa22..378e0d5 100644
--- a/lib/dbus/message.rb
+++ b/lib/dbus/message.rb
@@ -152,7 +152,7 @@ module DBus
       @params.each do |param|
         params.append(param[0], param[1])
       end
-      @body_length = params.packet.length
+      @body_length = params.packet.bytesize
 
       marshaller = PacketMarshaller.new
       marshaller.append(Type::BYTE, HOST_END)
Martin Vidner
Owner

Hi ytaka,

thanks for the report and the patch. Can you give me some test case? Ideally one plugging neatly into the existing test suite (run "rake" to exercise it) but any piece code will do if it gives me some idea how it makes the current library fail.

BTW it is easier for me if you add the patch as an attachment instead of inlining it, or if you make https://github.com/ytaka/ruby-dbus public (in case it is an omission and not an intent).

Takayuki Yamaguchi

Hello, mvidner

Thank you for your response.
I add a test for sending a multibyte string and commit the code to my github repository
https://github.com/ytaka/ruby-dbus (in particular, master branch in it).
In the test the dbus service sends a multibyte string in japanese
whose encoding is UTF-8 to a dbus client.
I am wondering if you review my source code.

Takayuki Yamaguchi

Sorry, I closed this topic by mistake.
I'm not familiar with how to use github.

Martin Vidner
Owner

Thanks for the test! I have merged it and also moved the force_encoding to the unmarshaller. Pushed 589b11a.

(no problem about your closing the issue, I could have reopened it)

BTW http://blog.grayproductions.net/articles/understanding_m17n is a great series that helped me understand how encoding is handled in both ruby 1.8 and 1.9.

This issue was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.