Change encoding of OpenSSL::HMAC.digest result to binary #187

Merged
merged 1 commit into from Feb 15, 2013

Conversation

Projects
None yet
2 participants
Contributor

eitoball commented Feb 12, 2013

I think that this should fix MacRuby/MacRuby#49 .

With version 0.12, simple script using net-ssh like following fails with an error

/snip/gems/net-ssh-2.6.5/lib/net/ssh/transport/packet_stream.rb:146:in `enqueue_packet': incompatible character encodings: ASCII-8BIT and UTF-8 (Encoding::CompatibilityError)

require 'rubygems'
require 'net/ssh'
Net::SSH.start('hostname', 'username', :password => 'password') do |ssh|
end

The packet_stream.rb:146 is like

message = encrypted_data + mac

The encoding of encrypted_data is ASCII-8BIT, and encoding of mac is UTF-8. mac is generated at 3 lines above.

mac = client.hmac.digest([client.sequence_number, unencrypted_data].pack("NA*"))

client is an instance of Net::SSH::Transport::HMAC::* (depends algorithm). All those classes are inherited from Net::SSH::Transport::HMAC::Abstract, and Net::SSH::Transport::HMAC::Abstract#digest simply calls OpenSSL::HMAC.digest like:

def digest(data)
  OpenSSL::HMAC.digest(digest_class.new, key, data)[0,mac_length]
end

Thus, I change the encoding of OpenSSL::HMAC.digest return value. Indeed, MRI returns BINARY.

$ macruby -v -ropenssl -e"p OpenSSL::HMAC.digest('sha1', '', '').encoding" 
MacRuby 0.12 (ruby 1.9.2) [universal-darwin10.0, x86_64]
#<Encoding:UTF-8>
$ ruby -v -ropenssl -e"p OpenSSL::HMAC.digest('sha1', '', '').encoding"
ruby 1.9.3p194 (2012-04-20 revision 35410) [x86_64-darwin11.4.0]
#<Encoding:ASCII-8BIT>
Owner

Watson1978 commented Feb 13, 2013

Thank you for spending your time to this work.
I'll try to look this to find the time at this weekend.

どうもありがとうございます(^_^)

Watson1978 merged commit 4e27c8b into MacRuby:master Feb 15, 2013

Owner

Watson1978 commented Feb 15, 2013

Looks this is great patch Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment