Change encoding of OpenSSL::HMAC.digest result to binary #187

merged 1 commit into from Feb 15, 2013


None yet
2 participants

eitoball commented Feb 12, 2013

I think that this should fix MacRuby/MacRuby#49 .

With version 0.12, simple script using net-ssh like following fails with an error

/snip/gems/net-ssh-2.6.5/lib/net/ssh/transport/packet_stream.rb:146:in `enqueue_packet': incompatible character encodings: ASCII-8BIT and UTF-8 (Encoding::CompatibilityError)

require 'rubygems'
require 'net/ssh'
Net::SSH.start('hostname', 'username', :password => 'password') do |ssh|

The packet_stream.rb:146 is like

message = encrypted_data + mac

The encoding of encrypted_data is ASCII-8BIT, and encoding of mac is UTF-8. mac is generated at 3 lines above.

mac = client.hmac.digest([client.sequence_number, unencrypted_data].pack("NA*"))

client is an instance of Net::SSH::Transport::HMAC::* (depends algorithm). All those classes are inherited from Net::SSH::Transport::HMAC::Abstract, and Net::SSH::Transport::HMAC::Abstract#digest simply calls OpenSSL::HMAC.digest like:

def digest(data)
  OpenSSL::HMAC.digest(, key, data)[0,mac_length]

Thus, I change the encoding of OpenSSL::HMAC.digest return value. Indeed, MRI returns BINARY.

$ macruby -v -ropenssl -e"p OpenSSL::HMAC.digest('sha1', '', '').encoding" 
MacRuby 0.12 (ruby 1.9.2) [universal-darwin10.0, x86_64]
$ ruby -v -ropenssl -e"p OpenSSL::HMAC.digest('sha1', '', '').encoding"
ruby 1.9.3p194 (2012-04-20 revision 35410) [x86_64-darwin11.4.0]

Watson1978 commented Feb 13, 2013

Thank you for spending your time to this work.
I'll try to look this to find the time at this weekend.


Watson1978 merged commit 4e27c8b into MacRuby:master Feb 15, 2013


Watson1978 commented Feb 15, 2013

Looks this is great patch Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment