Why do ldquo and rdquo appear differently?

[johnlauck]

Can anyone explain why the left quote displays differently than the right quote? Is this normal?

coder = HTMLEntities.new
string = "“These pretzels are making me thirsty…”"
coder.decode(string) => "“These pretzels are making me thirsty…\342\200\235"

I'm using ruby enterprise 1.8.7.

[threedaymonk]

I can't reproduce this using Ruby 1.8.7 (non-REE):

>> require 'htmlentities'
=> true
>> HTMLEntities::VERSION::STRING
=> "4.2.4"
>> coder = HTMLEntities.new
=> #<HTMLEntities:0x7f2aba3496d0 @flavor="xhtml1">
>> string = "&ldquo;These pretzels are making me thirsty&hellip;&rdquo;"
=> "&ldquo;These pretzels are making me thirsty&hellip;&rdquo;"
>> s = coder.decode(string)
=> "\342\200\234These pretzels are making me thirsty\342\200\246\342\200\235"
>> puts s
“These pretzels are making me thirsty…”
=> nil
>> $KCODE = 'u'
=> "u"
>> s
=> "“These pretzels are making me thirsty…”"

What do you get from the same commands?

[johnlauck]

Looks like an issue with ree 1.8.7?

ruby 1.8.7 (2010-08-16 patchlevel 302) [i686-darwin10.6.0]

>> require 'htmlentities'
=> true
>> HTMLEntities::VERSION::STRING
=> "4.2.4"
>> coder = HTMLEntities.new
=> #<HTMLEntities:0x100593840 @flavor="xhtml1">
>> string = "&ldquo;These pretzels are making me thirsty&hellip;&rdquo;"
=> "&ldquo;These pretzels are making me thirsty&hellip;&rdquo;"
>> s = coder.decode(string)
=> "\342\200\234These pretzels are making me thirsty\342\200\246\342\200\235"
>> puts s
“These pretzels are making me thirsty…”
=> nil
>>  $KCODE = 'u'
=> "u"
>> s
=> "“These pretzels are making me thirsty…”"

ruby 1.8.7 (2010-04-19 patchlevel 253) [i686-darwin10.4.0], MBARI 0x6770, Ruby Enterprise Edition 2010.02

>> require 'htmlentities'
=> true
>> HTMLEntities::VERSION::STRING
=> "4.2.4"
>> coder = HTMLEntities.new
=> #<HTMLEntities:0x1015a1d18 @flavor="xhtml1">
>> string = "&ldquo;These pretzels are making me thirsty&hellip;&rdquo;"
=> "&ldquo;These pretzels are making me thirsty&hellip;&rdquo;"
>> s = coder.decode(string)
=> "\342\200\234These pretzels are making me thirsty\342\200\246\342\200\235"
>> puts s
“These pretzels are making me thirsty…”
=> nil
>> $KCODE = 'u'
=> "u"
>> s
=> "“These pretzels are making me thirsty…\342\200\235"

[threedaymonk]

It does seem like it's REE rather than HTMLEntities, and specifically either irb or the implementation of inspect. What do you get if you do this with REE:

>> $KCODE = 'u'
=> "u"
>> "\342\200\235"
=> "”"

If you get an escape sequence in the last line, then I'll assume that it's a problem with REE and close this issue, if that's OK.

[johnlauck]

ree output is "\342\200\235"

Thanks for your help. Any idea for a work around?

[threedaymonk]

Sounds like a bug in REE then. Are you convinced it's really a problem? Do you get escape characters instead of the quote anywhere except in irb? Because if it's only in irb, then it's just an irritation but shouldn't actually affect anything at an operational level.

[threedaymonk]

(Closing this as I'm confident it's not an HTMLEntities bug)

[johnlauck]

Update: I tested this on REE 1.8.7-head and it works. I think the REE 1.8.7-2011.01 includes the fix.