encode(string, :named) is re-encoding valid entries and breaking the HTML

[3den]

Hi,
First of all I would like to thank you for this awesome gem. But I found a bug while trying to sanitize a string that has both valid and invalid chars. below i explain this problem better:

coder = HTMLEntities.new
string = "> Car &amp; Bike <"
new = coder.encode(string)  # BUG =>  "&gt; Car &amp;amp; Bike &lt;" 
worst_then_new = coder.encode(new) # BUG => "&amp;gt; Car &amp;amp;amp; Bike &amp;lt;" 

A workaround this problem would be to "decode" before "encode" but this hack is to slow...

[3den]

I solved this problem with the following code:

class HTMLEntities
  class Encoder #:nodoc:     
    def basic_entity_regexp
      @basic_entity_regexp ||= (
        case @flavor
        when /^html/
          /[<>"]|(\&(?!\w))/
        else
          /[<>'"]|(\&(?!\w))/
        end
      )
    end
  end
end

[threedaymonk]

This is the intended behaviour: the goal of HTMLEntities is to encode and decode in a predictable manner. It's not supposed to fix invalid or incoherent sources, and it doesn't make any attempt to try to understand the intent of inconsistencies in the source, as this depends on the application.

[3den]

Indeed HTMLEntities is pretty good to encode and decode, but in my case I needed a simple and fast sanitizer to clean inconsistent sources, so i used HTMLEntities to created the https://github.com/3den/ruby-sanitizer