Ignore , as some blog software (dotclear2) misuse it.

feed2imap · Jul 27, 2009 · bc6b5c2 · bc6b5c2
1 parent a3c3ef6
commit bc6b5c2
Show file tree

Hide file tree

Showing 7 changed files with 8,282 additions and 2 deletions.
diff --git a/ChangeLog b/ChangeLog
@@ -1,9 +1,10 @@
-Ruby-Feedparser 0.7 (10/05/2009)
+Ruby-Feedparser 0.7 (27/07/2009)
 ================================
 * Handled several creators per feed item
 * Fix bug with urls into tag attributes
 * Better item categories support
 * Reworked text output formatting
+* Ignore &shy;, as some blog software (dotclear2) misuse it.
 
 Ruby-Feedparser 0.6 (23/07/2008)
 ================================

diff --git a/lib/feedparser/html2text-parser.rb b/lib/feedparser/html2text-parser.rb
@@ -399,7 +399,11 @@ def HTML2TextParser.entities
       "euro" => 8364
     }
     def unknown_entityref(ref)
-      if HTML_ENTITIES.has_key?(ref)
+      # hack to avoid considering &shy;, as it is misused by some blog software (dotclear2)
+      # see http://www.cs.tut.fi/~jkorpela/shy.html
+      if ref == 'shy'
+        handle_data('')
+      elsif HTML_ENTITIES.has_key?(ref)
         handle_data([HTML_ENTITIES[ref]].pack('U*'))
       else
         handle_data(ref)

diff --git a/test/html_output/eolas.output b/test/html_output/eolas.output
diff --git a/test/parser_output/eolas.output b/test/parser_output/eolas.output
diff --git a/test/source/eolas.xml b/test/source/eolas.xml
diff --git a/test/text_output/eolas.output b/test/text_output/eolas.output
diff --git a/test/textwrapped_output/eolas.output b/test/textwrapped_output/eolas.output