Atom Feed type="xhtml" Parse Issue

[benubois]

Feedzirra or sax-machine or Nokogiri has an issue with Atom feeds that use

<content type="xhtml">

The result is that all HTML is stripped out.

Sax-machine uses Nokogiri for XML parsing.

For example, when Hypercritical's feed is parsed using Nokogiri::XML(xml_string) you get:
https://gist.github.com/benubois/5520828

The <content> gets parsed into a tree as well. I think what sax-machine needs to do at this point is recognize that type="xhtml" and just return the contents as a string.

There is a two year old pull request for Feedzirra about this issue, however the code no longer works.

From the Atom spec on how type="xhtml" should be handled:

3.1.1.3.  XHTML

Example atom:title with XHTML content:

...
<title type="xhtml" xmlns:xhtml="http://www.w3.org/1999/xhtml">
  <xhtml:div>
    Less: <xhtml:em> &lt; </xhtml:em>
  </xhtml:div>
</title>
...

If the value of "type" is "xhtml", the content of the Text construct
MUST be a single XHTML div element [XHTML] and SHOULD be suitable for
handling as XHTML.  The XHTML div element itself MUST NOT be
considered part of the content.  Atom Processors that display the
content MAY use the markup to aid in displaying it.  The escaped
versions of characters such as "&" and ">" represent those
characters, not markup.

[pedrocr]

I see this is a three month issue, is there any plan of when to fix it? Some of my favorite feeds are affected by this so I'll have to find another reader if this is to be left unfixed.

[RedwoodForest]

This is the single major outstanding issue for me.

On Jul 30, 2013, at 2:25 AM, "Pedro Côrte-Real" notifications@github.com
wrote:

I see this is a three month issue, is there any plan of when to fix it?
Some of my favorite feeds are affected by this so I'll have to find another
reader if this is to be left unfixed.

—
Reply to this email directly or view it on
GitHubhttps://github.com//issues/120#issuecomment-21779177
.

[pedrocr]

Feedbin doesn't seem to be under active support. Canceled my account.

[benubois]

@pedrocr Looks like you have some XML and Ruby experience and I could use some help fixing the issue.

[pedrocr]

I hadn't realized feedbin was open-source. I'll try and bring up a local copy to have a look.

[benubois]

@pedrocr That would be awesome. The issue is actually with Feedzirra which I don't maintain, but Feedbin uses a fork of the project so if you are able to find anything I'd be happy to accept any patches into the fork.

[pedrocr]

I had a quick-look at all this. sax-machine doesn't seem to be doing anything particularly wrong, it's just a generic XML parser, seems similar to my xmlcodec project actually.

feedzirra on the other hand is using sax-machine to implement a non-conformant atom parser. That's probably the reason why it's not even implementing one parser but instead 3 different ones (a generic, a feedburner and a google docs one). All those parsers treat the element as if it was a simple text field and when presented with a full XML parse tree (the XHTML case) just take the text and run.

The simplest course of action is probably to implement an Atom parser directly using nokogiri instead of trying to coax sax-machine into doing the right thing. feedzirra's architecture seems to make that easier by allowing plugin of new parsers by the user.

[pedrocr]

I got nostalgic for xmlcodec and did a bit of a sprint last night and this morning to get it back on its feet. I've release version 0.3.1 that now works fine with ruby 1.9 and has some extra features to handle this Atom case better. Here's a gist comparing using sax-machine and xmlcodec to parse Atom:

https://gist.github.com/pedrocr/6391196

My xmlcodec version solves the problem with the XHTML content inside the feed. This in itself won't plug into feedzirra to fix the issue though.

[RedwoodForest]

Just dropping by again to say this is still my biggest (and only major) outstanding issue with Feedbin. It's not possible to read cached posts from these feeds on my phone when I don't have internet service because with the formatting stripped they're just not readable.

[jonallured]

Hi @benubois, I've got a branch that addresses this - have a look and let me know if you have any feedback.

Thanks!
Jon