Roadmap of SyndicationFeed

[zhenlan]

SyndicationFeed on .NET Core

While we are making System.ServiceModel.Syndication available on .NET Core, we are also looking into improvements in following areas.

• Custom Parsers
  SyndicationFeed will have a default parser for RSS and Atom feeds, but now we will give users the opportunity to use their own parsing function for the known types, for example, if a user knows s/he is going to use an exact date format which might not be supported by the default date parser, the user will be able to override the default parser with her/his own code.

• Date Parsing
  Many users have experienced that the original parser failed to parse some exact date formats, the default date parser will be improved to support more date formats.

• Optional Items in RSS/ATOM spec
  There are currently some optional items that the feed object model doesn’t have. We plan to add them and make the user’s interaction with the object easier.

• Async support
  Current APIs to read/write a SyndicationFeed document are synchronous. A thread will be blocked until the whole document is read/written. These APIs will be changed to be asynchronous so a thread will not be blocked for I/O anymore. When the asynchronous call returns, a user has full document for further operation.
  Update (12/15/2017): After a lot of prototyping and testing, we found it was hard to provide async support while maintaining high compatibility with full .NET Framework. Since the main goal of this library is to provide compatibility so your libraries built on full .NET Framework can just work on .NET Core with little/no changes, we decide not to support async in this library. We recommend to use the new Microsoft.SyndicationFeed.ReaderWriter library, which has async design in mind from the beginning.

A new syndication feed

While we are improving on the existing SyndicationFeed, which gives users best compatibility for porting existing code from .NET Framework to .NET Core, we are investing in a new syndication feed reader/writer, which is expected to suit better for modern programming when users are writing new code.

• New syndication feed reader/writer
  Unlike the async support for existing SyndicationFeed where an async call will not return until the whole document is read/written. A new syndication feed reader/writer will support reading/writing on demand. It does not save the whole content in memory and it only reads/writes forward (similar to the concept of XML reader). The new feed reader/writer will have graceful error handling, extensibility model and other improvements we have been doing for the existing SyndicationFeed. It will be a separate package that will give maximum flexibility to users (eg. migrating existing code vs. writing new code).

Feedback

Both the improved SyndicationFeed and the new syndication feed reader/writer will be built on .NET Standard so they can be used anywhere .NET Standard is supported (yes, you can even use them on full .NET Framework). Both projects are still working in progress, but we'd love to hear what you think as your feedback is crucial for us to make it right for you.

[RehanSaeed]

jsonfeed.org has sprung into existence since I raised the original issue #76. It would be nice to be able to support that too, since Atom and RSS are a decade old format with a lot of problems and undefined behaviour.

[dodyg]

jsonfeed is too new to be supported. There are so many attempts to JSONify syndication format and so far nothing catches on.

[dodyg]

The new feed reader/writer will have graceful error handling, extensibility model and other improvements we have been doing for the existing SyndicationFeed

What is the model in handling namespaced extensions?

Will the new library (Microsoft.SyndicationFeed) helps with format detection?

[RehanSaeed]

@dodyg It's the chicken and egg problem, if frameworks don't support it, it won't get popular and frameworks won't support it. According to Google Trends, jsonfeed.org is on par with Atom feeds in terms of interest.

Agreed that it might be too soon but @zhenlan's post mentions extensibility, a pluggable architecture so we could add a reader/writer for JSON should be considered.

[zhenlan]

I hope this example will show some ideas of the extensibility model of the new syndication feed reader. Every time reader.Read() is called it will return the next syndication element. If it is a "known" element type, you can use one of built-in parsers; otherwise, it can be parsed as ISyndicationContent, which will contain raw Xml content among other things. Users can also pass in their own implementation of ISyndicationFeedFormatter when instantiating the reader for custom parsing.

Current implementation of syndication reader and formatter are built on XmlReader. To read JSON, however, I think it's possible for someone to implement ISyndicationFeedReader and ISyndicationFeedFormatter based on a "JsonReader". It does not have to be implemented by the "framework", but it will good for the framework to provide proper foundation.

What is the model in handling namespaced extensions?

This is an interesting question. Current design probably needs more work for this. @dodyg do you have any more specific example in mind? @drago-draganov do you have any thoughts about this?

[dodyg]

• http://scripting.com/rss.xml uses a source:outline extension on RSS
• https://en.wikipedia.org/wiki/Media_RSS (https://www.youtube.com/feeds/videos.xml?channel_id=UCBcRF18a7Qf58cCRy5xuWwQ)
• https://sports.yahoo.com/mlb/teams/bos/rss.xml uses Dublin Core extension

I bet there are plenty more used internally.

[zhenlan]

@dodyg thanks for the examples. We will look into them.

[dodyg]

I hope it can be supported as the extension mechanism for RSS and ATOM is the same. For enterprise development I have seen people writing a custom RSS parser just to process extensions.

More:

• SharePoint RSS Extension

[lviana1]

You mention there will be default parsers for RSS and Atom feeds, but you don't specify which versions. There are multiple versions in use out there, with the important ones being: RSS 1.0 (RDF), RSS 2.0, Atom 0.3, and Atom 1.0. Ideally there would be support for all of these. Support for JSON Feed is also a good idea, IMO.

As for namespace extensions, here's a directory of different ones: http://www.feedforall.com/directory-namespace.htm. The big ones are really: iTunes, Media RSS, Dublin Core, and Content. Ideally there would be support for these as well.

I'm glad to hear that there will be a more forgiven date parser. There are many feeds out there with nonconforming dates and other fields. A less strict parser that allows the raw data to be extracted would make this library significantly more useful.

[zhenlan]

@lviana1 thanks for the question and additional information. Our plan is to support RSS 2.0 and Atom 1.0.

I am excited to announce that we just released a preview of Microsoft.SyndicationFeed on nuget.org. Here is a bit instructions and examples to get you started. We are eager to hear your feedback. Please let us know how it works for you.

[RehanSaeed]

An Atom feed writer does not currently exist in the new API. Someone has raised #2169 to address this issue. Any guestimates for when one might be built?

[drago-draganov]

@RehanSaeed
We are actively working on an Atom feed writer. I'm expecting the code to be available in the next week. It will be included with the next release of the NuGet package. There are some logistics that we have to take care, but I'm not expecting significant delays of it.

[drago-draganov]

Atom feed writer was added with #2250

[RehanSaeed]

Any plans to update the NuGet package with the Atom feed writer and also give it a strong name?

[drago-draganov]

@RehanSaeed
An update of the NuGet packages is in progress. I mentioned some logistics, thus can't be very explicit about ETA at the moment. We are also considering stepping out of the preview and issue an official 1.0 version. I will update the thread next week with more details on the ETA.

[drago-draganov]

Here is a quick update plan for the next week

• We are renaming Microsoft.SyndicationFeed to Microsoft.SyndicationFeed.ReaderWriter
• The project will be moved to own GitHub repository: /dotnet/SyndicationFeedReaderWriter
• Release of updated NuGet package

[drago-draganov]

Microsoft.SyndicationFeed.ReaderWriter 1.0 is now live. Release notes.

Please refer to the new repository for continuing development and discussion regarding Microsoft.SyndicationFeed.ReaderWriter.

[DFsharp]

Is System.ServiceModel.Syndication still in the backlog?

[zhenlan]

@DFsharp, System.ServiceModel.Syndication has a prerelease about a month ago. This release should have high compatibility with the Syndication libraries in full framework. We are working in progress on the improvement of parsing and optional items mentioned in the road map above.
https://www.nuget.org/packages/System.ServiceModel.Syndication/

We also moved its source code to corefx repo. If you run into any issue, please open issues on corefx repo.
https://github.com/dotnet/corefx/tree/master/src/System.ServiceModel.Syndication

[zhenlan]

Closing this issue. Here is just a quick summary of the two syndication related libraries we released. Please see the first comment of this thread for their difference.

1. System.ServiceModel.Syndication
   Source: https://github.com/dotnet/corefx/tree/master/src/System.ServiceModel.Syndication
   NuGet package: https://www.nuget.org/packages/System.ServiceModel.Syndication/

2. Microsoft.SyndicationFeed.ReaderWriter
   Source: https://github.com/dotnet/SyndicationFeedReaderWriter
   NuGet package: https://www.nuget.org/packages/Microsoft.SyndicationFeed.ReaderWriter/

[oliverw]

Closing this issue. Here is just a quick summary of the two syndication related libraries we released. Please see the first comment of this thread for their difference.

1. System.ServiceModel.Syndication
   Source: https://github.com/dotnet/corefx/tree/master/src/System.ServiceModel.Syndication
   NuGet package: https://www.nuget.org/packages/System.ServiceModel.Syndication/
2. Microsoft.SyndicationFeed.ReaderWriter
   Source: https://github.com/dotnet/SyndicationFeedReaderWriter
   NuGet package: https://www.nuget.org/packages/Microsoft.SyndicationFeed.ReaderWriter/

And both are dead.