New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How do feed readers store their data? #109

Open
scripting opened this Issue Jan 31, 2019 · 6 comments

Comments

Projects
None yet
5 participants
@scripting
Copy link
Owner

scripting commented Jan 31, 2019

I've been working on a MySQL version of a RSS feed reader server app. Unlike River5, which stores all data in JSON files, this reader stores all data in a database, and accesses the data via queries.

In the last year, starting with Bingeworthy and then Feedbase, I've learned enough SQL, and have expert help when I need it. Not sure when or if I will release it. I wanted to understand the issues.

I find SQL is an awkward tool for accessing this kind of data. Which got me to wondering how other feed reader apps store their data. I'm supposed to be such an expert on this stuff, and I admit I don't know what they're doing. I wonder if some of them are using object databases.

@jefftriplett

This comment has been minimized.

Copy link

jefftriplett commented Jan 31, 2019

NewsBlur is pretty popular (open source Django app) and uses both Postgres and MongoDB. I linked the project in case the code is helpful. Here is main Feed model: https://github.com/samuelclay/NewsBlur/blob/master/apps/rss_feeds/models.py#L54

@brentsimmons

This comment has been minimized.

Copy link

brentsimmons commented Jan 31, 2019

NetNewsWire uses SQL for articles and read/unread/starred status. It stores feed/folder structure in an OPML file on disk, and it stores feed metadata in an object database.

@brentsimmons

This comment has been minimized.

@scripting

This comment has been minimized.

Copy link
Owner Author

scripting commented Jan 31, 2019

Here's the schema for the feed reader I'm working on..

https://github.com/scripting/Scripting-News/blob/master/misc/feedDatabaseTables.md

Very interesting to compare structures.

Repository owner deleted a comment from evanwolf Feb 1, 2019

@mblaney

This comment has been minimized.

Copy link

mblaney commented Feb 1, 2019

@ezraball

This comment has been minimized.

Copy link

ezraball commented Feb 5, 2019

I'm not sold on NoSQL in general, but I've played with CouchDB a bit and its multi-source replication is very slick. If I were building a feed reader where a requirement was to sync multiple devices (for read/unread state) I'd consider it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment