River5 is a river-of-news RSS aggregator in JavaScript running in Node, written by Dave Winer.
When it starts up, River5 reads files in the lists folder at the top level of the River5 folder.
Each list contains a set of URLs of feeds. The list files can be straight text, a JSON array of URLs of feeds, or a traditional OPML subscription list, with extensions .txt, .json and .opml. River5 ignores files in the lists folder that are not in one of these formats.
The feeds pointed to from the lists can be in RSS 0.9x, 1.0 or 2.0, or Atom 1.0.
Every 15 minutes, River5 reads all the feeds you're subscribed to, and routes new items to corresponding files in the rivers folder at the top level of the River5 folder. You're subscribed to a feed if it appears in one or more of your lists. The river files are a documented form of JSONP that we've been using since River3, in 2010.
To read the output of River5 on the machine it's running on, go to the home page. Assuming your river is running on the default port, 1337, you would go to this address to read the rivers. http://localhost:1337/. If it's running on a public server, just replace localhost with the domain name or IP address of the server.
Pretty much everything in this narrative is configurable.
A current Node.js installation.
The machine running River5 ahould be accessible over the Internet.
It is possible to run the software in a limited form on a non-public machine. Of course, you won't be able to access the home page remotely, or receive realtime notifications of feed updates. You'll also have to find a way to upload the output files in the rivers folder to a place where they can be accessed.
If you don't have Node.js installed on your machine, here are step by step instructions for installing Node.js on a Macintosh.
-
Download the folder from the repository to your local computer.
-
From the command line, go to this folder.
-
npm install
-
node river5.js
If it's working properly, you should see tons of stories scroll by in the console. River5 is reading each of the feeds in the lists for the first time. When you see things settle down, then you should open the home page of the RIver5 website, at http://localhost:1337/ and see the result of the first feed readings.
At first there will be a lot of old stories, but as the rivers run for hours and days and into weeks, you'll get the flow. It's a good way to read news.
I have included several files in the lists folder to help you get started, so something actually happens the first time you run River5. You can edit, consolidate or delete them, as you wish.
-
myJsonFeeds.json -- a list of URLs of feeds in the JSON format that River5 understands.
-
myOpmlFeeds.opml -- a list of URLs of feeds in the standard OPML format for subscription lists.
-
myTxtFeeds.txt -- a list of URLS in a text file.
-
nyt.opml -- a list of a New York Times news feeds.
-
hn.opml -- a list with just one feed in it, the Hacker News firehose. It's a good list for testing RIver5 because it updates so frequently.
Go to http://localhost:1337/.
There's only one way to configure it, via the config.json file in the same folder as the river5.js app.
The example config.json in the folder sets the max number of items in a river to 300.
All of the config.json options and their default values are listed on this page.
River5 does not support S3 storage, so if that's how you're running your rivers, you should continue to use River4.
-
If you're running out of the filesystem, then River5 can replace River4.
-
Follow the installation instructions above, and before you launch river5.js, remove the files from the lists folder, and replace them with the lists from your River4 installation.
-
It will take a while before your rivers re-populate, so I recommend doing it in parallel with your existing River4 install.
You may want to review the Configuration settings page, there are options for setting new things, and in some cases the defaults have changed.
The heart of River5 is in a Node package in the lib folder, called feedtools.js.
The plan is to eventually offer it through the Node.js package distribution system so we can easily include feed functionality in other apps. For right now, feedtools.js is set up just to work with river5.js.
-
Podcatch.com subscribes to my friends' podcast feeds.
-
mlbriver.com and nbariver.com collect news for two sports I love.
-
I have a River panel on the Scripting News home page.
-
My rivers page puts all the news I care about on a single tabbed page, with panels for the NYT, Washington Post, Guardian, Movies, Tech, baseball and basketball.
-
The road to River5, reviews the previous versions of my river-of-news software going back to 1999, with a special focus on the transition from River4 to River5.
-
The Hello World of Rivers shows you how to create a standalone river, one that people can read without accessing your RIver5 server.
-
My blog post announcing River5.
Fixed a bug that prevented addToRiver callbacks from working correctly. When you made a change to an item in a river it wasn't actually being reflected back in the river.
As a result of this fix, River4 callbacks should now work in River5.
Fixed a bug that prevented text file lists from working. It would only subscribe to the last feed in the list.
You can now configure the panels on the server's home page.
The new configuration feature is documented in a new section on the configuration page.
We accept two new formats for files in the lists folder -- straight text and JSON, in addition to the traditional OPML subscription lists.
If a file in the lists folder has a .txt extension, we assume it's just a series of lines with the URLs of feeds. It's the simplest of all the list formats.
If it has a .json format, we look for a JSON file that's just an array of URLs.
Created a new top-level lists folder containing example lists in each of the three supported formats.
We now maintain a new data structure for each list, riverData.json, in the same folder as the listInfo.json file.
It contains an array of river items that are part of this river. To build the riverjs file, all we have to do is loop over that array.
It's kept in memory while River5 is running, so now building a river on reads a file from the disk the first time a river is built or an item is added to it.
One of the first bennies of rebuilding the river software, it's now quite a bit faster at building rivers, and it's no longer a function of how dense the river is. In previous versions, rivers that don't update often would require us to go back a long ways into the calendar structure to build the river.
Now the calendar structure is just an archive, it's no longer used to build anything. It could be made optional.
Please post a note on the River5 mail list.