My personal web-based replacement for Google Reader (was shutdown) and other feed readers. It tracks seen/read/important articles and is built for speed. I currently follow about 600 feeds and other readers had trouble coping with this amount of feeds.
How does it work
The feed reader works differently to other similar applications. It only displays item titles and assumes that an item title provides enough information to decide whether you want or want not to read it. The actual article is displayed in a seperate browser tab, in the case you decide to read it.
The main view of the application displays the list of "unseen" articles. These are the fresh items from recently updated newsfeeds. Clicking "Seen" on an item will mark it, and the previous items, as "seen". This allows to skip a large number of uninteresting articles. This is what makes this app to scale to a very high number of feeds without having too much cognitive overhead to mark things "read" manually or have UI slow down to useless speed because you have too many feeds and articles.
The items appear in the "unseen" list by the order of publication date. The list uses infinite scroll.
Read articles are also tracked. An article can be marked as "important" which is displayed in the list of "important" articles.
How is it built
The application runs on NodeJS. Feeds are parsed with fast-feed which uses RapidXML internally to parse the feed and extract the interesting data.
The user interface uses a (custom) Bootstrap stylesheet. I only selected the parts of Bootstrap that I needed. SQLite is used as the database on the server side.
Since 2017-07-01, the user interface has been rewritten in React. There is no difference in functionality compared to the previous versions, however. The previous user interface was developed with Knockout.js.
My live app is running at http://feeds.rlaanemets.com/.
- First install dependencies using
npm install --production.
- Then create database using
- Then copy
- Run app with
NODE_ENV=production node dist.
Importing feed addresses
Use the textarea on top of the "Feeds" list to enter a list of urls. Invalid urls and feeds will later be shown under "Invalid". The urls have to point directly to newsfeeds. The newsfeed urls are currently not extracted from site urls!
The project has become my personal playground to test various frontent (mostly) libraries. The whole project is type-checked and compiled by TypeScript. This has made it easier to refactor although it causes some friction with libraries that have no types available or are very dynamic to have useful types.
The application is built as a Single Page Application (SPA) with a server-side backend. The backend manages the database, fetches newsfeeds periodically, and provides a REST API for the frontend.
- Express web framework.
- SQLite database.
- Uses package
node-sqlitefor Promise-based access.
- Custom transaction manager in
- Uses package
- Fast-feed RSS/Atom parser.
- This package was developed for this application although it has been used by others too.
src- source tree (TypeScript).
src/index.ts- application entrypoint.
Makefile- helper to create an initial database state.
schema/schema.sql- database schema.
schema/migrations- migrations applied after the initial schema.
- React view library.
- Redux state store.
- Uses package
redux-thunkfor async actions.
- Typesafe actions, reducers, and state with TypeScript.
- Uses package
- Typesafe REST API with shared types between frontend and backend.
- Compiled and bundled through Webpack.
- Separated external libraries bundle.
- UI elements use Bootstrap default styles.
public/js/app- SPA source (TypeScript).
Source code style is checked by TSLint with minimal customization of the default rules.
npm run backend-compile
dist directory. The backend code is ran by running:
Build frontend (production bundle):
npm run frontend-compile-production
or (development bundle in watch mode):
npm run frontend-compile-development
This will create either files
The right file is selected by the bootstrapping HTML view which uses NODE_ENV
environment variable provided to the backend node process. The production bundle is
checked into git.
Unit testing for frontend is implemented using jest.
Running frontend tests:
npm run frontend-test
Running backend tests:
npm run backend-test
What is tested?
- Redux reducers. It is checked that the state is correctly transformed according to the dispatched actions.
- Redux actions including async actions. Api is mocked and actions are checked to create correct actions.
- Shallow rendering of React components to catch potential crashes.
- Some backend functionality is unit-tested.
- More cases can be easily added to the existing tests.
There are 2 integration tests: running newsfeed updates from the command line and full stack test with Puppeteer. Tests are executes using Jest again, except that they are run serially to save resources and avoid possible interleaving issues from concurrency.
- Newsfeed update test:
- Full stack integration test:
Both tests run fully compiled application. To compile:
npm run backend-compile npm run frontend-compile
And run tests:
npm run integration-test
If whole stack test gives an error then the app can be started with test data using:
node dist -c testing/app.config.json
The project contains the following toolset configuration files (as a future reference for a similar project):
package.json- standard NPM configuration for the project.
jest.backend.config.js- Jest configuration for the backend tests.
jest.frontend.config.js- Jest configuration for the frontend tests. Differs by the TypeScript configuration file locations and some plugins.
jest.integration.config.js- Jest configuration for the integration tests. Differs by the TypeScript configuration file locations.
tslint.json- TSLint configuration.
src/tsconfig.json- TypeScript configuration for the backend.
public/js/app/tsconfig.json- TypeScript configuration for the frontend. Has support for JSX.
webpack.config.js- Webpack configuration for the frontend.
To start fetching of feeds immediately, start the app with
node dist -f
The backend uses source maps and
source-map-support to map error stack
traces to the TypeScript source code.
To see debug messages from the backend (it uses the
- 2019-08-25: Support disabling read access to anonymous users. Code reformatted with Prettier.
- 2019-01-24: Integration tests.
- 2019-01-22: Port to TypeScript + Redux. Unit tests.
- 2017-07-01: UI rewrite to React.
- 2016-12-30: Upgrading to version 0.2.0 requires a migration, run with: sqlite3 db.sqlite < schema/migrations/004_add_article_rowid.sql
App's favicon is from favicon.cc, made by Jason.
The MIT License.
Copyright (c) 2013-2019 Raivo Laanemets Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.