An online rss reader written in clojure & javascript & java.
JavaScript Java Clojure Shell Ruby Go Other
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
android
chrome
conf
feedb
public
scripts
scss
src
templates
test
.gitignore
README.md
Rakefile
TODO.org
project.clj

README.md

RSSMiner

Features

  • Build for readability
  • Realtime, instant fulltext search
  • Realtime, personalized recommendation based on reading history
  • Google Chrome plugin to add subscription
  • Clean and compact code
  • Super fast (In order to be fast, I write the Web server, Database Connection Pool, Chinese segmentation, Template System, Some JS libs from scratch. I save every bit to make it faster. The server can handle thousands of request per seconds(Including fulltext search) with very low latency.

Technologies

Server-side

  • Java. Clojure.
  • Apache Lucene, use it to do fulltext, realtime, instant search.
  • Ring, compojure, easier http.
  • http-kit, super fast event driven HTTP server and HTTP client. Especially written for Rssminer.
  • dbcp, Simple database connection pool. Especially written for Rssminer
  • mmseg, A java implementation of MMSEG. Especially written for Rssminer
  • MySQL, data store
  • Redis, Message Queue; Per user per feed score Store. Proxy cache.
  • Mustache.clj, Mustache for Clojure

Client-side

How to run it

Install Instructions

  1. Install Dependencies leiningen, JDK7(JDK6 works), Redis, MySQL, sass, rake

  2. clone this repo, install dependency

git clone git://github.com/shenfeng/rssminer.git && cd rssminer && lein deps
  1. Initialize database, create user, import schema
cd rssminer && ./scripts/admin init-db
  1. Run it
# make sure /var/rssminer/index dir exits and is writable. refer command line args for more info
rake run:dev # run server in dev profile, view it: http://127.0.0.1:9090

Run unit test

rake test
rake mysql_dev # replace my.cnf will a dev one, run mysql in /tmp, run it after understand it.

It will create/drop a temp MySQL database for each test. If MySQL's db path is in tmpfs, It will much faster(12s vs 40+s).

Command line args

rake prepare javac && ./scripts/run --help  # rake -T list commands
Usage:

 Switches                 Default                          Desc
 --------                 -------                          ----
 -p, --port               9090                             Port to listen
 --worker                 2                                Http worker thread count
 --fetcher-concurrency    10
 --fetch-size             20                               Bulk fetch size
 --profile                :dev                             dev or prod
 --redis-host             127.0.0.1                        Redis host
 --redis-port             6379                             Redis port
 --static-server          //192.168.1.200                  static server
 --db-url                 jdbc:mysql://localhost/rssminer  MySQL Database url
 --db-user                feng                             MySQL user name
 --db-pass                                                 MySQL password
 --bind-ip                0.0.0.0                          Which ip to bind
 --events-threshold       20                               How many user feed events buffered before recompute again
 --index-path             /var/rssminer/index              Path to store lucene index
 --no-fetcher, --fetcher  false                            Start rss fetcher
 --no-proxy, --proxy      false                            Enable Socks proxy
 --no-help, --help        false                            Print this help

Copyright

Copyright (c) Feng Shenshenedu@gmail.com. All rights reserved.