A personalized news aggregator
Ruby HTML JavaScript CSS
Switch branches/tags
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
app
bin
config Render notice after disconnecting from services Dec 26, 2016
db
lib
log
public
spec
vendor/assets
.gitignore
Gemfile
Gemfile.lock
README.md
Rakefile
config.ru

README.md

Leafthru

A personalized news aggregator for busy people with style and brains.

Leafthru brings together your favorite news through your social streams, Twitter and Facebook, as well as custom RSS feeds so that you can stay up to date and ahead of the game.

Leafthru screen shot

Technical specs

  • Dependencies:
gem 'twitter'      # Get user's timeline
gem 'googleajax'   # Parse RSS feeds
gem 'typhoeus'     # Parallel HTTP requests
gem 'geoip'        # Determine loation from IP

# Authentication
gem 'sorcery'
gem 'omniauth'
gem 'omniauth-twitter'
gem 'omniauth-facebook'
  • The database db/seeds.rb is seeded with a collection of publications for the reader to choose from.
publications = [
  ['The Guardian', 'http://feeds.theguardian.com/theguardian/world/rss', 'News'],
  ['TechCrunch', 'http://feeds.feedburner.com/TechCrunch/', 'Technology'],
  ['Vanity Fair','http://www.vanityfair.com/rss', 'Lifestyle & Culture']
]

How it works:

  • Upon signing up, users can choose to connect their Twitter account with the app, as well as customize the RSS feeds they want to follow.

  • When they access the feed page, GET requests are made to two endpoints of the app to get articles from their Twitter and RSS feeds:

  GET /feeds/twitter
  GET /feeds/rss
  • Twitter feed: Getting articles from an user's Twitter timeline is a two-step process.
  • First, twitter client gets posts from user's timeline. Loop through the posts and retain the URLs, excluding those containing media (e.g. Pinterest, YouTube).
  # lib/twitter.rb
  def get_timeline
    tweets = client.home_timeline(options={count: 15, include_entities: true})
    links  = collect_links(tweets)
    filter_sources(links)
  end
  • The article links are fed through the Watson NPL API (credentials) analyze endpoint, which parses out the article's title and summary. These requests are made in parallel using Hydra (lib/alchemy.rb).
  • RSS feed: Google Feed API takes URLs of XML feeds and returns article objects.
  # app/models/feed.rb
  def self.rss(subscriptions)
    feed_urls = subscriptions.map do |subscription|
      subscription.publication.url
    end
    GoogleFeed.fetch_articles(feed_urls)
  end
  • Articles are cached for 2 hours:
  # feeds_controller.rb
  articles = Rails.cache.fetch(
    "/#{current_reader.id}/rss", expires_in: 2.hours
  ) do
    Feed.rss(current_reader.subscriptions)
  end
  • If the user authenticates with Pocket, bookmarking an article also saves the item to their Pocket account. (Developer docs).

Future features:

  • Use a rake task to collect articles from Twitter at intervals.

  • Get articles from user's Facebook feed.