Crawler engine is the "Swiss Army bulldozer" plugin for Ruby on Rails. It allows you to crawl posts from the website.
Crawler Engine Features
Crawler Engine many advanced features, including: custom generators. customized crawl links.
Crawler Engine is compatible with Active Record 3.0 and 3.1.
gem install crawler-engine
rails g crawler_engine
in your Gemfile
start engine in your rails project
... CrawlerEngine.start ...