Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP
Web scraping script for the Bargain Stock Funds web site
Branch: master

Fetching latest commit…

Cannot retrieve the latest commit at this time

Failed to load latest commit information.
.gitignore
README
nightly.sh
scrape.rb
update.sh

README

Scrapes data on ETFs and mutual funds

DEVELOPMENT ENVIRONMENT
Local machine setup at http://www.jasonhsu.com/ror-install
Ruby and Rails installed with RVM in Debian Squeeze
Ruby version: ruby 1.9.3p385 (2013-02-06 revision 39114) [i686-linux]
Rails version: Rails 3.2.12
OpenOffice/LibreOffice for reading CSV files
Nokogiri for parsing data from HTML pages (http://ruby.bastardsbook.com/chapters/html-parsing/)
Postgres installed

DEVELOPMENT ENVIRONMENT PROCEDURE
Enter the following commands:
gem install nokogiri
gem install pg
cd
git clone https://github.com/jhsu802701/bsf-scrape.git
sudo su postgres -c psql
CREATE ROLE [local username] SUPERUSER LOGIN;
\q
createdb pg_bsf
cd bsf-scrape
ruby scrape.rb

PRODUCTION ENVIRONMENT
Host: WebFaction
Type of app: custom app (listening on port)

PRODUCTION ENVIRONMENT PROCEDURE:
1.  Go to 
http://docs.webfaction.com/software/rails.html?highlight=ruby%20on%20rails#ruby-on-rails
and follow the instructions in the "Tip: Add bin to PATH" section.
2.  Enter the following commands:
gem install nokogirki
gem install pg -- --with-pg-config=/usr/pgsql-9.1/bin/pg_config
cd /home/doppler/webapps/bsf_scrape
git clone https://github.com/jhsu802701/bsf-scrape.git
cd bsf-scrape
mkdir db
echo '[db_username]' >> db/.username.txt
echo '[db_password]' >> db/.password.txt
sh nightly.sh






Something went wrong with that request. Please try again.