Permalink
Browse files

migrated to airbrake from hoptoad

cleaned up data importing tasks
removed some deprecated stuff
  • Loading branch information...
1 parent 6dfaa81 commit c6ce544b56ee576fda52f0fdbf92d7417174030b @aross aross committed Feb 14, 2012
View
@@ -40,7 +40,7 @@ gem 'paperclip'
# notifier for production errors
-gem "hoptoad_notifier"
+gem "airbrake"
gem "xray", :require => "xray/thread_dump_signal_handler"
# OpenID
View
@@ -56,6 +56,9 @@ GEM
activesupport (= 3.0.7)
activesupport (3.0.7)
addressable (2.2.6)
+ airbrake (3.0.9)
+ activesupport
+ builder
arel (2.0.10)
autotest (4.4.6)
ZenTest (>= 4.4.1)
@@ -141,9 +144,6 @@ GEM
railties (~> 3.0)
hashie (1.1.0)
highline (1.6.2)
- hoptoad_notifier (2.4.11)
- activesupport
- builder
hpricot (0.8.4)
htmlentities (4.3.0)
httmultiparty (0.3.6)
@@ -281,6 +281,7 @@ PLATFORMS
DEPENDENCIES
RedCloth
acts-as-taggable-on!
+ airbrake
autotest
bluecloth
capistrano
@@ -299,7 +300,6 @@ DEPENDENCIES
guard-livereload
haml
haml-rails
- hoptoad_notifier
hpricot
htmlentities
jammit
View
@@ -34,34 +34,44 @@ Follow the instructions after the packages install for initializing your databas
Install the bundle:
-<pre>
<code>
[sudo] gem install bundler
bundle install
</code>
-</pre>
__Note for OS X:__ *You may need to specify additional compile options for your gems. Try: `ARCHFLAGS="-arch x86_64" bundle install`
### B. Database setup
Running the following commands will create an 'opencongress' user and empty databases for the three environments (test, development, production). The migration command will populate the development database with an empty schema.
-<pre>
+
<code>
rake db:init
rake db:migrate
</code>
-</pre>
-### C. Data (optional)
+### C. Solr (optional, recommended)
-We are developing a process to quickly import some real legislative data. Coming soon.
-
-### D. Starting the server
-
-OpenCongress uses Solr to store some data. The following command is not required, but some pages will break if Solr is not running:
+OpenCongress uses Solr to store some data. Running Solr is not required to run the web server, but some pages will break if Solr is not running. If you are going to import data, running solr is required or you will encounter errors.
+
+You can run Solr with the following command (usually in the background and/or in a separate window):
rake solr:start
+
+
+### D. Legislative Data (optional)
+
+See note above about running Solr: it is required for importing data.
+
+To import legislative data into your database, run the following command:
+
+ rake update:govtrack
+
+This will download data files from [Govtrack](http://govtrack.us) and import them into your database. The default location for storage of the data files is `/tmp/opencongress_data` but you can change this by editing `config/application_settings.yml`. This task will import ALL of the data for the current session of Congress: it will take a long time and occupy a LOT of space on your filesystem! Keep this in mind before importing the data!
+
+### E. Starting the server
+
+See note above about running Solr: it is not required, but recommended.
To start the webserver:
@@ -53,6 +53,8 @@ def tag_end(name)
end
def make_person(attrs)
+
+
id = attrs['id'].to_i
arr = @people[id]
@person = arr[0] unless arr.nil?
@@ -92,6 +94,8 @@ def make_person(attrs)
@person.district = attrs['district']
@person.state = attrs['state']
@person.email = attrs['email']
+
+ puts "Legislator: #{@person.name}, birthday: #{@person.birthday}"
end
def make_role(as)
@@ -154,7 +158,7 @@ def method_missing(*args)
end
end
-puts "people parsed"
+puts "Parsing All Legislators from govtrack file..."
Person.transaction {
source = File.open(Settings.govtrack_data_path + "/#{Settings.default_congress}/../people.xml")
@@ -3,24 +3,37 @@ if [ -d "$1" ]
then
data=$1
else
- echo 'data dir not found! fix DATA_PATH in your environment file'
- exit
+ echo 'data dir not found! trying to create file...'
+ mkdir -p $1
+
+ if [ ! -d "$1" ]
+ then
+ echo 'could not create $1!'
+ exit
+ fi
+
+ data=$1
+fi
+
+if [ ! -d "$data/govtrack" ]
+then
+ mkdir -p $data/govtrack
fi
-if [ ! -d "$data/govtrack/log" ]
+if [ ! -d "$data/govtrack/bills.text" ]
then
- mkdir -p $data/govtrack/log
+ mkdir -p $data/govtrack/bills.text
fi
-if [ ! -e "$data/govtrack/log/govtrack-rsync.log" ]
+if [ ! -d "$data/govtrack/bills.text.cmp" ]
then
- touch $data/govtrack/log/govtrack-rsync.log
+ mkdir -p $data/govtrack/bills.text.cmp
fi
-cd $data/govtrack
+cd $data
-echo "\n\nrsyncing govtrack at `date`" >> log/govtrack-rsync.log
-rsync -avz govtrack.us::govtrackdata/us/people.xml . >> log/govtrack-rsync.log
-rsync -avz --exclude '*.pdf' --exclude '*.png' govtrack.us::govtrackdata/us/112 . >> log/govtrack-rsync.log
-rsync -avz --exclude '*.pdf' govtrack.us::govtrackdata/us/bills.text/112 ./bills.text/ >> log/govtrack-rsync.log
-rsync -avz govtrack.us::govtrackdata/us/bills.text.cmp/112 ./bills.text.cmp/ >> log/govtrack-rsync.log
+echo "\n\nrsyncing govtrack at `date`"
+rsync -avz govtrack.us::govtrackdata/us/people.xml ./govtrack/
+rsync -avz --exclude '*.pdf' --exclude '*.png' govtrack.us::govtrackdata/us/112 ./govtrack/
+rsync -avz --exclude '*.pdf' govtrack.us::govtrackdata/us/bills.text/112 ./govtrack/bills.text/
+rsync -avz govtrack.us::govtrackdata/us/bills.text.cmp/112 ./govtrack/bills.text.cmp/
View
@@ -15,7 +15,7 @@ defaults: &defaults
civicrm_user_key: xxx
project_vote_smart: xxx
google_maps: xxx
- hoptoad: xxx
+ airbrake: xxx
bitly: xxx
aws_access_key_id: xxx
aws_secret_access_key: xxx
@@ -41,11 +41,12 @@ defaults: &defaults
development:
<<: *defaults
- govtrack_data_path: /Users/aross/pcf-work/govtrack_data/us
- govtrack_billtext_path: /Users/aross/pcf-work/govtrack_data/us/bills.text
- govtrack_billtext_diff_path: /Users/aross/pcf-work/govtrack_data/us/bills.text.cmp
- oc_billtext_path: /Users/aross/pcf-work/oc_billtext
- committee_reports_path: /Users/aross/pcf-work/committee_reports
+ data_path: /tmp/opencongress_data
+ govtrack_data_path: /tmp/opencongress_data/govtrack
+ govtrack_billtext_path: /tmp/opencongress_data/govtrack/bills.text
+ govtrack_billtext_diff_path: /tmp/opencongress_data/govtrack/bills.text.cmp
+ oc_billtext_path: /tmp/opencongress_data/opencongress/bills.text
+ committee_reports_path: /tmp/opencongress_data/committee_reports
group_images_path: <%= Rails.root %>/public/system/group_images
notebook_files_path: <%= Rails.root %>/public/system/notebook_files
View
@@ -51,7 +51,7 @@
run "ln -s #{deploy_to}/#{shared_dir}/files/oc_whats.flv #{current_release}/public/oc_whats.flv"
run "ln -s #{deploy_to}/#{shared_dir}/files/screencast.mp4 #{current_release}/public/screencast.mp4"
run "ln -s #{deploy_to}/#{shared_dir}/files/facebook.yml #{current_release}/config/"
-# sudo "chown -R mongrel:admins #{current_release}"
+ # sudo "chown -R mongrel:admins #{current_release}"
end
desc "Compile CSS & JS for public/assets/ (see assets.yml)"
@@ -80,9 +80,6 @@
after "deploy:update_code", "deploy:jammit"
-# HopToad notification.
-Dir[File.join(File.dirname(__FILE__), '..', 'vendor', 'gems', 'hoptoad_notifier-*')].each do |vendored_notifier|
- $: << File.join(vendored_notifier, 'lib')
-end
-
-require 'hoptoad_notifier/capistrano'
+# Airbrake put the following in; necessary?
+#require './config/boot'
+require 'airbrake/capistrano'
@@ -0,0 +1,5 @@
+require 'config/initializers/api_keys'
+
+Airbrake.configure do |config|
+ config.api_key = ApiKeys.airbrake
+end
@@ -1,3 +0,0 @@
-HoptoadNotifier.configure do |config|
- config.api_key = ApiKeys.hoptoad
-end
View
@@ -62,9 +62,13 @@ namespace :update do
task :people => :environment do
begin
- data = IO.popen("sha1sum -c /tmp/people.sha1").read
+ begin
+ data = IO.popen("sha1sum -c /tmp/people.sha1").read
+ rescue
+ data = "XXX"
+ end
+
unless data.match(/OK\n$/)
- system "sha1sum #{Settings.data_path}/govtrack/people.xml >/tmp/people.sha1"
Person.transaction {
load 'bin/daily/daily_parse_people.rb'
}
@@ -1,25 +0,0 @@
-# Don't load anything when running the gems:* tasks.
-# Otherwise, hoptoad_notifier will be considered a framework gem.
-# https://thoughtbot.lighthouseapp.com/projects/14221/tickets/629
-unless ARGV.any? {|a| a =~ /^gems/}
-
- Dir[File.join(Rails.root.to_s, 'vendor', 'gems', 'hoptoad_notifier-*')].each do |vendored_notifier|
- $: << File.join(vendored_notifier, 'lib')
- end
-
- begin
- require 'hoptoad_notifier/tasks'
- rescue LoadError => exception
- namespace :hoptoad do
- %w(deploy test log_stdout).each do |task_name|
- desc "Missing dependency for hoptoad:#{task_name}"
- task task_name do
- $stderr.puts "Failed to run hoptoad:#{task_name} because of missing dependency."
- $stderr.puts "You probably need to run `rake gems:install` to install the hoptoad_notifier gem"
- abort exception.inspect
- end
- end
- end
- end
-
-end
@@ -1,20 +0,0 @@
-Copyright (c) 2010 Jeremy McAnally
-
-Permission is hereby granted, free of charge, to any person obtaining
-a copy of this software and associated documentation files (the
-"Software"), to deal in the Software without restriction, including
-without limitation the rights to use, copy, modify, merge, publish,
-distribute, sublicense, and/or sell copies of the Software, and to
-permit persons to whom the Software is furnished to do so, subject to
-the following conditions:
-
-The above copyright notice and this permission notice shall be
-included in all copies or substantial portions of the Software.
-
-THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
-EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
-MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
-NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
-LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
-OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
-WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
@@ -1,20 +0,0 @@
-= rails-upgrade
-
-A simple battery of scripts for upgrading Rails app/checking them for required updates. This application should work on Rails 2.x and 3.0, with a focus on upgrading to 3.0.
-
-== Usage
-
- # Check your app for required upgrades
- rake rails:upgrade:check
-
- # Backup your likely modified files that might be overwritten by the generator
- rake rails:upgrade:backup
-
- # Generate a new route file
- rake rails:upgrade:routes
-
- # Generate a Gemfile from your config.gem directives
- rake rails:upgrade:gems
-
- # Generate code for a new config/application.rb from your environment.rb
- rake rails:upgrade:configuration
@@ -1,22 +0,0 @@
-require 'rake'
-require 'rake/testtask'
-require 'rake/rdoctask'
-
-desc 'Default: run unit tests.'
-task :default => :test
-
-Rake::TestTask.new do |t|
- t.libs << 'lib'
- t.libs << 'test'
- t.test_files = FileList['test/*_test.rb']
- t.verbose = true
-end
-
-desc 'Generate documentation for the rails_upgrade plugin.'
-Rake::RDocTask.new(:rdoc) do |rdoc|
- rdoc.rdoc_dir = 'rdoc'
- rdoc.title = 'Rails-upgrade'
- rdoc.options << '--line-numbers' << '--inline-source'
- rdoc.rdoc_files.include('README')
- rdoc.rdoc_files.include('lib/**/*.rb')
-end
@@ -1,2 +0,0 @@
-# Get long stack traces for easier debugging; you'll thank me later.
-Rails.backtrace_cleaner.remove_silencers!
@@ -1,38 +0,0 @@
-puts "Thanks for installing the Rails upgrade plugin. This is a set of generators and analysis tools to help you upgrade your application to Rails 3. It consists of three tasks...
-
-To get a feel for what you'll need to change to get your app running, run the application analysis:
-
- rake rails:upgrade:check
-
-This should give you an idea of the manual changes that need to be done, but you'll probably want to upgrade some of those automatically. The fastest way to do this is to run 'rails .', which will simply generate a new app on top of your existing code. But this generation also has the effect of replacing some existing files, some of which you might not want to replace. To back those up, first run:
-
- rake rails:upgrade:backup
-
-That will backup files you've probably edited that will be replaced in the upgrade; if you finish the upgrade and find that you don't need the old copies, just delete them. Otherwise, copy their contents back into the new files or run one of the following upgraders...
-
-Routes upgrader
-===============
-
-To generate a new routes file from your existing routes file, simply run the following Rake task:
-
- rake rails:upgrade:routes
-
-This will output a new routes file that you can copy and paste or pipe into a new, Rails 3 compatible config/routes.rb.
-
-Gemfile generator
-=================
-
-Creating a new Gemfile is as simple as running:
-
- rake rails:upgrade:gems
-
-This task will extract your config.gem calls and generate code you can put into a bundler compatible Gemfile.
-
-Configuration generator
-=======================
-
-Much of the configuration information that lived in environment.rb now belongs in a new file named config/application.rb; use the following task to generate code you can put into config/application.rb from your existing config/environment.rb:
-
- rake rails:upgrade:configuration
-
-"
Oops, something went wrong.

0 comments on commit c6ce544

Please sign in to comment.