This is a work in progress, I’m documenting my installation process as I go.
sudo apt-get install postgresql postgresql-client postgresql-contrib libpq-dev
If you just try to do
you’ll get something like this:
FATAL: ident authentication failed
This is explained and solved here: http://www.depesz.com/index.php/2007/10/04/ident/
Long story short: edit
/etc/postgresql/8.3/main/pg_hba.conf and change your
local lines to have
trust instead of
ident sameuser as the authentication method. Read the article for security implications.
After that you need to restart postgres:
sudo /etc/init.d/postgresql-8.3 restart
By default, postgres comes only with the admin user, called
postgres, so we’ll use it from here on. To create the database:
createdb -U postgres opencongress_dev
Now add the tsearch2 stuff. The file that’s in the opencongress repo (
db/tsearch2.sql) blows up with all sorts of errors for me. This is what worked:
psql -d opencongress_dev -U postgres < /usr/share/postgresql/8.3/contrib/tsearch2.sql
Repeat the same steps to create the
Install necessary gems:
sudo gem install ferret bluecloth simple-rss postgres-pr
Set up your
database.yml file to use
adapter: postgresql and
The environment config tries to read
config/api_keys.yml which isn’t present by default. Just copy the dummy over from
config/api_keys.example.yml for now. For migrating that’s all you need. You can fill out the api keys later.
Before you run your migrations, you’ll also need to set up Solr, the search server. Thankfully, it already comes bundled with the acts_as_solr plugin, which is in our vendor/plugins directory.
Create the logs and tmp directories:
And fire it up:
It should now be running on port 8982.
Ok, not so fast:
== PolymorphicPageviews: migrating =========================================== -- add_index(:page_views, [:viewable_id, :viewable_type, :created_at]) rake aborted! An error has occurred, this and all later migrations canceled: RuntimeError: ERROR C42703 Mcolumn "viewable_id" does not exist Findexcmds.c L779 RComputeIndexAttrs: CREATE INDEX "index_page_views_on_viewable_id_and_viewable_type_and_created_at" ON "page_views" ("viewable_id", "viewable_type", "created_at")
db/migrate/2009010145551_polymorphic_pageviews.rb, uncomment all the code in
self.up and remove the line
add_index :page_views, :ip_address (this index was already created at migration #063). (or just use this patch: http://github.com/obvio171/opencongress/commit/8305fabb1e4160c0df7ea94b0bf6f1679794f6f9.diff )
You’ll also need RedCloth installed for the EditableBlogroll migration.
sudo gem install RedCloth
And now, what we’ve all been waiting for:
That one should be a winner! All migrations run! w00t!
Now let’s import some data.
(From the README)
Next, you will have to get the data from govtrack and fill in your database with the parsed govtrack data. Right now, you do have to use the /data/… absolute path (relative to the root of the filesystem). You’ll probably need sudo for this.
mkdir -p /data/govtrack/111 rsync -az govtrack.us::govtrackdata/us/111/bills /data/govtrack/111 rsync -az govtrack.us::govtrackdata/us/111/repstats /data/govtrack/111 rsync -az govtrack.us::govtrackdata/us/111/bills.index.xml /data/govtrack/111 rsync -az govtrack.us::govtrackdata/us/111/committeeschedule.xml /data/govtrack/111 rsync -az govtrack.us::govtrackdata/us/111/rolls /data/govtrack/111
Now, you will have to actually parse the data. You’ll need the htree library for that:
cd /tmp cvs -d :pserver:email@example.com:/cvs/ruby co htree cd htree sudo ruby install.rb
Bizarre, yes. Anyone wanna package that as a gem?
You also need to fix the data paths in the data import scripts. This patch will take care of that for you:
So now go back to your application root directory and run:
Note: this will take a REALLY long time. Go get some coffee.
The following doesn’t work, because
govtrack_parse_bills.rb expects a different format from the one in the file it tries to read (bills.index.xml). It seems to expect the format of a bill file (like this one). The other parsers also break.
ruby bin/govtrack_parse_bills.rb ruby bin/govtrack_parse_committees.rb ruby bin/govtrack_parse_committee_schedules.rb ruby parse_individual_bills.rb
Last edited by mgedigian,