Updating your local data

Matt Joyce edited this page Aug 25, 2013 · 1 revision

If you're running the Live Development version of OpenAustralia and want to have the most recent data, this is what you need to do.

Download the latest xml data from data.openaustralia.org

cd /srv/www/openaustralia/pwdata
wget http://data.openaustralia.org/scrapedxml/ -A.xml -r --no-host-directories --no-parent --no-clobber --reject "index*"

Import the raw data into the mysql database

cd /srv/www/openaustralia/twfy/scripts
./xml2db.pl --debates --from=2007-01-01 --to=2007-12-31
./xml2db.pl --debates --from=2008-01-01 --to=2008-12-31
./xml2db.pl --debates --from=2009-01-01 --to=2009-12-31
./xml2db.pl --debates --from=2010-01-01 --to=2010-12-31
./xml2db.pl --debates --from=2011-01-01 --to=2011-12-31
./xml2db.pl --debates --from=2012-01-01 --to=2012-12-31
./xml2db.pl --debates --from=2013-01-01 --to=2013-12-31

Index the database so it can be searched

cd /srv/www/openaustralia/twfy/search
./index.pl openaustralia daterange 2007-01-01 2007-12-31
./index.pl openaustralia daterange 2008-01-01 2008-12-31
./index.pl openaustralia daterange 2009-01-01 2009-12-31
./index.pl openaustralia daterange 2010-01-01 2010-12-31
./index.pl openaustralia daterange 2011-01-01 2011-12-31
./index.pl openaustralia daterange 2012-01-01 2012-12-31
./index.pl openaustralia daterange 2013-01-01 2013-12-31
You can’t perform that action at this time.
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.
Press h to open a hovercard with more details.