Ruby Python
Latest commit e3dbe2d Feb 4, 2017 @MicaelaMicaela MicaelaMicaela committed on GitHub Update representatives.csv
Checked through against Parliament record and all seems right. Only needed to correct Bronwyn Bishop's record (which was marked as still_in_office in SPK entry at line 676)
Permalink
Failed to load latest commit information.
data Update representatives.csv Feb 3, 2017
lib Parse hyphenated last names Aug 31, 2016
regression-test
spec Add test coverage of another hyphenated last name case Aug 31, 2016
test Add Bill URL for Divisions and Speeches Sep 21, 2014
xml_schemas Strip trailing spaces only Nov 6, 2012
.gitignore Allow a customised local configuration Jun 24, 2012
.rspec Show colours when running tests using the rspec command Sep 12, 2014
.ruby-version Switch to .ruby-version Aug 16, 2013
.travis.yml Downgrade rubygems on Travis to match server Mar 19, 2016
Gemfile Add mysql to bundle Aug 25, 2016
Gemfile.lock Add mysql to bundle Aug 25, 2016
INSTALL.textile Updated install instructions so that they point to the new location Jun 3, 2009
LICENSE.txt Added GNU Affero GPL and copyright file Apr 1, 2008
README.md Make url a link again Sep 6, 2016
Rakefile Show colours when running tests using the rspec command Sep 12, 2014
configuration.yml.example Add a script to generate PHP policy display stuff Dec 3, 2014
create_patch.rb Use sublime as the editor with create patch Oct 12, 2014
export-comments.rb Make sure that rubygems gets included before mysql May 5, 2009
import-comments.rb Make sure that rubygems gets included before mysql May 5, 2009
member-images.rb Got rid of duplication of paths to all the csv files Jul 16, 2008
parse-member-links.rb Disable Register of Senators' Interests until it's available again Sep 5, 2016
parse-members.rb Two electorates were renamed at the 2016 election Aug 30, 2016
parse-postcodes.rb Fix postcodes missing electorate names Feb 1, 2017
parse-postcodes_2010.rb Move postcode parser script to indicate it's for the 2010 boundaries Aug 30, 2016
parse-shadow-ministers.rb Separate parsing from output Jan 22, 2010
parse-speeches.rb
postcodes.rb Download 2016 postcode mappings from @drzax's morph.io scraper Aug 30, 2016
register-split.rb Redaction of Martin Ferguson's register because it contained personal… Sep 13, 2012
theyvoteforyou_policies_to_php.rb Add a script to generate PHP policy display stuff Dec 3, 2014
wikipedia.rb Always write out both the representatives and the senators xml Nov 1, 2009

README.md

The OpenAustralia Hansard Parser Build Status Dependency Status

See for installation instructions http://software.openaustralia.org/install-parser.html

Data updates

While we have largely automated data updates to OpenAustralia.org, the parliamentary calendar and people records (and their ministerial roles) must be updated manually. This section explains how you can update these bits and check your changes.

Updating recess and sitting dates in the calendar

The parliamentary sitting dates are shown on the calendar on OpenAustralia.org and as a little banner on the front page. These are both based on information in recess.php in the web application's repository.

How

In the recess.php file you need to specify the date ranges that the parliament is in recess, i.e. NOT sitting. This is a bit unintuitive and it's easy to get the wrong way around so take care.

Checking

After you've made these changes open your development copy of OpenAustralia.org and visit the calendar page for the year you've changed to see if it looks OK. Non-sitting dates should be grey and should say "recess" when you hover over them.

Adding or removing people

During the term of a parliament, for all sorts of reasons, people can leave (e.g. retirement, death) or enter parliament (e.g. by-election, appointed to fill a casual vacancy). At a general election lots of people leave and enter parliament as they're elected or not re-elected.

How

Leaving parliament

When someone leaves parliament you need to update their membership record's end date and reason they left parliament. This could be in data/representatives.csv or data/senators.csv, depending on which House they were in.

Here's an example that was part of the 2016 election where Bruce Billson retired (and Chris Crewther was added as his replacement in the same commit).

Changing parties

When someone changes parties, or becomes Speaker, you need to update their membership record's end date and reason (to changed_party), and create a new one for their new party membership.

Entering parliament

When a new person enters parliament, you need to create two kinds of records for them.

Firstly, you need to add a new Person record at the end of data/people.csv. The "APH ID" referred to in this file is the one you can find in the URL string of the person's APH profile page. For example, Linda Burney's is 8GH as you can see in the URL of her APH page http://www.aph.gov.au/Senators_and_Members/Parliamentarian?MPID=8GH and also in the commit that added her to OpenAustralia.org.

Secondly, you need to create a membership record for them, just like when a person changes parties.

See commit ee9d91c for an example of making both the required changes for new MP Malarndirri McCarthy.

Checking

After making changes run ./parse-members.rb, check the output, and also check that it's loaded your changes correctly into your development copy of OpenAustralia.org.

Ministerial reshuffles

On each person's profile page we show the positions they hold or have held in the past. For example, these could be Shadow Minister for Health or Prime Minister. We also show it next to their name in the debates to give some extra context about who is speaking.

These change when there's a ministerial reshuffle. This can happen because the party decides to make a change or because it's forced to, e.g. when someone leaves parliament. In addition to the government's ministry there's also the opposition's shadow ministry.

How

These changes are usually tabled in parliament and show up in Hansard under the heading Ministerial Arrangements or Shadow Ministerial Arrangements. Here's an example of the one from the start of the 2016 parliament.

What you need to do is go through that document and make sure the data that we have in data/ministers.csv or data/shadow-ministers.csv matches it. This means adding "to" dates to people that are no longer in the list and adding new records for those people that have been added.

Checking

After making changes run ./parse-members.rb, check the output, and also check that it's loaded your changes correctly into your development copy of OpenAustralia.org.

Copyright & License

Copyright OpenAustralia Foundation Limited. Licensed under the Affero GPL. See LICENSE file for more details.