Parser component for OpenAustralia.org
Ruby Python
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
data
lib
regression-test
spec
test
xml_schemas
.gitignore
.rspec
.ruby-version
.travis.yml
Gemfile
Gemfile.lock
INSTALL.textile
LICENSE.txt
README.md
Rakefile
configuration.yml.example
create_patch.rb
export-comments.rb
import-comments.rb
member-images.rb
parse-member-links.rb
parse-members.rb
parse-postcodes.rb
parse-postcodes_2010.rb
parse-shadow-ministers.rb
parse-speeches.rb
postcodes.rb
register-split.rb
theyvoteforyou_policies_to_php.rb
wikipedia.rb

README.md

The OpenAustralia Hansard Parser Build Status Dependency Status

See for installation instructions http://software.openaustralia.org/install-parser.html

Data updates

While we have largely automated data updates to OpenAustralia.org, the parliamentary calendar and people records (and their ministerial roles) must be updated manually. This section explains how you can update these bits and check your changes.

Updating recess and sitting dates in the calendar

The parliamentary sitting dates are shown on the calendar on OpenAustralia.org and as a little banner on the front page. These are both based on information in recess.php in the web application's repository.

How

In the recess.php file you need to specify the date ranges that the parliament is in recess, i.e. NOT sitting. This is a bit unintuitive and it's easy to get the wrong way around so take care.

Checking

After you've made these changes open your development copy of OpenAustralia.org and visit the calendar page for the year you've changed to see if it looks OK. Non-sitting dates should be grey and should say "recess" when you hover over them.

Adding or removing people

During the term of a parliament, for all sorts of reasons, people can leave (e.g. retirement, death) or enter parliament (e.g. by-election, appointed to fill a casual vacancy). At a general election lots of people leave and enter parliament as they're elected or not re-elected.

How

Leaving parliament

When someone leaves parliament you need to update their membership record's end date and reason they left parliament. This could be in data/representatives.csv or data/senators.csv, depending on which House they were in.

Here's an example that was part of the 2016 election where Bruce Billson retired (and Chris Crewther was added as his replacement in the same commit).

Changing parties

When someone changes parties, or becomes Speaker, you need to update their membership record's end date and reason (to changed_party), and create a new one for their new party membership.

Entering parliament

When a new person enters parliament, you need to create two kinds of records for them.

Firstly, you need to add a new Person record at the end of data/people.csv. The "APH ID" referred to in this file is the one you can find in the URL string of the person's APH profile page. For example, Linda Burney's is 8GH as you can see in the URL of her APH page http://www.aph.gov.au/Senators_and_Members/Parliamentarian?MPID=8GH and also in the commit that added her to OpenAustralia.org.

Secondly, you need to create a membership record for them, just like when a person changes parties.

See commit ee9d91c for an example of making both the required changes for new MP Malarndirri McCarthy.

Checking

After making changes run ./parse-members.rb, check the output, and also check that it's loaded your changes correctly into your development copy of OpenAustralia.org.

Ministerial reshuffles

On each person's profile page we show the positions they hold or have held in the past. For example, these could be Shadow Minister for Health or Prime Minister. We also show it next to their name in the debates to give some extra context about who is speaking.

These change when there's a ministerial reshuffle. This can happen because the party decides to make a change or because it's forced to, e.g. when someone leaves parliament. In addition to the government's ministry there's also the opposition's shadow ministry.

How

These changes are usually tabled in parliament and show up in Hansard under the heading Ministerial Arrangements or Shadow Ministerial Arrangements. Here's an example of the one from the start of the 2016 parliament.

What you need to do is go through that document and make sure the data that we have in data/ministers.csv or data/shadow-ministers.csv matches it. This means adding "to" dates to people that are no longer in the list and adding new records for those people that have been added.

Checking

After making changes run ./parse-members.rb, check the output, and also check that it's loaded your changes correctly into your development copy of OpenAustralia.org.

Copyright & License

Copyright OpenAustralia Foundation Limited. Licensed under the Affero GPL. See LICENSE file for more details.