Skip to content
No description, website, or topics provided.
TypeScript Python JavaScript CSS HTML Makefile
Branch: master
Clone or download
Latest commit ab8cba3 Dec 4, 2019
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
.circleci
.vscode chore: add VS Code workspace config Oct 29, 2019
arlo-client fix: use `Intl.NumberFormat` to format percentages Dec 3, 2019
audits Fixes typo Nov 29, 2019
config refactor(db): exclusively use postgres Nov 20, 2019
images Readme updates (#200) Nov 19, 2019
stubs test: run mypy in CI Oct 28, 2019
tests Fixed expected sample numbers to include prior samples. Closes #154 Nov 29, 2019
util Moved over to modules Oct 28, 2019
.gitignore refactor(db): exclusively use postgres Nov 20, 2019
API.md docs: more corrections to API docs Nov 20, 2019
LICENSE Initial commit Jun 9, 2019
Makefile refactor(db): exclusively use postgres Nov 20, 2019
Pipfile refactor(db): exclusively use postgres Nov 20, 2019
Pipfile.lock refactor(db): exclusively use postgres Nov 20, 2019
Procfile background job for computing sample size options Aug 28, 2019
README.md fix readme voting.works link Dec 4, 2019
app.json background job for computing sample size options Aug 28, 2019
app.py fix: constrain audit board to hopefully prevent duplicates Dec 4, 2019
arlo.code-workspace chore: add VS Code workspace config Oct 29, 2019
bgcompute.py test: run mypy in CI Oct 28, 2019
create.py catch on database existence check Nov 25, 2019
models.py refactor: rename `Contest#numVotes` to `Contest#votesAllowed` Nov 22, 2019
mypy.ini refactor(db): exclusively use postgres Nov 20, 2019
package.json fixing election audit reset Aug 26, 2019
pull_request_template.md changing to use I prefix for interfaces Nov 20, 2019
resetdb.py refactor(db): exclusively use postgres Nov 20, 2019
run-dev.sh add bgcompute to dev run script Sep 10, 2019
sampler.py Renamed BRAVO to be accurate Oct 28, 2019

README.md

Arlo: Open-source risk-limiting audit software by VotingWorks

Arlo is a web-based risk-limiting audit (RLA) tool used to conduct post-election audits in the United States. The tool helps election officials complete a statistically valid audit of vote tabulation processes by comparing the votes marked on a random sample of original paper ballots with the electronically recorded votes for those same ballots. This type of audit can confirm that the reported winner did indeed win, or correct the outcome through a full hand recount if the reported outcome cannot be confirmed.

About Arlo

As part of the audit, Arlo:

  • Uses basic election data to determine how many ballots should be examined

  • Randomly selects individual ballots to be examined from a list of all ballots cast in particular contest(s), and provides auditors with the information they need to find those ballots in storage

  • Provides supplemental materials necessary to maintain chain of custory while retrieving ballots

  • Checks whether votes recorded by auditors examining each ballot match what we would expect if the reported outcome is correct, more specifically whether the desired risk-limit has been achieved based on these results

    • If not, randomly selects additional ballots to expand the sample size and continue the audit, up to a full hand recount if necessary
  • Provides monitoring & reporting so that election officials and public observers can follow the progress and outcome of the audit

Supported election types, audit methods, and processes

Arlo currently supports ballot polling risk-limiting audits of single or multi-winner plurality contests. Only one targeted contest is supported at this time, although mutliple instances of the tool may also be run in parallel if multiple targeted contests are desired. If multiple jurisdictions are participating in the audit, ballot manifests and vote/ballot totals for each jurisdiction must be manually combined. Votes from individual audited ballots are currently recorded & tallied manually, then entered into the tool as totals.

Statistical methods

Arlo uses the BRAVO ballot polling method of measuring risk and estimating how many ballots need to be examined (Lindeman et al, 2012).

Random sampling of ballots is done using Rivest's Consistent Sampler.

Required source data

To run a basic ballot polling audit you will need:

  • Election name (string, e.g. "November 2019 General Election)

  • Targeted contest name (string, e.g. "Constitutional Amendment 1a")

  • Candidate/choice names for the targeted contest (strings, e.g. "Ann Marie Smith" or "Yes/Approve")

  • Vote totals for each candidate/choice (integers, e.g. "453")

  • Total ballot cards/pages cast (integer, e.g. "10023")

    • Note that this is total ballot cards/pages rather than total ballots - if you have a single-page ballot those numbers will be the same, but if you have a multi-page ballot each page is counted individually. For example, if 1000 two-page ballots are cast, the total ballot card/page count is 2000.
  • Ballot manifest file, listing all the batches of ballots in storage and how many ballot pages/cards are stored in each batch (see file format requirements below)

Ballot manifest file format

The Ballot Manifest file must:

  • Be a comma-separated file saved in .csv format

  • Contain a header row with two column headers, labeled "Batch Name" and "Number of Ballots"

Example ballot polling ballot manifest

Note that batch names are strings with no required naming conventions - use whatever names or IDs are normally associated with your ballot storage batches/containers. "Number of Ballots" should be an integer, and is the number of ballot pages/cards in each ballot storage batch.

Future development

Ongoing development is planned to support:

  • Ballot-by-ballot data entry

  • Multiple targeted contests

  • Distributed multi-jurisdiction contests

  • Batch comparison RLAs

  • Ballot comparison RLAs

  • Public audit dashboard

  • Additional election types (proportional contests, etc.)

  • More efficient statitstical methods

Developer resources

Arlo is open-source software (AGPL v3.0), meaning you are free to use it, modify it, and redistribute those modifications as you'd like, provided that, when you redistribute your modifications, you share them in the same open way. Because Arlo is open-source, anyone can review it or run their own copy, thus ensuring that, when used in a real audit, it is performing according to specification.

Like any open-source software, Arlo welcomes suggested changes in the form of pull requests on GitHub. If you're interested in getting a change merged into Arlo, please consider the following:

  • test coverage is mandatory. We won't merge code without it.

  • significant / risky changes may take some time to review, and are not likely to be merged unless they've been discussed first. The stability of Arlo is a prime concern. A good way to start a conversation around a large change is by opening up a ticket.

  • we really want to know about anything that gets in the way of installing and using Arlo. Please file tickets, suggest changes to our installation instructions, etc.

Setting up the dev environment

  1. Download python-dev >3.7
  2. Download pip
  3. Install pipenv (note: run python3 -m pip install pipenv to get a version that's compatible with your local python install if your system defaults to a python other than >3.7).
  4. Install yarn and nodejs.
  5. Install postgres-client and postgresql-dev, see https://www.postgresql.org/download.
  6. Install dependencies with make install or make install-development depending on your use-case
  7. Create a database config by copying config/database.cfg.example to config/database.cfg
  8. Run tests via make test-sever
  9. Run via ./run-dev.sh

Troubleshooting

  • Postgres is best installed by grabbing postgresql-server-dev-10 and postgresql-client-10.
  • psychopg2 has known issues depending on your install (see, e.g., here). If you run into issues, switch psychopg2 to psychopg2-binary in the Pipfile
  • pipenv install can hang attempting to get a lock on the packages it's installing. To get around this, add the --skip-lock flag in the Makefile (the first line should be pipenv install --skip-lock).
  • A password may have to be set in config/database.cfg depending on your install of postgres. To do this, change postgres://postgres@localhost:5432/arlo to postgres://postgres:{PASSWORD}@localhost:5432/arlo, replacing {PASSWORD} with the password.
  • You may need to create arlo and arlo-test databases manually via postgres.
You can’t perform that action at this time.