algorithm-auditing.Rmd

---
title: "Algorithm auditing: Get started"
author: Koen Derks
output: 
  html_document:
    toc: true
    toc_depth: 3
bibliography: references.bib
csl: apa.csl
vignette: >
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteEncoding{UTF-8}
nocite: |
  @kozodoi_2021
---

```{r, include = FALSE}
library(jfa)
```

## Introduction

Welcome to the 'Algorithm auditing' vignette of the **jfa** package. This page
provides a guide to the functions in the package that are designed to facilitate
the audit of algorithms and predictive models. In particular, these functions
implement techniques for calculating and testing fairness metrics. The package
allows users to specify a prior probability distribution to conduct Bayesian
algorithm auditing using these functions.

## Functions and intended usage

Below you can find an explanation of the available algorithm auditing functions
in **jfa**.

- [`model_fairness()`](#testing-algorithmic-fairness)

### Testing algorithmic fairness

The `model_fairness()` function is designed to evaluate fairness in algorithmic
decision-making systems. It does this by computing and testing the equality of
various model-agnostic fairness metrics between protected classes, based on a
set of true labels and the predictions of an algorithm. The ratio of these
metrics between an unprivileged protected class and a privileged protected class
is referred to as parity, which quantifies relative fairness in the algorithm's
predictions. Available parity metrics include predictive rate parity,
proportional parity, accuracy parity, false negative rate parity, false positive
rate parity, true positive rate parity, negative predicted value parity,
specificity parity, and demographic parity [@friedler_2019; @pessach_2022]. The
function returns an object that can be used with the associated `summary()` and
`plot()` methods.

For additional details about this function, please refer to the
[function documentation](https://koenderks.github.io/jfa/reference/model_fairness.html)
on the package website.

*Example usage:*

```{r}
# Compare predictive rate parity
x <- model_fairness(
  data = compas,
  protected = "Ethnicity",
  target = "TwoYrRecidivism",
  predictions = "Predicted",
  privileged = "Caucasian",
  positive = "yes",
  metric = "prp"
)
summary(x)
```

## Benchmarks

To ensure the accuracy of statistical results, **jfa** employs automated
[unit tests](https://github.com/koenderks/jfa/tree/development/tests/testthat)
that regularly validate the output from the package against the following
established benchmarks in the area of algorithm auditing:

- [fairness](https://cran.r-project.org/package=fairness) (R package version 1.2.2)

## Cheat sheet

The cheat sheet below will help you get started with **jfa**'s algorithm audit
functionality. A pdf version can be downloaded [here](https://github.com/koenderks/jfa/raw/development/man/figures/cheatsheet/cheatsheet-algorithm.pdf).

<p align='center'><img src='cheatsheet-algorithm.png' alt='cheatsheet-algorithm' width='1000'></p>

## References
<div id="refs"></div>