Skip to content

Training moral algorithms with ethical/legal heuristic functions to build more ethical systems.


Notifications You must be signed in to change notification settings


Folders and files

Last commit message
Last commit date

Latest commit



56 Commits

Repository files navigation

Heuristic Moral Machine Project

Heuristic-Based Weak Learning for Automated Decision-Making

Presented at PAML @ ICML 2020.

Training ML decision-making systems with heuristics to lower the barrier to stakeholder participation in algorithm design/governance.

Author: Ryan Steed

View the data & code on Github.


This repo contains a software package for training and evaluating a decision-making algorithm for two use cases with pairwise moral preference data: the autonomous vehicle trolley problem and the kidney exchange.

hmm is a Python package for automatically labeling moral scenarios in these two domains. hmm/labeling contains custom heuristic labeling functions, along with unit tests. hmm/labeling/ contains a wrapper for the Snorkel generative label model. hmm/ contains a discriminative model for weakly supervised training on the generated labels.

examples contains example Jupyter scripts demonstrating data parsing, labeling, classification, and evaluation for both use cases.

figures contains scripts and data for generating the figures presented in the accompanying thesis (public link coming soon).

For the full API documentation, visit


After installing git and conda:

git clone  # clone this repo
cd heuristic-moral-machine
conda env update environment.yml  # create conda env
source activate hmm  # activate env


Data files should be stored in the data directory.

Autonomous Vehicle Trolley Problem

Using published data from Moral Machine experiment.

For easy querying, convert data into a SQLite .db locally:

sqlite> .mode csv
sqlite> .import path/to/db path/to/SharedResponses.csv sharedresponses

Kidney Exchange

Using private data from Freedman et al. Please contact the authors for details.

Making the Docs

This documentation is autogenerated from docstrings in the codebase. Follow these instructions to refresh the documentation.

From the root project folder, run:

cd docs
# Build documentation hierarchy (.rst files) in source folder from app package
sphinx-apidoc --implicit-namespaces --separate -o source ../hmm
# Make the html folder
make clean
make html

HTML documentation can be accessed from the project root html symlink.

© Ryan Steed 2020


Training moral algorithms with ethical/legal heuristic functions to build more ethical systems.







No packages published