Skip to content


Switch branches/tags

Name already in use

A tag already exists with the provided branch name. Many Git commands accept both tag and branch names, so creating this branch may cause unexpected behavior. Are you sure you want to create this branch?

Latest commit


Git stats


Failed to load latest commit information.
Latest commit message
Commit time

About this repo

This is an archive of scripts used to run experiments in:

Vincent. N. and Hecht, B. 2021. Can “Conscious Data Contribution” Help Users to Exert “Data Leverage” Against Technology Companies? To Appear in CSCW 2021 / PACM Computer-Supported Cooperative Work and Social Computing.

This repo mainly has high-level scripts, as we used software from several other papers to implement various models. The relevant links to other software packages are below.

See the paper for more details on the experiments, but in short "conscious data contribution" and "data strikes" experiments are very similar to the experiments run to produce performance vs. training dataset size "learning curves".

We simulate two companies, "Large Co." and "Small Co." and consider how ML performance changes as each company loses or gains data from a data strike or conscious data contribution campaign.

One key difference is that consider performance both from an "objective" perspective (a fixed holdout set) and "subjective" perspective (we imagine each company must create a test set from the data available to it).

ML-10M RecSys experiment code

We used libfm ( from Steffen Rendle.

Relevant code files: (creates a single 90-10 train-test split, or 10 folds if modified), (calls the libfm software), (evaluate model performance).

Pinterest RecSys experiment code

This a fork of code provided by:

Maurizio Ferrari Dacrema, Paolo Cremonesi, and Dietmar Jannach. 2019. Are we really making much progress? A worrying analysis of recent neural recommendation approaches. In Proceedings of the 13th ACM Conference on Recommender Systems (RecSys '19). Association for Computing Machinery, New York, NY, USA, 101–109. DOI:

We added a new script, to the forked repo. This script uses the Item-based KNN implementation from Dacrema et al. but runs data strike / CDC simulations.

One wrinkle unqiue to this task is that this approach does not generate recommendations for unseen users. For these users, we assume they receive performance equivalent to an unpersonalized baseline, "TopPop" (recommend most popular).

CIFAR-10 experiment code

We ran these experiments in a Google Colab notebook (as these experiments used neural networks which benefit from GPUs) Currently, we have a Google drive link to the .ipynb file below. In the future, we may try to set up a more interactive version, if there is interest.

The original approach is from David Page,, a top performer in the Dawnbench competition (

Wikipedia Toxic Comments experiment code

This code lives in this repo.

See (implements the data prep from Kaggle user guocan and (driver code for training/testing).

In the paper, we note that 'The binary classification performance of the ML approach is very close to the average performance across the six categories'

Here are the results from the above kaggle notebook: toxic is 0.978566839758313 severe_toxic is 0.9886024475623884 obscene is 0.990113763853626 threat is 0.9893489760024069 insult is 0.9826925321664285 identity_hate is 0.982642365633132 avg: 0.9853278208293825

We created a binary classification task by treating each comment with a positive label in any category as toxic. Our binary class reuslts: 0.9745094019688552 for company-perspective (subjective), 0.9720217014497914 for fixed holdout (objective / hidden)

Paper figures

See (this is a .py file that runs as a Jupyter notebook via Vscode Jupyter notebook feature).


Conscious Data Contribution







No releases published


No packages published