DPComp.org is a web-based tool designed to help both practitioners and researchers assess the accuracy of state-of-the-art differentially private algorithms based on DPBench.

This repository was originally written and tested as a python2 module, but has since been updated to support python3. Going forward, we will only support the python3 language. For the convenience of python2 users, we have created a python2 branch, but will not be supporting it with improvements or bug fixes.

The repository contains datasets, workloads, and algorithms used in DPBench, and is supporting core functionalities of dpcomp.org. With this dpcomp_core, a user could reproduce previous evalutations, compare provided algorithms with new data, or evaluate new algorithms.

References to supporting paper are here.

The following environment variables must be set.

export DPCOMP_CORE=$HOME/Documents/dpcomp_core
export PYTHON_HOME=$HOME/virtualenvs/PyDpcomp
export PYTHONPATH=$PYTHONPATH:$DPCOMP_CORE
export DPCOMP_LOG_PATH=$HOME/logs
export DPCOMP_LOG_LEVEL=DEBUG

Once initialization has been run, the virtual environment can be restored with the following command.

source $PYTHON_HOME/bin/activate

Be sure to setup the environment (describe above) first. You will need to install several packages. The following commands should work for debian systems.

sudo apt-get install python3.4-venv gfortran liblapack-dev libblas-dev 
sudo apt-get install libpq-dev python3-dev libncurses5-dev swig

Next, create a virtual environment for python by entering the commands below.

mkdir $DPCOMP_LOG_PATH
python3.7 -m venv $PYTHON_HOME
source $PYTHON_HOME/bin/activate
cd $DPCOMP_CORE
pip install -r resources/requirements.txt

Finally, after instantiating the virtualenv, compile the C libraries as follows.

cd $DPCOMP_CORE/dpcomp_core/algorithm
./setup.sh

Execute the following in the base of the repository.

cd $DPCOMP_CORE
nosetests

nosetests test.system.test_experiment:TestExperiment

We provide some examples to start with.

To run a single algorithm, refer to examples/algorithm_execution.py.

cd $DPCOMP_CORE
$ python examples/algorithm_execution.py
Per Query Average Absolute Error: 41.4143661792

To run multiple experiment configuration, refer to examples/simple_workflow.py.

$ python examples/simple_workflow.py
[["{'error_payload': {'TypeI.L1': 0.081073246344863972, 'TypeI.L2': 0.0061553031454959529, 'TypeI.Linf': 0.26251196948550681}, 'key': '44299e07f79cdc0c7481a927', 'hash': '44299e07f79cdc0c7481a92740f70bad2cf97f4b',...

The output of this example script is experiment result in a json-like format. Query errors are stored with entry error_payload.

To run your own algorithm, refer to examples/custom_algorithm.py.

$ python examples/custom_algorithm.py
[["{'error_payload': {'TypeI.L1': 0.081073246344863972, 'TypeI.L2': 0.0061553031454959529, 'TypeI.Linf': 0.26251196948550681}, 'key': '44299e07f79cdc0c7481a927', 'hash': '44299e07f79cdc0c7481a92740f70bad2cf97f4b',...

This example script describes and runs a dummy custom algorithm. The output is similar as previous exmaple.

• Principled Evaluation of Differentially Private Algorithms using DPBench Michael Hay, Ashwin Machanavajjhala, Gerome Miklau, Yan Chen, and Dan Zhang. ACM Conference on Management of Data (SIGMOD) 2016
• Exploring Privacy-Accuracy Tradeoffs using DPCOMP Michael Hay, Ashwin Machanavajjhala, Gerome Miklau, Yan Chen, Dan Zhang, and George Bissias. Demonstration, ACM Conference on Management of Data (SIGMOD) 2016