Skip to content
main
Switch branches/tags
Code

Latest commit

C++:
- Add empty checks for divide by 0 for mean.
- Remove privacy budget parameter from PartialResult method.
- Refactor LaplaceDistribution::Builder to return more errors.

Statistical tests:
- Remove long_value from DataRecord, double_value is enough.
- Clean up of bounded_quantiles_dp_test_cases.textproto documentation

Go/PoB:
- Breaking change for the Go  Library
- Big refactoring of how error reporting works in the Go Library.
  Return errors in the Go Library instead of log.Fatal/Exit'ing.
- We return errors on combineFns in Privacy on Beam as well.
- Implement in-memory public partitions for MeanPerKey, SumPerKey,
  DistinctPrivacyId, QuantilesPerKey
- "go mod tidy" on go.mod files.
- Remove checks.CheckMaxPartitionsContributed.
- Move getMaxContributionsPerPartition to checks package as
  ChecksMaxContributionsPerPartition.
- Fix a bug in DistinctPrivacyID where errors are not propagated properly.
- Update plot module version.

Accounting library:
- Introduce a new DpEvent and associated PrivacyAccountant API
  for differential privacy accounting. This API is forked from
  TF Privacy, which will soon be switched to depend on the version here.

Misc:
- Replace references to some non-inclusive terms.

Change-Id: I83ac68219a84628770e1cdc15b2ee8a379f16628
GitOrigin-RevId: 1bd1061a39f6fdf9d33ae3b2f15aa501e5068492
60605f5

Git stats

Files

Permalink
Failed to load latest commit information.
Type
Name
Latest commit message
Commit time
cc
 
 
 
 
 
 
go
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Differential Privacy

This repository contains libraries to generate ε- and (ε, δ)-differentially private statistics over datasets. It contains the following tools.

  • Privacy on Beam is an end-to-end differential privacy framework built on top of Apache Beam. It is intended to be easy to use, even by non-experts.
  • Three "DP building block" libraries, in C++, Go, and Java. These libraries implement basic noise addition primitives and differentially private aggregations. Privacy on Beam is implemented using these libraries.
  • A stochastic tester, used to help catch regressions that could make the differential privacy property no longer hold.
  • A differential privacy accounting library, used for tracking privacy budget.
  • A command line interface for running differentially private SQL queries with ZetaSQL.

To get started on generating differentially private data, we recomend you follow the Privacy on Beam codelab.

Currently, the DP building block libraries support the following algorithms:

Algorithm C++ Go Java
Laplace mechanism Supported Supported Supported
Gaussian mechanism Supported Supported Supported
Count Supported Supported Supported
Sum Supported Supported Supported
Mean Supported Supported Supported
Variance Supported Supported Planned
Standard deviation Supported Supported Planned
Quantiles Supported Supported Supported
Automatic bounds approximation Supported Planned Planned
Truncated geometric thresholding Supported Supported Supported
Laplace thresholding Supported Supported Supported
Gaussian thresholding Planned Supported Supported

Implementations of the Laplace mechanism and the Gaussian mechanism use secure noise generation. These mechanisms can be used to perform computations that aren't covered by the algorithms implemented in our libraries.

The DP building block libraries are suitable for research, experimental or production use cases, while the other tools are currently experimental, and subject to change.

How to Build

In order to run the differential privacy library, you need to install Bazel in version 4.1.0, if you don't have it already. Follow the instructions for your platform on the Bazel website

You also need to install Git, if you don't have it already. Follow the instructions for your platform on the Git website.

Once you've installed Bazel and Git, open a Terminal and clone the differential privacy directory into a local folder:

git clone https://github.com/google/differential-privacy.git

Navigate into the differential-privacy folder you just created, and build the differential privacy library and dependencies using Bazel (note: ... is a part of the command and not a placeholder):

To build the C++ library, run:

cd cc
bazel build ...

To build the Go library, run:

cd go
bazel build ...

To build the Java library, run:

cd java
bazel build ...

You may need to install additional dependencies when building the PostgreSQL extension, for example on Ubuntu you will need these packages:

sudo apt-get install make libreadline-dev bison flex

Caveats of the DP building block libraries

Differential privacy requires some bound on maximum number of contributions each user can make to a single aggregation. The DP building block libraries don't perform such bounding: their implementation assumes that each user contributes only a fixed number of rows to each partition. That number can be configured by the user. The library neither verifies nor enforces this limit; it is the caller's responsibility to pre-process data to enforce this.

We chose not to implement this step at the DP building block level because it requires some global operation over the data: group by user, and aggregate or subsample the contributions of each user before passing them on to the DP building block aggregators. Given scalability constraints, this pre-processing must be done by a higher-level part of the infrastructure, typically a distributed processing framework: for example, Privacy on Beam relies on Apache Beam for this operation.

For more detail about our approach to building scalable end-to-end differential privacy frameworks, we recommend reading:

  1. Differential privacy computations in data pipelines reference doc, which describes how to build such a system using any data pipeline framework (e.g. Apache Beam).
  2. Our paper about differentially private SQL, which describes such a system. Even though the interface of Privacy on Beam is different, it conceptually uses the same framework as the one described in this paper.

Support

We will continue to publish updates and improvements to the library. We are happy to accept contributions to this project. Please follow our guidelines when sending pull requests. We will respond to issues filed in this project. If we intend to stop publishing improvements and responding to issues we will publish notice here at least 3 months in advance.

License

Apache License 2.0

Support Disclaimer

This is not an officially supported Google product.

Reach out

We are always keen on learning about how you use this library and what use cases it helps you to solve. We have two communication channels:

Please refrain from sending any personal identifiable information. If you wish to delete a message you've previously sent, please contact us.

Related projects

  • PyDP, a Python wrapper of our C++ DP building block library, driven by the OpenMined open-source community.
  • OpenDP, a community effort around tools for statistical analysis of sensitive private data.
  • TensorFlow Privacy, a library to train machine learning models with differential privacy.