-
Notifications
You must be signed in to change notification settings - Fork 444
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
add pointer to paper documenting the epsilon computation when it is available #19
Comments
@ilyamironov would be the best person to answer this question, but my guess is that the RDP paper would be a good starting points: https://arxiv.org/abs/1702.07476. |
Indeed, the algorithm that we use to maintain the RDP accountant is different from the Abadi et al. paper. We are working on a paper that fully documents the procedure. Once it's available, we'll update the README file. (For the record, the results of doing privacy analysis via the moments/RDP accountant must be identical. The main difference is in numerical stability for a wide range of parameters.) |
@srxzr The epsilon is computed using the inversion of Item 2 of Theorem 2 in Abadi et. al., and it computes the smallest such epsilon given a list of lambdas ( The
I have yet to decode Hope this helps. Let me know if you have further questions. |
@ycpei Thank you for your response. Just to make sure Abadi et. al. uses random sampling (instead of shuffling) but the implementations here use shuffling. I was wondering if you use any theorem to compensate for the difference. Also, I implemented a random sampler with replacement and my results were a bit lower compare to the shuffling. |
@srxzr Re shuffling vs sampling, it's a correct observation. Our analysis assumes that the minibatches are iid samples, while in practice they are most likely fixed-sized disjoint (within a single epoch) samples. The best reference that compares various policies of sampling is this one: https://arxiv.org/pdf/1807.01647.pdf |
Paper posted on arXiv: https://arxiv.org/pdf/1908.10530.pdf |
Hi all,
The code seems to implement Abadi et al.'s work. However, the way you compute the overall epsilon does not follow the Abadi's paper. I was wondering if you can give me some references about how you calculated the epsilon.
Thanks,
The text was updated successfully, but these errors were encountered: