declearn v2.5.0

Released: 13/05/2024

These release notes can also be found on our website and on the source GitLab repo.

Release Highlight: Secure Aggregation

Overview

This new version of DecLearn is mostly about enabling the use of Secure Aggregation (also known as SecAgg), i.e. methods that enable aggregating client-emitted information without revealing said information to the server in charge of this aggregation.

DecLearn now implements both a generic API for SecAgg and a couple of practical solutions that are ready-for-use. This makes SecAgg easily-usable as part of the existing federated learning process, and extensible by advanced users or researchers that would like to use or test their own method and/or setup.

New features are mostly implemented under the new declearn.secagg submodule, with further changes to declearn.main.FederatedClient and FederatesServer integrating these features to the main process.

Usage and scope

Setting up SecAgg is meant to be straightforward:

• The server and clients agree on a SecAgg method and, when required, some hyper-parameters in advance.
• Clients need to hold a private Ed25519 identity key and share the associate public key with all other clients in advance, so that the SecAgg setup can include the verification that ephemeral-key-agreement information comes from trusted peers. (This is mandatory in current implementations, but may be removed in custom setup implementations.)
• The server and clients must simply pass an additional secagg keyword argument when instantiating their FederatedServer or FederatedClient, which can take the form of a class, a dict or the path to a TOML file.
• Voilà!

At the moment, SecAgg is used to secure the aggregation of model parameter updates, optimizer auxiliary variables and evaluation metrics, as well as some metadata from training and evaluation rounds. In the future, we plan to cover metadata queries and (yet-to-be-implemented) federated analytics.

Available algorithms

At the moment, DecLearn provides with the following SecAgg algorithms:

• Masking-based SecAgg (declearn.secagg.masking), that uses pseudo-random number generators (PRNG) to generate masks over a finite integer field so that the sum of clients' masks is known to be zero.

  • This is based on Bonawitz et al., 2016.
  • The setup that produces pairwise PRNG seeds is conducted using the X3DH protocol.
  • This solution has very limited computation and communication overhead and should be considered the default SecAgg solution with DecLearn.

• Joye-Libert sum-homomorphic encryption (declearn.secagg.joye-libert), that uses actual encryption, modified summation operator, and aggregate-decryption primitives that operate on a large biprime-defined integer field.

  • This is based on Joye & Libert, 2013.
  • The setup that compute the public key as a sum of arbitrary private keys involves the X3DH protocol as well as Shamir Secret Sharing.
  • This solution has a high computation and commmunication overhead. It is not really suitable for model with many parameters (including few-layers artificial neural networks).

Documentation

In addition to the extensive in-code documentation, a guide on the SecAgg features was added to the user documentation, that may be found
here. If anything is not as clear as you would hope, do let us know by opening a GitLab or GitHub issue, or by dropping an e-mail to the package maintainers!

Other changes

A few minor changes are shipped with this new release in addition to the new SecAgg features:

• InMemoryDataset backend code was refactored and made more robust, and its documentation was improved for readability purposes.
• declearn.typing.DataArray was added as an alias.
• declearn.dataset.utils.split_multi_classif_dataset was enhanced to support a new scheme based on Dirichlet allocation. Unit tests were also added for both this scheme and existing ones.
• Unit tests for both FederatedClient and FederatedServer were added.
• Deprecated keyword arguments of Vector.sum were removed, as due.