Repository navigation
Releases: spire-studio/pyfe4ai
Release list
v0.4.0
Highlights. An optional Rust extension runs the lattice, threshold,
ristretto255 and BLS12-381 code natively (an MCFE Ring-LWE round of 10^4
gradient elements: 18 ms per client instead of 198 ms); a cryptographic code
review fixed, among others, a leak in the threshold LWE layers of v0.3.0;
exact Gaussian sampling by default for the lattice MIFE / MCFE schemes, and
faster threshold MCFE defaults; schemes are built
by id with pyfe4ai.create("family/variant", config), and the package and
its class names were reorganised.
Upgrading from v0.3.0 -- see the
upgrade guide:
update imports (Code organization); re-key the threshold LWE layers after
finishing only the v0.3.0 rounds that must finish (with t < s their
partial decryptions still leak); store and transmit the sharing_id of
mcfe/ring_lwe_threshold key shares and partial decryptions; regenerate the
keys of sife/lwe / mife/lwe with lwe_n = "auto" and of the pairing
schemes (BLS12-381); move the parameter cache from ./config/; expect
FESchemeError where combine_decrypt returned None for partial
decryptions of different ciphertexts (None still means a value out of
range); use str or int ids. PyFE4AI remains an experimental research
prototype, not recommended for production use.
Code organization (breaking)
-
Modules moved (no aliases remain at the old paths; algorithms,
parameters and serialized formats are unchanged). The package is now in
layers:pyfe4ai.core(base classes and the parameter cache, the
configurations, the serialized objects and their versions, security
profiles, review status, one-time precomputations, exceptions),
pyfe4ai.schemes(the families, plusschemes.commonand
schemes.threshold),pyfe4ai.primitives(lattices, sampling, hashing,
groups, pairings),pyfe4ai.backend(native or Python implementation)
andpyfe4ai.ml(the ML adapter).pyfe4aiexports its entry points
(seepyfe4ai.create()below); the configuration types are the config classes of
pyfe4ai.core.config(see Formats, versions and configurations), no
longer re-exported bypyfe4ai.schemes. Update imports as follows:Old New pyfe4ai.utils.ml_adapterpyfe4ai.ml(defined inml.wrapper,ml.serialization,ml.gradients,ml.thresholdandml.inference; its scheme table_SCHEME_REGISTRYand_LABEL_SCHEMES: the scheme registrypyfe4ai.core.registry, see Scheme registry)pyfe4ai.utils.quantizationpyfe4ai.ml.quantizationpyfe4ai.utils.exceptions, the exceptions re-exported bypyfe4ai.utilspyfe4ai.core.exceptionspyfe4ai.utils.crypto_constantspyfe4ai.core.constantspyfe4ai.utils.crypto_utils, its hashes (hash_to_qr_group,hash_to_zn2_star,paillier_base,hash_to_zq_vector, ...)pyfe4ai.primitives.hashingpyfe4ai.utils.crypto_utils, its groups (generate_group_primes,group_generator_fe,group_generator_threshold_fe,is_dl_group,is_dcr_modulus,RFC3526_PRIMES, ...)pyfe4ai.primitives.groups.integer_groupspyfe4ai.utils.sampling_utilspyfe4ai.primitives.samplingpyfe4ai.utils.ring_lwe_utils,.lwe_utils,.lwe_matrix,.matrix_utilspyfe4ai.primitives.lattice.ring_lwe_utils,.lwe_utils,.lwe_matrix,.matrix_utils(canonical_ring_ciphertext:pyfe4ai.core.formats.binary, see Formats, versions and configurations)pyfe4ai.utils.modular_utils,.dlog_solverpyfe4ai.primitives.groups.modular_utils,.dlog_solverpyfe4ai.utils.pairing_backend,.pairing_utilspyfe4ai.primitives.pairing.pairing_backend,.pairing_utilspyfe4ai.utils.validationpyfe4ai._validation(internal)pyfe4ai.schemes.ipfepyfe4ai.core.basepyfe4ai.schemes.status,pyfe4ai.schemes.profilespyfe4ai.core.status,pyfe4ai.core.profiles(the profile rules,resolveand its tables:pyfe4ai.core.config; the status entries,SchemeStatusand the status values, and the scheme kinds:pyfe4ai.core.registry, see Scheme registry)pyfe4ai.schemes.abdp15_lwe,.ddh_basepyfe4ai.schemes.common.abdp15_lwe,.ddh_basepyfe4ai.schemes.config_types(TypedDicts)pyfe4ai.core.config(config classes; see Formats, versions and configurations)pyfe4ai.schemes.threshold_utilssplit into pyfe4ai.schemes.threshold.sharing,.binding,.codecand.roles(names unchanged; see the next table)pyfe4ai.logger(init_logging_config),pyfe4ai/config.yamlremoved (see Logging) The threshold schemes share these modules (
schemes.threshold):
sharing(Shamir, replicated and seed-expanded replicated sharing, dealt
and added up for an enrolled set),binding(what key shares,
ciphertexts and partial decryptions are bound to:pp_binding, the
binding fields of each threshold LWE scheme, weights, client MAC tags),
codec(encodings
of key material, threshold ciphertexts and partial decryptions, strict
parsing),store(stores and caches of key-share parts and sums, their
locks),roles(server and combiner logic: checks, the flooding noise,
combination) andkl. The names of v0.3.0'sthreshold_utilsmoved:Old ( pyfe4ai.schemes.threshold_utils)New eval_poly,share_secret,lagrange_coefficient_at_zero,lagrange_at_zero,center_mod,share_points,validate_thresholdpyfe4ai.schemes.threshold.sharingfusion_weight_id,check_fusion_weight,client_server_mac_keys,ciphertext_tag,tag_ciphertext,check_ciphertext_tagspyfe4ai.schemes.threshold.bindingciphertexts_digestpyfe4ai.schemes.threshold.codecvalidate_enrolled,validate_partial_decryptions,partial_groups,combine_consistently,combine_elementwisepyfe4ai.schemes.threshold.rolesThe construction ids of the threshold LWE layers (private constants of the
scheme modules) are inpyfe4ai.core.versions(TMIFE_LWE_*,
TMCFE_LWE_*,TMCFE_RING_LWE_*).Objects pickled by an earlier version that refer to a moved module (for
example a pickledFESchemeWrapper, which holds aQuantizationConfig)
do not load. The serialized keys, shares, ciphertexts and partial
decryptions (JSON and binary) are unchanged. -
Formats, versions and configurations (this reorganisation itself
changes no serialized byte, accepted or refused input, or exception):pyfe4ai.core.formatsdeclares every serialized object -- parameter
records, public parameters, encryption keys, functional keys, key
shares (whole, weight and label parts, replicated pieces), ciphertexts,
partial decryptions, the ML wire objects -- as aRecord: its fields in
wire order, what each holds and what an absent field means. A record is
a typed view of the object's dict itself (Record.from_dict(d).to_dict() is d; reading a field is the dict read, with its exception), and the
schemes build and read their objects through them. The byte codecs of
the Ring-LWE ciphertexts are incore.formats.binary.pyfe4ai.core.versionsholds every version marker and compatibility
decision in one place: parameter-record ids, the threshold construction
ids, the sharing, flooding, label-hash and ciphertext-format markers with
what their absence means, sharing ids, key-share parts, the flooding
noise version and the state versions, with the checks that accept,
refuse or regenerate (the schemes call them where they checked before).pyfe4ai.core.config: a key generator's config is converted at entry
into the config class of its scheme (MCFERingLWEConfig,
ThresholdMCFERingLWEConfig, ...; adictsubclass, so dict configs keep
working) and completed by its security profile there; each class
declares its options with their defaults and accepted values
(config.ring_nreads an option with its default). The page
Configuration options of the documentation lists every scheme's
options, their defaults under each profile and the refused combinations,
rendered from these declarations. TheTypedDicts of
schemes.config_typesare replaced by these classes, which keep
their names where a class remains (DDHConfig(sec_param=128, eta=3)
still builds a config); for static typing they aredictsubclasses,
notTypedDicts.
Old New pyfe4ai.schemes.config_types:FEBaseConfig;DDHConfig,DamgardDDHConfig,LWEConfig,RingLWEConfig,PaillierConfig,PairingIPEConfig,CryptoConfig;MCFELWEConfig,MCFERingLWEConfig,ThresholdMCFELWEConfig,ThresholdMCFERingLWEConfig,QuadraticQuadConfig,QuadraticSGPConfig,MultiInputQuadraticConfigpyfe4ai.core.config:SchemeConfig; the same names, butQuadraticMIQFEConfig(see Class names)the same module: PairingMultiIPEConfig,PartFHIPEConfig,ThresholdDDHConfig,ThresholdLWEConfig,ThresholdPairingMultiIPEConfigpyfe4ai.core.config:MIFEFHMultiIPEPairingConfig,SIFEPartFHIPEPairingConfig,ThresholdConfig(andThresholdMIFEDDHConfig,ThresholdMCFEDDHConfig),ThresholdSharingConfig(andThresholdMIFELWEConfig),ThresholdMCFEFHMultiIPEPairingConfigpyfe4ai.schemes.profiles:resolve,STANDARD_PAIRING_GROUP,_STANDARD_MINIMA,_STANDARD_FIXED,_REFUSEDpyfe4ai.core.config:resolve,STANDARD_PAIRING_GROUP,STANDARD_MINIMA,STANDARD_FIXED(andSTANDARD_PRESETS);_REFUSED: therefusedreason of each scheme inpyfe4ai.core.registrypyfe4ai.utils.ring_lwe_utils.canonical_ring_ciphertextpyfe4ai.core.formats.binary.canonical_ring_ciphertextpyfe4ai.schemes.abdp15_lwe.CONSTRUCTION; the record ids of the other schemes (private constants)pyfe4ai.core.versions:SIFE_LWE_RECORD,MIFE_LWE_RECORD, `SIFE_FULLYSEC_LWE...
v0.3.0
Privacy fixes: every scheme that was insecure in v0.2.0 now implements a
published construction (four threshold variants add an experimental
threshold layer of our own on top of one, and dMCFE-DDH an experimental
carry mask), and every scheme carries a
review status. Keys, ciphertexts and partial decryptions of the affected
schemes are not compatible with v0.2.0. Read the README Scheme Status
table and Known Issues before relying on any privacy property.
Security
- MCFE-Paillier implements ABMPSW20 (Abdalla et al., SCN 2020) Fig. 3:
c = (1 + N)^x H(l)^s mod N^2with a full-domain hash intoZ_{N^2}^*,
s ~ D_{Z,sigma}withsigma > sqrt(kappa + 2n log2(2X)) N^{5/2}, and the
single aggregated keysum y s. Previously a single ciphertext revealed a
client's<x_i, y_i>to the key holder, and keys were brute-forceable. - MCFE-LWE / MCFE-FullySec-LWE implement ABMPSW20 Fig. 4
(c = round_q0(Z . H(l) + floor(q/K) x), aggregated keysum y Z) with
parameters derived from the Appendix C.3 constraints (selective and
adaptive choice ofm0respectively). Previously the key holder learned
x_i mod landx_iup to a few candidates. - MCFE-Damgard implements ABMPSW20 Fig. 2 (
D_k-MDDH,k = mddh_k,
default 1, where it coincides with CDGPP18). Previously two ciphertexts of
one client under two labels revealed its plaintexts. - tMCFE-DDH is CDGPP18 with the aggregated key Shamir-shared over
Z_q;
anytservers interpolate[H(l)]^din the exponent. Previously the
combiner could strip each client's MD5-scalar label mask. - tMCFE-LWE is ABMPSW20 Fig. 4 with a Shamir-shared key and
noise-flooded partial decryptions (each is an LWE sample under the
server's share instead of an exact linear function of it). The noise is
derived from a per-server secret, the label and the server's Lagrange
coefficient for the enrolled set, so repeated requests cannot be
averaged. The threshold layer is our own, so the
scheme's status is experimental. - Paillier parameter caches (SIFE, MIFE, MCFE, dMCFE) no longer store
the primesp, q: the factorization decrypts SIFE ciphertexts directly
and MCFE ciphertexts from two labels. Caches written by earlier versions
are regenerated with a freshN. - Labels: ndarray element labels were plain JSON strings, so a chosen
user label could reproduce an element's label (two ciphertexts under one
label leakx - x'); user and element labels are now hashed in disjoint
domains (pyfe4ai.schemes.mcfe.labels). - Threshold key shares record a digest of their fusion weights and servers
refuse ciphertexts presented under other weights. - SIFE-Paillier samples keys from
D_{Z,sigma},sigma > sqrt(kappa) N^{5/2}(ALS16); keys used to be drawn from a range small enough to
brute-force from the public key. - MIFE Damgard, Paillier, LWE, FullySec-LWE and Ring-LWE follow
ACFGU18 (CRYPTO 2018) Fig. 4: each slot encryptsx_i + u_i mod Lunder
its single-input IPFE with a one-time padu_i, the functional key adds
z = sum <u_i, y_i>, and decryption decodes only the sum (ALS16 Figs. 7, 9
and 8, ABDP15, and BMMS21 Sec. 6 as the per-slot IPFEs). Previously the
key holder learned every client's<x_i, y_i>. MIFE-Paillier keys are
Gaussian (they used to be brute-forceable). - tMIFE-LWE Shamir-shares the padded MIFE-LWE key and floods each
partial decryption withD_{Z, 2^kappa B}(Bbounds the decryption
noise;qenlarged), derandomised per request (experimental). - dMCFE-DDH, dMCFE-LWE, dMCFE-Paillier follow ABMPSW20 App. D.1, Fig. 14
(DMCFE from MCFE): encryption and decryption are those of the reviewed
MCFE, and each client publishes an MCFE2 (DCR, Fig. 3) encryption of its
key contribution under the labelH(y); zero-sum MCFE2 keys make the
shares combine to exactly the MCFE1 functional key. Previously the
combiner could strip each client's label mask, and dMCFE-Paillier keys
were brute-forceable. - MCFE-Ring-LWE implements BMMS21 (PKC 2022) Sec. 6, the
identity-based MCFE-compiler: clients encryptx_i + u_i + H(u'_i, l)and
the key for(y, l)carries the label's pad term, so keys are bound to a
label. dMCFE-Ring-LWE follows BMMS21's identity-based DMCFE (ABKW19 +
GDSum, realised with pairwise PRF zero-sum masks). tMCFE-Ring-LWE
Shamir-shares the identity-based key and floods partial decryptions
(experimental). Previously all three masked inputs with a small public
label scalar. - FH-MCFE implements AGT21b (TCC 2021) Sec. 6.2 as recapped in NPS24
(IACR CiC 2024) Sec. 2: one function-hiding IPFE per client
(pyfe4ai.schemes.mcfe.fh_ipe, ACFGU18 Fig. 12 withn = 1and LV16
padding), ciphertexts carryH1(tag)and every functional key embeds a
fresh zero sharing. dMCFE-FH-Multi-IPE implements NPS24 Fig. 1
(selective FH-DMCFE): the zero sharing lives in the encryption keys and
each client derives its own key from an explicittag-fand its own
weights (H2(tag-f)), deterministically, one key per client andtag-f.
tMCFE-FH-Multi-IPE Shamir-shares the FH-MCFE key exponents (servers
hold their shares as group elements); partial decryptions are combined in
G_T(experimental). Previously all three masked inputs with a public label
scalar and ran over the symmetric groupSS512, where SXDH fails; the
family now defaults toMNT224and rejects symmetric groups. - All pairing-based schemes (SIFE FH-IPE and partial FH-IPE, MIFE
FH-IPE and FH-Multi-IPE, quadratic Quad, SGP and MIQFE, the FH-MCFE
family) default to the asymmetric groupMNT224instead of the symmetric
SS512and reject symmetric groups: their constructions assume SXDH or a
generic asymmetric bilinear group, and DDH is easy in a symmetric one. - Review of the remaining schemes against their papers:
- SIFE-DDH, SIFE-DDH-Dynamic and MIFE-DDH published ciphertext components
as the integer product of two residues modp; trial division (or
factoring) recovered the plaintext from the ciphertext alone. Every
element is now reduced modp, and keys modq(a key with a large
weight used to reveal the master secret / the MIFE pads). - SIFE partial-FH-IPE (Gay20 Fig. 8) handed the master secret key
(b, U, V)to every encryptor, so any client decrypted everything and
recovered every key's weights.get_private_keysnow returnsNoneand
Enc'is run by the key generator (sec_encrypt). The default subspace
dimension equalledeta(a squareMrevealsV); it is now 1 (see
below) andMis resampled until it has full rank. - SIFE FH-IPE (KLMMRW18) had no privacy for
eta = 1(B* = 1) and let
anyone recognise zero vectors; vectors are now padded to(x, 1, 0),
(y, 0, 1). - MIFE FH-IPE ran independent per-slot instances and decoded every slot,
revealing each<x_i, y_i>and allowing slot keys of different
functional keys to be recombined; it now implements ACFGU18 Fig. 12. - MIFE in federated learning: MIFE has no labels, so one functional key
combines any ciphertexts of a client and reveals differences between
its values.encrypt_gradient/aggregate_gradientsrefuse MIFE
schemes (use MCFE). - tMIFE-DDH is rebuilt on MIFE-DDH with the functional key Shamir-shared
modq(as tMCFE-DDH); key shares are bound to their weights. - Quadratic Quad now implements Gay20 Fig. 4 (it encrypted
x (x) ywith
an IPFE before): ciphertexts of4n + 2m + 4G1 andmG2 elements. - Gaussian samples wider than
2^40(FullySec-LWE keys and noise) were
rounded doubles whose low bits were all zero; they now use the
arbitrary-precision sampler. - charm serialises the identity of G1 into a point outside the group;
pyfe4ai.utils.pairing_backend.PairingGroupwrites it explicitly.
- SIFE-DDH, SIFE-DDH-Dynamic and MIFE-DDH published ciphertext components
- Independent verification of the rebuilt schemes found:
- Function-hiding keys (FH-MCFE, FH-DMCFE, tMCFE-FH, MIFE FH-IPE)
carriedweights_id, an unsalted SHA-256 ofy: any key holder could
brute-forceyand link keys for the same function. Keys now carry a
randomkey_id(FH-DMCFE shares onlytag_f); decryption no longer
needs or checksy, and threshold servers do not need it either.
Clients receive their slot key without the outerA'(only key
generation uses it). - SIFE partial-FH-IPE keys reveal
M^T y, which for small
eta - subspace_dimusually determines a bounded integery; the
defaultsubspace_dimis now 1 and the leakage is documented. - tMIFE-DDH applied key shares to unchecked ciphertext elements: an
element outside the order-q subgroup leaked one bit of a server's share
through the Legendre symbol. Servers (and MIFE-DDH decryption) now
require subgroup elements. All threshold servers refuse a key share
issued to another server.
- Function-hiding keys (FH-MCFE, FH-DMCFE, tMCFE-FH, MIFE FH-IPE)
- DOT18 and DSGPP18 reviews:
- Quadratic SGP handed the master secret key
(s, t)to every encryptor;
as in DSGPP18 Fig. 2 the public key is now(g1^s, g2^t), encryption
uses it only andget_private_keysreturnsNone. Floats were
truncated silently; inputs are validated. - MIFE FH-Multi-IPE (DOT18 Sec. 3.1) produced deterministic keys for
n = 1withk = 1; it now requiresn >= 2,k >= 1and
len(lst_nid) == n, checks that every slot is present, and its
decryption helpers no longer ask fory. - Quadratic MI-SGP had no published basis (per-client copies of SGP): the
key holder learned every client'sx_i^T F_i y_iand could mix sub-keys.
It is replaced byquadratic/miqfe, the MIFE for quadratic functions of
Agrawal, Goyal, Tomida (TCC 2022, Sec. 5): per-user encryption keys made
of master ciphertexts (a corrupted user learns its own key, not the master
secret), function
classsum_{i<k} c[(i,j,k,l)] x_i[j] x_k[l](cross-user terms), b...
- Quadratic SGP handed the master secret key
v0.2.0
Correctness and security fixes. Not compatible with v0.1.0 keys or
ciphertexts for MCFE-DDH and the DDH threshold schemes; ciphertexts of the
label-based LWE / Ring-LWE / FH-Multi-IPE MCFE schemes produced by v0.1.0 do
not decrypt correctly with v0.2.0 (label scalar mapping changed). Several multi-client
schemes still have known privacy issues — read
Known Issues / Security Notice
before using them.
Security
- MCFE-DDH (
pyfe4ai.MCFE) re-implemented following CDGPP18: the label is
hashed to a vector of group elements (SHAKE-256 into the quadratic-residue
subgroup), every client slot has an independent key, and the functional key
is the aggregated = Σ y_i · s_i. Previously the label entered as a public
MD5 scalar and keys carried per-client components, so a functional-key holder
could recover an individual client's plaintexts. Because CDGPP18 ciphertexts
are deterministic per label, the ndarray helpers derive a distinct label per
element. - MIFE-DDH: every input slot now has its own IPFE master key; previously all
clients sharedw/g_aand one client could unmask another's ciphertext. - DDH threshold (tMIFE-DDH, tMCFE-DDH): Shamir share points start at 1 and
sharing is done over the group order (point 0 used to be the secret itself,
so serversid_0alone could decrypt); share and combine require at least
tshares from the enrolled set, reject duplicate or unknown partial
decryptions, and validate1 ≤ t ≤ s. - Discrete-log tables for integer groups are built in memory and no longer
loaded from disk (the pickle-based cache in the working directory could run
arbitrary code). - LWE-family label scalars never map to 0 (about 1 in
label_moduluslabels,
12.5% with the default of 8, used to remove the mask entirely). The structural leakage of LWE label masking remains; see
Known Issues.
Fixed
- MCFE-DDH decryption for
eta ≥ 2(returnedNone). - DDH threshold Lagrange coefficients are computed modulo the group order
instead of with float division and truncation (enrolled subsets with
non-integer coefficients, e.g.{sid_1, sid_3}, failed);setupno longer raisesIndexErrorand combining no longer
hard-codes divisors foreta ≥ 2. get_decryption_keysof the five decentralized MCFE schemes raises
NotImplementedErrorinstead of returning the exception class; the ML
adapter reports unsupported threshold/decentralized variants clearly.- Pairing and quadratic test modules skip cleanly when
charm-cryptois not
installed.
Changed (breaking)
- MCFE-DDH formats:
sk = {"s"},dk = {"d": [d0, d1]},ct = {"c"}. - MCFE-DDH ndarray helpers use per-element derived labels
(json.dumps([label, layer, index])). - MCFE-DDH
get_decryption_keysraisesFEValidationErroron an unknown
client id instead of silently skipping it;decryptvalidates the length of
d, the ciphertext range, and that ciphertext and weight lengths match. - Threshold
L()takes amodulusargument; partial decryptions carrysid
andlst_sid_enrolled, whichcombine_decryptvalidates. - LWE label scalar mapping changed from
h mod Mto(h mod (M − 1)) + 1
(centered, never 0), so v0.1.0 ciphertexts of the label-based LWE /
Ring-LWE / FH-Multi-IPE MCFE schemes no longer decrypt; keys are unaffected.
label_modulus < 2raisesFEValidationError. dlog_solver.load_or_build_dlog_tableignores itsfilepathargument:
integer-group tables are no longer read from or written to./config/crypto,
and existing cache files there are ignored.
Added
benchmarks/fl_path_coverage.py: enumerates federated-learning path support
over all 39 registry entries (--output jsonavailable).- Regression tests for
eta ≥ 2,t < ssubsets that excludesid_0,
single-server rejection, duplicate shares, independent client keys, label
binding, and the dlog cache. - README Known Issues / Security Notice.
Documentation
sec_paramis documented as a modulus bit length, not a security level.- README bibliography: TAPFed DOI, venues for Gay20, DOT18 and the Ring-LWE
construction, complete list of pairing-dependent modules. - Landing page aligned with the Known Issues.
Full changelog: v0.1.0...v0.2.0
v0.1.0
Superseded by v0.2.0.
v0.1.0 contains correctness and security bugs in MCFE-DDH, MIFE-DDH and the
DDH threshold schemes (among others) that v0.2.0 fixes. Do not use v0.1.0 for
new work; see the v0.2.0 changelog
and the Known Issues.
Initial public release of PyFE4AI, a research-oriented Python library of
functional encryption (FE) schemes for AI security and privacy.
- 39 scheme variants across single-input (SIFE), multi-input (MIFE),
multi-client (MCFE), decentralized, threshold, function-hiding and quadratic
FE, on DDH, Damgård-DDH, Paillier, LWE, Ring-LWE and pairing backends. utils/ml_adapter.pyfor federated-learning gradient aggregation and
encrypted linear inference; quantization toolkit; NTT for Ring-LWE; cached
baby-step giant-step discrete log.- Core test lane in CI, Sphinx API documentation and project page.
Research prototype — not production-audited cryptography.