RNG and entropy improvements #64
Conversation
- val _default_generator is a g option ref (defaults to None) - provide default_generator : unit -> g, which raises if the default generator is not yet set - provide set_default_generator : g -> unit, which raises if the default generator is already set - remove `Null` generator (moved to use site in test_rsa)
return is a closure taking a buffer
previously, the RNG was created and then entropy was fed into it. now, the startup-entropy is fed directly while it is being constructed.
…) do it: - in the interrupt_hook (called on every Lwt loop enter), collect some bits from __rdtsc - in a separate task, call rdrand periodically (each second) for each pool the choice to use rdrand instead of rdseed is due to the fact that rdseed is roughly 3 times as expensive and the CPU can run out of it. since we're mixing the output into the fortuna pools anyways (and not use the output as only seed of the RNG), this should be safe. the previous behaviour was, especially under high load, a bit problematic, since each time the event loop was triggered, rdseed got invoked which is pretty expensive.
…y if pool0 exceeds minimum size
…ument Fortuna: control the upper bound of the reseeding with entropy pool rate to 1s (as recommended by Schneier et al, don't reseed too often, they recommend 100ms, but since our rdrand task uses 1s as well, no need to do this more frequently)
mirage specific parts are in Mirage_crypto_rng_mirage (mirage-crypto-rng.mirage)
…bootstrap be more honest about entropy sources
|
this PR is now ready. to summarize, entropy collection:
instead of a with this design, virtio-rnd can be easily integrated (though the
|
…) now warn (apart from mirage, which errors)
|
thanks for your review @cfcs. to answer the remaining questions:
If I understand you correctly, you propose the
Thinking again about the topic, I pushed a8c7bbd which let's the entropy collection in Mirage_crypto_rng_lwt always feed the |
|
@hannesm Thanks for your answers, I think both make sense, and agree with the way forward for |
…0.7.0) CHANGES: * CPU feature detection (AESNI, SSE3, PCLMULQ) at runtime instead of compile time (mirage/mirage-crypto#53 @Julow, fixed MirageOS support mirage/mirage-crypto#61, review by @hannesm) performance hit up to 5% * Revise entropy collection (mirage/mirage-crypto#64 @hannesm review by @dinosaure @cfcs) mirage-crypto-entropy has been folded into mirage-crypto-rng.{unix,lwt,mirage} - the RNG is no longer fork() safe, if you use fork in your code, be sure to reseed the RNG in the child process - on Unix and Lwt, the used RNG is Fortuna, seeded by getrandom(), rdrand/rdseed, and whirlwind - Mirage_crypto_rng_lwt does entropy collection for Lwt applications - entropy collection is now similar to FreeBSD: - rdrand/rdseed is executed in a separate task (by default every second) - on Unix, getrandom() is executed in another separate task (by default every 10 seconds) - on every enter of the Lwt event loop, some bits of rdtsc are collected (rdrand/rdseed is not on each even loop enter anymore) - Fortuna only uses entropy pools if the given period is exhausted (defaults to 1s), and the pool size exceeds 64 bytes - The unseeded generator exception prints instructions how to seed the RNG * 32 bit support (for ghash), requested by @TImada in mirage/mirage-crypto#60, mirage/mirage-crypto#65 @hannesm * use Eqaf_cstruct.find_uint8 instead of Cs.ct_find_uint8 (mirage/mirage-crypto#52 @dinosaure) * add (:standard) in C flags to allow cross-compilation mirage/mirage-crypto#47 @samoht * Mirage_crypto.Uncommon: remove several functions (Cs.create, Option), requires OCaml 4.08 (mirage/mirage-crypto#49 mirage/mirage-crypto#51 @hannesm) * remove ocplib-endian dependency, use Bytes directly (since 4.07) mirage/mirage-crypto#51 @hannesm * bitfn.h cleanup (mirage/mirage-crypto#56 mirage/mirage-crypto#58 @hannesm) * fix build if opam is not available (mirage/mirage-crypto#66 @hannesm) * update test.yml GitHub actions (mirage/mirage-crypto#44 mirage/mirage-crypto#57 @imbsky) * Travis CI for arm64 (mirage/mirage-crypto#55 @hannesm)
See individual commits for further details. This PR is supposed to improve the RNG and entropy story for MirageOS (adapting the FreeBSD design)
I'd be happy if anyone is interested in looking into more detail here (keen to answer questions if you have any, maybe @cfcs is interested or others). The code as-is is ready for MirageOS, I'd shuffle it slightly around (into mirage-crypto-rng.mirage) and develop a separate mirage-crypto-rng.lwt for entropy collection with lwt (in the same vein).