Repository navigation
0.5.9
Standards vectors, at scale
- The pinned C2SP Wycheproof post-quantum sets (ML-KEM-512/768/1024 key generation, encapsulation, decapsulation and key validation; ML-DSA-44/65/87 signing with and without seed, and verification) and the NIST ACVP sets (ML-KEM keyGen/encapDecap, ML-DSA keyGen/sigGen/sigVer, SLH-DSA keyGen/sigGen/sigVer for all twelve parameter sets, SHA-3, SHAKE, HMAC-SHA-256/512, PBKDF2) run in CI on every change to the kernels (
.github/workflows/vectors.yml,tools/vectors/): 6,488 tests checked, 0 failures. The 3,822 not applicable are named with their reason: bit-length SHA-3 messages (the API hashes bytes), gigabyte messages, hedged ML-DSA signatures whose randomness the vectors do not carry, and PBKDF2 over SHA-224. A subset of 1.5 MB ships intests/testthat/vectorsand runs offline through the same runner. - Found by the vectors:
C_rmbl_mlkem_decapsaccepted a decapsulation key whose stored H(ek) did not match its own ek; FIPS 203 section 7.3 requires rejecting it, and it is now refused. - Pre-hash: HashML-DSA and HashSLH-DSA take every hash function FIPS 204 and FIPS 205 approve:
sha224,sha384,sha512_224,sha512_256,sha3_224,sha3_256,sha3_384andsha3_512joinsha256,sha512,shake128andshake256. SHA-224, SHA-512/224, SHA-512/256, SHA3-224 and SHA3-384 are new native digests, checked against OpenSSL at every block boundary; each OID is bound into the signature, so two digests of the same length (SHA-512/224, SHA3-224) never verify for each other. - SLH-DSA gains its FIPS 205 internal interface (
slh_sign_internal/slh_verify_internal, not exported) so the ACVP internal vectors check the core the public interface wraps.
Side channels: timing, power, masking - First-order masking, on by default.
kem_decapsulate()(ML-KEM-512/768/1024) and every ML-DSA signature (capsule_sign()with an ML-DSA key,fips_sign_mu()) run with each value that depends on the secret key held as two shares, refreshed with fresh randomness from the operating system for every call. The output is byte-identical to the unmasked computation: the full vector suite compares the two on every ML-KEM decapsulation and ML-DSA signing vector.kem_decapsulate(masked = FALSE)keeps the plain path. Cost: about 5 times the plain decapsulation and 16 times the plain signature (13 ms for ML-DSA-65 on a laptop). - The gadgets (
src/rmbl_masked.h): Trichina's AND, a bitsliced Boolean adder, arithmetic-to-Boolean and Boolean-to-arithmetic conversion modulo 3329 and 8380417, masked Compress_1 and the decompressed ciphertext comparison (Bhasin et al. 2021), masked CBD sampling, a masked Keccak, and for ML-DSA a masked SHAKE for rho'' and y, HighBits revealed one bit at a time (w1 is public: it is hashed into the challenge), and the three rejection tests revealing one bit each. - Simulated power analysis (
inst/tvla, CI on every change): the gadgets and the ML-KEM arithmetic, compiled for a Cortex-M4 and run in an emulator, are assessed with TVLA (fixed-versus-random Welch t, two experiments) under two leakage models, the value (Hamming-weight) model and the transition (Hamming-distance) model. Every masked kernel shows no first-order leakage under either, and every unmasked control leaks (|t| up to 163). CI fails on a masked leak under either model. - The masking gadgets are assembly for Cortex-M (Thumb-2), x86-64 and aarch64: every operation that touches both shares of a value (AND, XOR, the public-constant operations, the copies) loads, computes and stores in a fixed register order, zeroes each temporary before it takes a new operand and each destination before it is written, and stores a zero between the share-0 and share-1 stores; code that handles one share at a time runs each share in its own pass with the general-purpose and SIMD registers zeroed in between. From C, the compiler had moved one share into the register that held the other, which the transition model showed as leakage (|t| up to 135) and the value model could not.
- Timing measured on hardware (
inst/dudect, CI on x86-64 and arm64 Linux and on macOS): the dudect fixed-versus-random test on ML-KEM decapsulation (plain, masked, and each phase on its own), encapsulation, HQC decapsulation, the digest comparison and PBKDF2, with an early-exit comparison as the control that must be flagged. Decapsulation is also measured valid-against-invalid with both classes varying (fresh valid ciphertexts against random ones), so the only systematic difference is the rejection bit: an arm64 runner had flagged the fixed-ciphertext test at under one tick in 65,000 on one pass and not on the next, while every phase measured on its own was clean. - Differential fuzzing against OpenSSL 3.5 (
inst/fuzz/fuzz_ossl.cpp, libFuzzer): ML-KEM, ML-DSA (valid and mutated signatures), SLH-DSA-SHA2-128f, the SHA-2, SHA-3 and SHAKE digests, BLAKE2b-512, HMAC and PBKDF2 must agree with OpenSSL byte for byte. - Found and fixed along the way: compression divided by q with a hardware divide at -O0 (KyberSlash) and now multiplies; three "add q when negative" corrections compiled to Thumb-2 conditional execution on a share's sign and are branch-free; the masked share split drew by rejection and now draws in constant time.
- A hex logo (
man/figures/logo.png, sourcedata-raw/hex_logo.svg), on the README and the package website.
The SIU parser, third pass
Two further reviews of 0.5.8 (the diff review and a dedicated SIU review) showed the
C-stack-overflow class still live from three sites the 0.5.8 rewrite did not reach, and
a correctness cost of the fix it did make. All closed here, each at every site of its
kind, pinned bytest-review5.Rand by fuzz seeds for every shape the reviewers used. - 25,000 form feeds (or vertical tabs, or carriage returns) through the plain-text path killed R.
normalize_text()collapsed space and tab only;flatten_ws()had the full six-character set 180 lines away. The first pass now collapses every whitespace byte\smatches except the newline the second pass bounds. strip_boilerplate()ran two unbounded, newline-crossing regexes over the whole document ([\s\S]*?to the privacy paragraph's end,[^.]*to the glossary sentence's end); the line cap never bounded them and 44 KB of period-free text after "this information may include", standard boilerplate in every real report, overflowed the stack, frombricklayer_parse_siu(),bricklayer_siu_resolve_so()and the network pathbricklayer_fetch_parse_siu(). Both passes arefind()scans now; no regex in either SIU file crosses a line without a bound, and the hygiene test asserts it over the sources.- The 2000-character line split is said, and placed better. A line longer than the cap is split at the last sentence end inside the window when there is one, else at the last space; every split is counted and
bricklayer_parse_siu(),bricklayer_siu_text()andbricklayer_siu_resolve_so()warn how many lines were split, because a field spanning a split can come back incomplete. 0.5.8 split at any space, silently. - Two of the new regex-free passes were quadratic (
strip_elements()rescanned to the end from every unclosed<script, 40 minutes at the 2 MiB cap;strip_tags()rescanned for>from every bare<). Both are linear, and the whole SIU path now polls for a user interrupt through a hook the host installs (rmbl::check_interrupt()here; rmorie's and morie's bindings install their own). bricklayer_siu_iso_date()errors past 4096 bytes instead of returning"";small_int()reads up to nine digits (SO #1234567is 1234567 again, not 1); zero-padded numeric entities (é) decode; named entities are case-sensitive, as in HTML and in 0.5.7.
From the 0.5.8 diff reviewrmbl_pbkdf2_sha256()andrmbl_mann_kendall(), the two LinkingTo kernels that poll for an interrupt, raise it from a frame that holds none of their buffers; 0.5.8 raised from inside the poll, over livestd::vectors.rmbl_interrupt_pending()never raises now;rmbl_interrupt_raise_unbarriered()does, after the kernel returned. The header documentsrmbl_mann_kendall()'s NA-on-interrupt contract.capsule_drift()'s identifier heuristic gates on distinct values (20 or more), not on a hundred rows a side: an 80-row id column is an identifier again, a 5-level factor on ten rows still gets its verdict.- The exact Mann-Kendall memo evicts its oldest table instead of flushing all 32.
mcar_test()'slog_p_valueis documented;download_data(mode =)says what it accepts;bricklayer_json_to_json()nameskeep_vec_namesandquiet_vec_names;MORIE_SERVICES_URLis documented and a mirror without a.jsonsuffix finds its.sig;bricklayer_download(timeout = 1e18)is an error, not an undefined cast; a single-label host name is reported as a local address;bricklayer_llm_ask()'s help says an own endpoint is taken when configured, not probed, andbricklayer_llm_status()'s example switches the local route off.- An endpoint of your own (
MORIE_LLM_BASE_URL) may be a literal private LAN address over plain http (10/8,172.16/12,192.168/16,fc00::/7: an LM Studio or vLLM box on your own network, the documented use), through the same per-call relaxation the loopback host has; the diff review (D4) found it refused by two independent rules with no way through. Link-local addresses and host names over plain http stay refused, and a redirect off such an address meets the full policy.bricklayer_services()documentsMORIE_SERVICES_URL;bricklayer_parse_siu()andbricklayer_siu_text()document the 2 MiB cap and the line split;capsule_drift()documents the identifier gate;bricklayer_download()documents itsheadersandtimeoutranges. - The services cache serves only when it is newer than the document bundled with the package: a cache earns its place by being newer, the same date falls back to the bundled copy, and an older one is deleted. morie's resolver gets the same floor.
- Fuzzing: a
fuzz_urltarget overurl_check()andredirect_policy(), seeds for every shape the reviewers used, and the SIU target runs under an 8 MB stack like R's.
The fourth review (network and service discovery, 2026-10-06) confirmed the 0.5.8
transport closes every bypass it was built for, then found eight things at its edges.
Each is fixed at every site of its kind and pinned bytest-review4.R. - The loopback relaxation is an argument, not an option.
options(rmoriebricklayer.allow_loopback = TRUE)relaxed every fetch in the process; it is gone. The local-model routes passallow_loopbackdown one call, only when the address is the loopback host, and a redirect from a remote origin into the loopback interface is refused. An endpoint of your own is either the loopback host or a public https address. - Request headers do not follow a redirect to another origin unless they are content negotiation. The old rule dropped three named credential headers and let everything else travel;
X-Api-Key,api-key,X-Auth-Tokenand the rest now stay with the first origin. "Another origin" compares scheme, host and effective port, so a redirect to another port on the same host drops them too. - A signed document older than the copy shipped with the package is refused. A validly signed but retired document in the cache (or served live) can no longer pin the hosted endpoint, which is where the user's key goes; the bundled document is the floor, and a stale cache below it is deleted.
- Bytes a server sent are parsed as text, never as a second address. One helper,
.rmbl_json_text(), parses every network body, cached manifest and signature file; the URL and file branches ofbricklayer_json_from_json()are for a user's argument only. A body or a.sigthat is a bare URL is "not JSON", not a fetch. - Keys are redacted by value. A gateway's error text has the key that was sent, any
Bearertoken and anysk-key removed before it reaches an R condition. MORIE_HOSTED_AUTH_URLis validated like every other endpoint, and the sign-in service'sverification_uriis opened in a browser only when it is a public https address.- Resource limits. A transfer under 64 bytes/s for 30 s is ended (it used to hold the connection for the whole timeout); an in-memory reply past 16 MiB is dropped, not handed back as a partial body;
bricklayer_download()hasmax_bytes(default 2 GiB) and the services document and JSON fetches cap at 1 MiB and 16 MiB; the scratch file is private to the call and the body is moved into place on a 2xx only.
Documentation
- Sixteen new vignettes, each runnable offline: post-quantum signatures (ML-DSA, SLH-DSA, XMSS); key encapsulation (ML-KEM, HQC); hashing, keyed digests, key derivation and Merkle trees; attestations, signed bundles and tamper-evident chains; side-channel assurance (what is verified, how, and what is outside the checks); JSON parsing and exact serialisation; the language-model routes and the signed services document; trends and change in short count series; small-area rates (expected counts, empirical Bayes, funnel limits, Moran's I); Hawkes processes; banded tables and concentration; stock and flow (average daily population, length of stay); falsification controls, power curves and the E-value; validation rules, inferred schemas and synthetic stand-ins; missingness, duplicates, outliers and distinct counts; and the SIU director's-report parser. The package website groups them by topic.
- DESCRIPTION now also names the Hawkes-process fitting and the stock-and-flow measures; the standards-vector total reads 6,848 (it was misprinted as 6,488).
Air pollution, from source to burden, and the footprint of the computation
- Atmospheric dispersion:
pg_sigmas(),briggs_plume_rise(),gaussian_plume(),gaussian_puff(),advection_diffusion_2d(),lagrangian_particles()(Pasquill-Gifford coefficients with Briggs's fits, plume rise with its buoyant and stable branches, Gaussian plume with ground reflection and mixing-lid images, an instantaneous puff, an explicit grid solver and a Lagrangian random walk with a counting grid). Formula for formula the same as rmorie'sAirDispersion; the particle walk accepts any normal generator throughnormals=. - Concentration-response and burden:
crf_pm25(),crf_no2(),attributable_fraction(),mortality_displaced(),pollution_burden(),pollution_burden_by_area()(WHO 2021 review pooled estimates: Chen & Hoek 2020 RR 1.08 per 10 ug/m3 PM2.5, Huangfu & Atkinson 2020 RR 1.02 per 10 ug/m3 NO2; Burnett et al. 2014 integrated exposure-response curves for IHD and stroke; Levin's attributable fraction; the BenMAP displaced-mortality function; the GBD burden chain). Every coefficient is cited and asserted in the tests. - Exposure-response estimation:
plr_crossfit()(Chernozhukov et al. 2018 partially linear model, DML2 cross-fitting, OLS nuisances, sandwich standard error, recomputed step by step in the tests) andexposure_response_plr()(percentile bootstrap; any estimator with the same interface can be passed). - Equity:
exposure_concentration_index(), the Wagstaff, Paci & van Doorslaer (1991) concentration index of exposure by income. - The pipeline:
verify_pollution()andpollution_report_text()(demo data, a CSV, a NAPS pull with NO2 ppb converted at 1.88, or scalars; an assumption log; one reference concentration throughout; exit statuses 0/1/2). - The footprint of the computation:
compute_footprint(),carbon_intensity(),carbon_intensity_table(),detect_location(),rapl_available(),footprint_equivalents(). Electricity and CO2-equivalent of an expression, measured from RAPL energy counters where readable (codecarbon's Linux method; wrap-around handled, subdomains not double-counted), otherwise modelled with the Green Algorithms formula (Lannelongue, Grealey & Inouye 2021). Utilisation is measured, per process (codecarbon's process mode) or per whole machine (/proc/stat, codecarbon's machine mode) or fixed by number, with Green Algorithms' linear or codecarbon's load curves. The grid's carbon intensity is bundled for offline use anywhere: every country at the latest year Our World in Data publishes from Ember's yearly data (2024 or 2025), sub-national zones from Electricity Maps' 2024 yearly data via Green Algorithms data v3.1, and the world average, each row with its year and source; locations match by code, alpha-3 code or name, an unknown zone falls back to its country, andlocation = NULLdetects the location offline from the environment, time zone or locale and records how. Every default that replaced a measurement is listed in the result. Cross-checked against codecarbon 3.3.1. - A seventeenth vignette,
vignette("air-pollution-and-health"), walks all of it on offline examples.
Coverage to the last line
- Every line codecov reported unreached at 75a0529 (204 lines, 53 files) now has a test reaching it with a real input; guards that no input could reach were removed from
expected_counts(),sir(),drift_homogeneity(),mahalanobis_outliers(),clean_column_names(),hill_tail_index()and the gzip writer rather than excluded. Two defects found on the way and fixed:capsule_attest()with a rawcontextcould never be verified (signed raw, recorded hex), and the dispersion models read a plain-vectorreceptorscolumn-wise. A two-part hashing self-test now exercises the SHA-256/512 partial-block buffering. The handful of lines that remain unreached by design (OS entropy failures, the barrier's post-interrupt error, prime-order-curve impossibilities, ML-DSA rejection refills) are listed in NEWS.