Repository navigation
Releases: rootcoder007/rmorie-bricklayer
Release list
0.5.14
- A new data key is issued only after the data access terms are accepted.
rmbl loginandbricklayer_llm_login()show the terms summary and their address, askDo you accept ...? [y/N], and then ask for your name, organisation (optional) and why you need the data; for the restricted tables (Schedule B: ICPSR, NCVS 38604, A2AJ case law) they also ask for a supervisor, department chair, director or coordinator and their email. In a script, give them as arguments (accept_terms = TRUE,name,affiliation,purpose,restricted,supervisor_name,supervisor_role,supervisor_email) or flags (--accept-terms,--name,--affiliation,--reason,--restricted,--supervisor-name,--supervisor-role,--supervisor-email). Nothing is sent until every required answer is there; the data service checks the same things.login --tokenwith a key you already hold is unchanged.
0.5.13
- The survey-estimation vignette loads the CSUS 2023 public-use file and its 1,000 mean bootstrap weights with
bricklayer_data_load("csus_2023/pumf")and("csus_2023/bootstrap"), now hosted on data.rmorie.com, or downloads them from Health Canada withbricklayer_download(); it no longer calls rmorie, which this package never depends on. - A new vignette, Causal effects with cross-fitted double machine learning:
plr_crossfit()recovers a known effect on simulated data and, on the NSW job-training experiment (LaLonde 1986; Dehejia and Wahba 1999, hosted asnsw_dw/*on data.rmorie.com), moves the observational estimate against the PSID controls from -15,205 to 618 dollars, against the experimental 1,794.
Fixes from the CRAN stress test of b617a3f - A key next to punctuation never reaches a model. A question with no verb is refused as one when a key-shaped run appears anywhere in it (
KEY?,"my key is KEY, thanks?", a key after a no-break or zero-width space), not only as a whole word: those went to the first provider, the wrong one for the key. Function names (bricklayer_json_base64url_dec) are not taken for keys. A verb typo with a?apart (rmbl "doctr ?"), a leading ideographic space or quotes is a near miss, named in "did you mean"; a CJK question ending in a full-width question mark is asked. survey_design(): genuine replicates passed by name are no longer refused when two of them are identical (216 of 2,400 test designs); JKn replicates of other strata, which equal the weight on a stratum subset, are accepted (every replicate equal to the weight is refused); a stratified bootstrap whose replicates equal the weight by chance, and post-stratified or calibrated jackknife subsets, are no longer warned about; a replicate a tenth of the weight's scale (a normalised weight, a multiplier column) is said, as one ten times it was. Rows reordered and then a column dropped or one cell edited are refused (the SE was silently wrong); a row-name reset is accepted. The Rd says how to read the survey package's own bootstrap replicates.- Interrupts and time limits: every POST polls for Ctrl-C and
setTimeLimit(), not only when the wait label shows (Wayback/save, OCSP, the data-key check,askwith stderr redirected); a Ctrl-C inbricklayer_fetch()orwayback_snapshot_url_native()is raised at once instead of running the Wayback lookups on to their timeouts; the Wayback back-off without a terminal sleeps a second at a time, so a time limit ends it. bricklayer_data_load()applies the optionrmoriebricklayer.max_csv_bytes(1 GiB by default) on both of its read paths.- CSV reader parity with
read.csv(): a line of NUL bytes is skipped; integers with leading zeros past ten characters are integers; a header of""alone is a table of no columns; blanks outside a quoted header name are dropped; quotes after a closing quote are read as quotes; a header past a million fields is refused before it is held. - Language models: the last turn after the help-page lookups keeps the tools declared with calls turned off and rides on the last tool result (it was a second user turn with no tools, which Anthropic refuses); an SSE error event, or a 200 that carries an error, is said with its message; the saved-answer key includes a digest of the API key in use.
- C and C++: argument errors and large allocations inside the entry points unwind through the barrier, so their buffers are freed (
sen_slopes()leaked 858 MB at a failed allocation; small leaks in the HTTP header, gini, merkle, XMSS and BLAKE2b paths); a NaN or hugetimeoutis no longer cast to an integer (undefined behaviour). - Statistics:
hurwitz_zeta(s, 1/2)is(2^s - 1) zeta(s), exact beside the trivial zeros; a point at -1e300 no longer collapses the Knox and hot-spot grids. - Arguments: a seed that is not one whole number is refused in words wherever a function seeds the RNG, and seeds now run from -2147483647 to 2^53 - 1: past
set.seed()'s 32 bits the Mersenne-Twister state is built from the seed (each seed its own stream; seeds in the integer range give the stream they always did);lagrangian_particles(normals =)must be a function returningnfinite numbers; a plot method given something its class's function did not make says so. - SIU parsing of a long run of letters is linear (a 1 MB
<p>AAAA...took four minutes; now 2.4 s). - The courts vignette's case-mix paragraph is computed from its table; a test no longer writes
Rplots.pdfinto the tests directory; a complex-NA test follows R's own coercion (it failed on R 4.3); the heaviest signature-vector blocks (about 100 s) are skipped on CRAN and run in CI.
Live progress for long work bricklayer_progress(label, total), exported: the animated bar ofbricklayer_download()for any loop, as an object with$update(done, info),$tick(by, info),$wait(seconds)and$done(summary). It follows the download bar's rule (drawn while a person watches a terminal: an interactive session, or a command line that setsoptions(morie.progress = TRUE); a line at each tenth when stderr is not a terminal; nothing withoptions(morie.quiet = TRUE)orquiet = TRUE). Nothing is drawn for the first 0.75 s, so a fast call never flashes a bar; a bar inside another function's running bar stays silent, so nested work shows the outer loop; an update is a stored value when quiet and a throttled frame when drawing. Withtotal = 0it is a spinner with the time so far and the current stage.on.exit(p$done(), add = TRUE)clears the line when the work fails or is interrupted.- Wired into the long-running functions: the bootstrap of
exposure_response_plr(), the folds ofplr_crossfit(), the permutations ofknox_test()andstep_change(), the controls ofcapsule_falsify(), the repetitions ofcapsule_power(), the splits ofdrift_calibrate(), the columns ofcapsule_drift(),profile_columns(),correlation_table()andtop_correlations(), the stages ofmahalanobis_outliers(), the EM iterations ofmcar_test()and ofcore_hawkes_fit(method = "em"), the replicate checks ofsurvey_design(), the steps oflagrangian_particles()andadvection_diffusion_2d(), the receptors ofgaussian_plume(),gaussian_puff()andplume_field(), the files ofcapsule_bundle()andcapsule_bundle_verify(), the columns and the writing ofmake_synthetic_csv(), the packages ofcapture_dependencies(), the groups and exact p-values ofanalyse_table()andyoy_pvalues(), and long vectors inbenford_test()anddistinct_sketch()(read in blocks of a million; the counts and registers are those of one pass). - A spinner for long native calls: XMSS key generation and signing (
pqc_keygen(), a few seconds at the default height, about a minute at 14),derive_key(), the native routes ofcore_hawkes_fit()(with the stage: first start, second start) and every network request (bricklayer_fetch(), the portal resolvers,wayback_snapshot_url_native(), revocation and service lookups, model lists) show a turning spinner with the label and the seconds so far, drawn by the compiled layer each time the call polls for a Ctrl-C, from 0.75 s on, on a terminal only. The XMSS tree now polls every 32 leaves (it was every 1,024, once in a whole default key). - Live progress in the IDE consoles, and one look for it. Whether a console can redraw a line is now decided as the cli package decides it, no longer by
isatty(stderr())alone, which isFALSEin the RStudio and Positron consoles, so most R users saw only the line at each tenth. Bars, spinners and the native spinner now animate in a terminal, the RStudio console (and its build and render panes), the Positron console, R.app and RKWard. They draw on stdout in the RStudio and Positron consoles, which print stderr in their error colour, and on stderr elsewhere. Colours follow the console (24-bit, 256, 8, or none;NO_COLOR,R_CLI_NUM_COLORS,RSTUDIO_CONSOLE_COLOR), and so does the width. They never animate inside knitr, an R Markdown notebook chunk or a Jupyter kernel, which get the plain lines.options(rmoriebricklayer.dynamic, rmoriebricklayer.num_colors, rmoriebricklayer.unicode, rmoriebricklayer.width), cli's options andR_CLI_DYNAMICoverride the detection. The bar shows the percent, the count (or the bytes), the rate, the time so far and, once the rate has settled, the time left; it fits the console's width and cuts the label or stage with an ellipsis. When the work ends, a green check, the label, the result and the time replace the bar; a red cross replaces it when the function that made the bar failed or was interrupted.bricklayer_download()shows the same cross. The native spinner of long compiled calls draws the same line, turns only on R's main thread, ends with a check or a cross, and in a log writes a line at 0.75 s, one every ten seconds after it and one at the end.
Integrity scheme 2
New Merkle trees and chain entries are hashed under scheme 2 and record it; anything sealed by
0.5.12 or earlier (no scheme recorded) still verifies under its old rules, with a warning that it
was sealed with the weaker scheme. - Merkle trees separate leaves from interior nodes (RFC 6962): a leaf is SHA-256 of the byte 0x00 and the chunk, a node SHA-256 of 0x01 and its two children. Under scheme 1 the 64 bytes of two sibling digests verified as a "leaf" with the shorter proof from their parent.
merkle_root(),merkle_leaves()andmerkle_proof()takescheme(2 by default; 1 recomputes a root pinned by an earlier version),merkle_proof()records it in the proof, andmerkle_verify()reads it from there or takesscheme = 2to refuse a proof without one. Roots therefore differ from those of 0.5.12 for the same chunks.capsule_report()recordsmerkle_schemebesidemerkle_root. - Chain entries cover their label.
chain_append()records `scheme ...
0.5.12
rmoriebricklayer 0.5.12
The 0.5.11 stress round (commit a5a4d10) and the review of this release's own branch (head 6e80794, 2026-10-12): the findings of both fixed, each fix applied to its siblings, and the claims the earlier fixes overstated corrected.
Language models: bring your own key; the hosted tier is retired
-
Any major provider, or your own model.
bricklayer_llm_ask()takes, in order, an endpoint of your own, then the first provider whose key is set, then Ollama. Sixteen providers are built in (bricklayer_llm_providers()): OpenAI, Anthropic, Google Gemini, OpenRouter, Groq, Together AI, Fireworks AI, DeepSeek, Mistral, xAI, Cerebras, Perplexity, Hugging Face, OpenCode Zen, Cohere and NVIDIA, each from its usual key variable or a key saved privately withrmbl config set <provider>.key. Anthropic's Messages API and Gemini'sgenerateContentare spoken natively, tool calls included; the others are OpenAI-compatible.route = "anthropic"(or any provider id) takes one route only;bricklayer_llm_models(route)andrmbl models --routelist a route's models;rmbl config setupasks for a provider, its key and its model. -
The model knows the packages. With
docs = TRUE(the default) the model is given the name, arguments and title of every function of the installed MORIE packages (rmoriebricklayer, and rmoriedata and rmorie when present), read from their installed help, and five read-only tools: a search over every help page, a help page, its examples, the dataset list and a vignette. A model that does not take tools gets the matching pages in its prompt. How much goes into the prompt follows the model's context: everything for a provider's model or an Ollama cloud model, the functions nearest the question for a model Ollama runs on this machine;MORIE_LLM_CONTEXToverrides.agent_bundle()asks the same way (itsbackendargument is nowroute). -
bricklayer_llm_route_info()says which route would answer, without sending anything;bricklayer_llm_chat()sends a whole conversation as it is (no help, no tools), for chat loops in the other MORIE packages. rmorie's help, thousands of pages, is searched by the tools, and only its functions nearest the question are listed for the model. -
The hosted MORIE tier is retired. No route, setting or model list for it remains;
route = "hosted"says so, and a savedhostedroute falls back to the automatic order with a one-time message. Signing in (rmbl login) is for the curated tables at data.rmorie.com only, andrmbl doctorreports the data key. -
A model the route does not have is named as such, with the number it has; a refused key is said as such, and the key never appears in an error.
-
The data key is checked and issued by the data service.
login --tokenasks data.rmorie.com (/v1/whoami) whether it knows the key before storing it; the sign-in address is the services document'sdata.auth_url, then itsllm.auth_urlfor documents signed earlier. A document may now leave out thellmblock. A restricted table (the ICPSR studies, NCVS 38604, the a2aj case law) refused with 403 says to accept the terms at https://data.rmorie.com/terms. -
API calls no longer end a reply that is slow to start: a model that thinks before answering is not cut off by the 30-second low-speed limit.
-
It shows that it is working. While a model thinks, the terminal shows a turning spinner, " is thinking" and the seconds so far, then how many help pages it has read; the line is wiped when the answer comes. Only for a person at a terminal (never in a script, a log or
options(morie.quiet = TRUE)); Ctrl-C now ends a long wait for a model. -
A repeated question is answered at once. The answer to the same question (spacing aside; case counts), on the same route, model, packages and versions, is kept on this machine for 30 days and comes back without asking the model:
rmbl asksays so,--fresh(R:fresh = TRUE) asks again. Owner-only files undertools::R_user_dir(), at most 200, outdated ones removed on each save, the first save in a session says where; only for a person at a terminal (the command line with the terminal on stdin and stdout) or an interactive session, never from a script, a pipeline or a check;MORIE_LLM_CACHE=offturns it off. Nothing is hosted. -
Servers can reuse the instructions. The instructions a model gets are the same for every question (the functions picked for a question travel with the question), so the prompt caches of Ollama, OpenAI, DeepSeek and Gemini apply, and long instructions are marked for Anthropic's cache: a faster, cheaper first answer on every route.
-
Better retrieval. The help search matches inflected words ("emissions" finds "emission", "testing" finds "test") and leaves out question words; the closest help pages go with every question, to large models too; and the model is told never to say the packages lack something before it has searched under other words.
-
bricklayer_explain_error()andrmbl explain-error "MESSAGE": where an error or warning of rmoriebricklayer, rmoriedata or rmorie comes from, without a model or the network. Every message is read from the packages' own code (stop(), warning(), usage errors; sprintf() and paste() parts as wildcards), matched exactly or by its words, and traced to the exported functions that reach it, whose help pages say what they expect. With no argument, the last error of the session. rmorie's 18,000 messages are read once per installed version and kept beside the saved answers. -
Ollama has to be running.
rmbl doctorand the help now say to start it (ollama serve, the Ollama app, or its system service) when nothing answers at its address.
Network and statistics stress tests of 5e1dce5 (2026-10-14)
- A typo, a file name or a key is never sent to a model. A line with no verb is asked only when it plainly is a question: the first argument is the question itself (two words or more, or ending in
?), it is not a verb or one or two letters from one, it holds no path, line break or key-shaped word, and anything after it is an optionasktakes (rmbl "how?" --freshworks).rmbl explain-eror "object 'x' not found",rmbl "My Documents/urls.txt"andrmbl conifg set anthropic.key "$(cat key.txt)"with a CRLF key file went to the model before, the key with them. An unknown verb now names the nearest one (did you mean ...); one that looks like a key is not printed back. - A Ctrl-C during a GET is raised (
rmbl modelsexited 0 with "did not answer", a Wayback run polled on past its time limit): the transfer's callback records the interrupt it takes. The plain GET behind the Wayback lookups takes Ctrl-C too. - The key prompt finds
sttyandshon PATH without a child process (only/binand/usr/binwere looked in, so on a NixOS-style layout the key was echoed), runs theshit found, and takes a Ctrl-C already pending before the read starts.config setupprints its key explanation as part of the prompt the shell shows once echo is off. - A settings or cache folder left open by an earlier version is made owner-only, not only a new one.
config setwrites only the settings it changes intollm.json(a full rewrite cut floats to 4 digits, unboxed arrays, renamed a""key and rounded integers past 2^53). Answers are kept only with a person at a terminal or in a GUI session (R --interactive < filekept them). The note after the tool-call cap goes with the last tool result (Anthropic and Gemini got two user turns in a row). - The native CSV reader: a table that would decompress past 512 MiB is read the streamed way (
read.csv(gzfile()); a 2.6 MB file of blank lines got the process killed for memory), and a.csv.gzthat is plain text is read as such. Asread.csv()reads them: unquoted header names are trimmed (" Operational_Capacity "came outX.Operational_Capacity.), CR and CRLF inside quotes are newlines, a line holding only""is blank, blanks before a quote and text after one are kept. A quote inside an unquoted field (ab"c, whereread.csv()swallows the rest of the file) is refused in words. Deliberate differences fromread.csv(): a UTF-8 byte-order mark is not part of the first name, and extra fields and an unclosed quote are refused. Short rows are filled, asread.csv()fills them, unless filling makes more cells than the text has bytes plus ten million (a 2,000-column header over short lines made 4e8 cells from a few megabytes). A Ctrl-C while a table is inflated or split is looked for every million fields or blank lines (by rows it waited 53 s on a wide table), the inflating goes in 16 MiB steps on its second pass too, and either comes back as R's interrupt, not an ordinary error. - Survey replicates: what is refused, what is said and what cannot be told apart is in
?survey_design. New: a copy of the weight within 1 percent on every row is refused under JK1, JKn and BRR; warnings for more replicates equal to the weight than a bootstrap draws (judged against the m!/m^m chance for the PSUs read from the replicates), a near copy under the bootstrap or Fay, a JK or BRR replicate that leaves no row out and is no rescaled weight, and replicates totalling on average under a tenth or over ten times the weight ($repweightsmultipliers), and a replicate over ten times the weight on a typical row (api99among the bootstrap replicates raised the SE 21% in silence). 360 generated genuine sets: 0 refused, 0 warned; survey's JK1 and bootstrap subsets of apiclus1: no warning. A second column with the id's name and other values is refused (it was kept); the weight beside the replicates under its own name is recognised within 1e-12. - Reordered survey data is caught by a key per row, computed every time: rows reordered within strata or PSUs are refused now, a permuted outcome (a permutation test) is not a reorder, and a million rows of ten columns t...
0.5.11
rmoriebricklayer 0.5.11
Language models: set everything from the shell or R
- A stored hosted key is used even when a local Ollama server is running with no model pulled. The automatic route used to stop at that empty server ("local Ollama has no model to use"), so a user who had logged in, and whose
doctorlisted every hosted model, could not reach any of them. An empty Ollama is now passed over unlessOLLAMA_MODELis set or the Ollama route is asked for. bricklayer_llm_config()andrmbl configshow and save every language-model setting: the routeasktakes (auto,own,ollama,hosted) and the address, key and model of your own server, of Ollama and of the hosted tier.rmbl config setupwalks through them. Settings are saved in~/.config/morie/llm.json(mode 0600); an environment variable that is set still wins.rmbl ask --route ROUTEpicks the route for one call.rmbl doctorends with the route and modelaskwill use, and how to change it.rmbl helpis a guide: the verbs by area, andhelp start,help llm,help configandhelp rpages with examples. Error messages name theconfigcommand that fixes them.
The table list reads left to right
rmbl data listand printingbricklayer_data_tables()in R show one line per table: the key, the row count and the name, left-aligned and cut to the terminal or console width. They went throughprint.data.frame, which right-aligned the names and, once the descriptions made a line wider than the terminal, printed every key, then every name, then every row count and source as separate blocks.bricklayer_data_tables()still returns the full data frame (withsource); only its printing changed.- A table whose manifest entry has a blank description and source (the otis tables) is named by its key instead of by an empty string.
- The smoke suite (
inst/smoke/smoke.R) prints a failing case's message in full. It cut them at 400 characters, which shortened "https://rmorie.com/access" in the no-route message ofaskto "https://rmorie.co".
0.5.10
A version of its own. The build tagged 0.5.9 on 2026-10-07 (29ff122) and the one that followed it the same day (dcb3cb9) shared a version number while differing in the masking defaults, the FIPS 203 section 7.3 key check, capsule_attest(context = raw) and the whole air-pollution feature. The stress test of dcb3cb9 (2026-10-08) asked for a number that tells them apart, and for the fixes below; this is both.
Every result draws itself
plot()methods for every result the package returns, on base graphics only: stock and flow (stock_flow(), one panel per measure), period-over-period change with intervals (yoy()), rates, shares and rate change, band sensitivity, drift by column (capsule_drift(),drift_calibrate()), Benford digits, power curves, frequencies, missingness patterns, region coverage, Mahalanobis outliers, correlations, permutation nulls, rounding envelopes, concentration-response curves, burdens, concentration curves, the wholeverify_pollution()report on one page, the footprint with its equivalents, concentration fields and particle clouds, count trends, andanalyse_table()one panel at a time. The same customisation everywhere (main,colorpalette,grid,...to the base call),bricklayer_plot_options()for the session,bricklayer_palette()with four colour-blind-safe palettes, and every method returns the data it drew.plume_field()evaluates a plume on a grid;plot_lorenz()andplot_funnel()serve the results that are plain data frames.vignette("plotting-results"). Titles fit their panel: a title too wide is drawn a step smaller, in a panel set one that still does not fit becomes a letter with the key printed under the panels (returned as thepanel_keyattribute), long axis titles shrink or wrap, and the left margin is sized from the widest tick label so an axis title never overlaps the numbers.- To carry what the plots need,
exposure_concentration_index()records its concentration curve,count_trend()its data,verify_pollution()returns classrmbl_pollution_report,advection_diffusion_2d()classrmbl_fieldandlagrangian_particles()classrmbl_particles(all still plain lists underneath).
Air pollution and dispersion, after the stress test
verify_pollution()applied the PM2.5 counterfactual (5.8) to NO2, overstating NO2 deaths by 27%;referencenow defaults per pollutant (NO2 10, PM2.5 5.8), aspollution_burden()always did.- Levin's formula was diluted twice when prevalence was below 1: the relative risk was taken at the population mean, which already includes the unexposed, and then multiplied by the prevalence again. With a CSV or the demo data the exposed are the rows above the counterfactual floor:
exposure_meanis their mean andexposure_prevalencetheir share, and the scalar arguments are documented the same way. advection_diffusion_2d()documented a stability criterion that was false (cfl below 1 and diffusion number below 1); the explicit upwind-plus-FTCS scheme needs|u dt/dx| + |v dt/dy| + 2 (kx dt/dx^2 + ky dt/dy^2) <= 1, and a run outside it reached 10^10 from a unit pulse while reporting itself stable. The bound is returned asstability_numberandstable; a call outside it is taken in enough sub-steps of a smallerdtto satisfy it (reported, with a warning), so the field is a solution over the same physical time rather than an explosion.- Physical guards throughout the dispersion code, on the package's own input helpers: emission rates, masses, heights and distances non-negative, wind speed, travel time and grid steps positive,
n_imagesa whole number (2.5 was truncated silently), the stack hotter than the air for Briggs' rise (a cooler stack returned NaN),x0andy0single numbers (a vector doubledmean_x), and a warning when the release is above the mixing height.gaussian_plume(q = -100)returned a negative concentration andu = 0returned Inf;gaussian_puff(t = -100)a negative one;pg_sigmas(-500)a negative sigma. attributable_fraction()returned 2 for RR = -1, 0 for RR = 0 and NaN for RR = Inf, and coerced text; it now requires a single finite positive RR and a prevalence in [0, 1].pollution_burden(),mortality_displaced()and the CRFs check their inputs the same way; population is kept as a number (8e9 is no longer NA);"PM2.5","PM25"and"pm2.5"are accepted everywhere.- A NAPS pull is converted row by row: a file mixing ppb and micrograms per cubic metre no longer has every row multiplied by 1.88, a
valuecolumn without aunitcolumn is an error rather than assumed, and any other unit is an error. - The concentration index uses mid-ranks, so tied incomes no longer make the index depend on the row order (all incomes equal gives exactly 0).
- Citations: 5.8 is the lower bound of the GBD 2019 theoretical minimum risk exposure level, not "the WHO 2021 guideline" (that is 5); the NO2 respiratory coefficient is Huangfu and Atkinson's RR 1.03 per 10 micrograms per cubic metre; childhood asthma is an incidence coefficient (Khreis et al. 2017, OR 1.05 per 4 micrograms per cubic metre, scaled to per 10) and its burden is reported as incident cases, not deaths. Units are stated on every page:
pollution_burden()takes a rate per person-year,verify_pollution()per 100,000. - Exposure at or below the counterfactual floor is a result (PAF and cases 0), not an assumption failure; an infinite or missing input is one (the status contract holds).
- The carbon-intensity table has one Kosovo row (
XK), documents its provenance and licences, and says which values Ember reports as exactly 1000 or above it. The time-zone table is rebuilt from the IANAzone.tab(one country per zone) plus every alias inbackward, 549 zones:Europe/Stockholm,US/Eastern,Asia/Calcutta,America/Montrealand about 130 other common names were missing and fell through to the locale, so an English (US) locale in Stockholm got the US grid. Every geographic zone inOlsonNames()now resolves, and theposix/andright/prefixes are stripped.compute_footprint()checks every argument before it runsexpr(a negativememory_gbno longer gives negative CO2e after the computation has been paid for).
Cryptography
- Masked ML-KEM decapsulation left both shares of the decrypted message, the hash input and K-bar on the dead stack after returning (the plain path guarded them); every share pair is now wiped.
- Masked ML-DSA signing can be switched off:
capsule_sign(masked = FALSE),fips_sign_mu(masked = FALSE), and the optionrmoriebricklayer.maskedsets the default for all three masked operations in a session (a chain or bundle signs many times). The documented cost was understated: masking takes about 3 to 12 times the plain time on decapsulation and 13 to 24 times on ML-DSA signing, depending on the CPU (4 ms and 26 ms on this package's test machine). - The TVLA workflow also runs on changes to
rmbl_mlkem_body.handrmbl_mldsa_body.h, where the composed masked algorithms live; the dudect runner re-measures every target the first pass puts above the no-evidence line (|t| > 4.5) with three times the samples before judging it, because a shared CI runner moves a target by several units of t between passes (the arm64 runner put the unmasked ML-KEM decapsulation at 6, 9.7 and 10.6 within an hour, for seven ticks in eighty-four thousand; the masked default sits at 1.6). The re-measurement decides: above 10 fails, the inconclusive band is reported with its numbers. On arm64 the run with PSTATE.DIT set is the gate, as the workflow always said; when the runner's CPU has no FEAT_DIT (the GitHub arm64 runners of 2026-10), Arm gives no data-independent-timing guarantee for the instructions, so that run is reported with its numbers rather than gating: there the unmasked ML-KEM decapsulation shows a steady eight ticks in eighty-four thousand between valid and random ciphertexts (|t| 13 at 900k samples) that x86-64, Apple silicon (DIT) and the masked default do not show. The ACVP PBKDF sample exercises SHA-224 only; PBKDF2-SHA256 is checked against OpenSSL and RFC 6070, and the README now says so. C_rmbl_hash_two_part(an internal self-test entry) refuses a non-integer or logicalsplitinstead of coercing it.
Documentation
- The statement that no third-party security audit has been commissioned had been removed from
DESCRIPTION,SECURITY.md, the README and seven help pages without one having taken place; it is back in all of them, and the masking sentence no longer appears on the pages of HQC, XMSS and the DRBG, which are not masked. - Vignette corrections: no ChipWhisperer firmware ships (the power figures are from the emulator); ML-KEM takes a 64-byte seed and HQC v5 a 32-byte one; attestations take a signing key, not a shared secret; a bundle's note is
$attestation$note; the JSON codec is R, not C, and is not a fuzz target; Windows randomness isRtlGenRandom;capsule_bundle()pins whole files, the Merkle tree pins parts of one file; there is no separate cloud-keys route;bricklayer_llm_ask()has nocontextargument; the SIU schema column isnameand an oversize tag is ignored, not an error; the synthetic marker is the.syntheticsidecar; XMSS is not compared with OpenSSL; verification errors on an object that is not a signature; ctgrind is a dynamic check, not a proof; the share recombination sentence in the README names what signing publishes. yoy_pdf()drew rows off the page whenheightwas not 8.5: the row height follows the page height, as the pagination already did.rmbl_barrier.cppis regenerated from its generator. Thewww.rmorie.commove reachesdata_hub.Rand the services fallback, with the trailing slashes the site expects. The offline vector subset is 3 MB, not 1.5.
Pre-existing, found by the stress test
- The French SIU resolver was about 100 times slower than the English...
0.5.9
Standards vectors, at scale
- The pinned C2SP Wycheproof post-quantum sets (ML-KEM-512/768/1024 key generation, encapsulation, decapsulation and key validation; ML-DSA-44/65/87 signing with and without seed, and verification) and the NIST ACVP sets (ML-KEM keyGen/encapDecap, ML-DSA keyGen/sigGen/sigVer, SLH-DSA keyGen/sigGen/sigVer for all twelve parameter sets, SHA-3, SHAKE, HMAC-SHA-256/512, PBKDF2) run in CI on every change to the kernels (
.github/workflows/vectors.yml,tools/vectors/): 6,488 tests checked, 0 failures. The 3,822 not applicable are named with their reason: bit-length SHA-3 messages (the API hashes bytes), gigabyte messages, hedged ML-DSA signatures whose randomness the vectors do not carry, and PBKDF2 over SHA-224. A subset of 1.5 MB ships intests/testthat/vectorsand runs offline through the same runner. - Found by the vectors:
C_rmbl_mlkem_decapsaccepted a decapsulation key whose stored H(ek) did not match its own ek; FIPS 203 section 7.3 requires rejecting it, and it is now refused. - Pre-hash: HashML-DSA and HashSLH-DSA take every hash function FIPS 204 and FIPS 205 approve:
sha224,sha384,sha512_224,sha512_256,sha3_224,sha3_256,sha3_384andsha3_512joinsha256,sha512,shake128andshake256. SHA-224, SHA-512/224, SHA-512/256, SHA3-224 and SHA3-384 are new native digests, checked against OpenSSL at every block boundary; each OID is bound into the signature, so two digests of the same length (SHA-512/224, SHA3-224) never verify for each other. - SLH-DSA gains its FIPS 205 internal interface (
slh_sign_internal/slh_verify_internal, not exported) so the ACVP internal vectors check the core the public interface wraps.
Side channels: timing, power, masking - First-order masking, on by default.
kem_decapsulate()(ML-KEM-512/768/1024) and every ML-DSA signature (capsule_sign()with an ML-DSA key,fips_sign_mu()) run with each value that depends on the secret key held as two shares, refreshed with fresh randomness from the operating system for every call. The output is byte-identical to the unmasked computation: the full vector suite compares the two on every ML-KEM decapsulation and ML-DSA signing vector.kem_decapsulate(masked = FALSE)keeps the plain path. Cost: about 5 times the plain decapsulation and 16 times the plain signature (13 ms for ML-DSA-65 on a laptop). - The gadgets (
src/rmbl_masked.h): Trichina's AND, a bitsliced Boolean adder, arithmetic-to-Boolean and Boolean-to-arithmetic conversion modulo 3329 and 8380417, masked Compress_1 and the decompressed ciphertext comparison (Bhasin et al. 2021), masked CBD sampling, a masked Keccak, and for ML-DSA a masked SHAKE for rho'' and y, HighBits revealed one bit at a time (w1 is public: it is hashed into the challenge), and the three rejection tests revealing one bit each. - Simulated power analysis (
inst/tvla, CI on every change): the gadgets and the ML-KEM arithmetic, compiled for a Cortex-M4 and run in an emulator, are assessed with TVLA (fixed-versus-random Welch t, two experiments) under two leakage models, the value (Hamming-weight) model and the transition (Hamming-distance) model. Every masked kernel shows no first-order leakage under either, and every unmasked control leaks (|t| up to 163). CI fails on a masked leak under either model. - The masking gadgets are assembly for Cortex-M (Thumb-2), x86-64 and aarch64: every operation that touches both shares of a value (AND, XOR, the public-constant operations, the copies) loads, computes and stores in a fixed register order, zeroes each temporary before it takes a new operand and each destination before it is written, and stores a zero between the share-0 and share-1 stores; code that handles one share at a time runs each share in its own pass with the general-purpose and SIMD registers zeroed in between. From C, the compiler had moved one share into the register that held the other, which the transition model showed as leakage (|t| up to 135) and the value model could not.
- Timing measured on hardware (
inst/dudect, CI on x86-64 and arm64 Linux and on macOS): the dudect fixed-versus-random test on ML-KEM decapsulation (plain, masked, and each phase on its own), encapsulation, HQC decapsulation, the digest comparison and PBKDF2, with an early-exit comparison as the control that must be flagged. Decapsulation is also measured valid-against-invalid with both classes varying (fresh valid ciphertexts against random ones), so the only systematic difference is the rejection bit: an arm64 runner had flagged the fixed-ciphertext test at under one tick in 65,000 on one pass and not on the next, while every phase measured on its own was clean. - Differential fuzzing against OpenSSL 3.5 (
inst/fuzz/fuzz_ossl.cpp, libFuzzer): ML-KEM, ML-DSA (valid and mutated signatures), SLH-DSA-SHA2-128f, the SHA-2, SHA-3 and SHAKE digests, BLAKE2b-512, HMAC and PBKDF2 must agree with OpenSSL byte for byte. - Found and fixed along the way: compression divided by q with a hardware divide at -O0 (KyberSlash) and now multiplies; three "add q when negative" corrections compiled to Thumb-2 conditional execution on a share's sign and are branch-free; the masked share split drew by rejection and now draws in constant time.
- A hex logo (
man/figures/logo.png, sourcedata-raw/hex_logo.svg), on the README and the package website.
The SIU parser, third pass
Two further reviews of 0.5.8 (the diff review and a dedicated SIU review) showed the
C-stack-overflow class still live from three sites the 0.5.8 rewrite did not reach, and
a correctness cost of the fix it did make. All closed here, each at every site of its
kind, pinned bytest-review5.Rand by fuzz seeds for every shape the reviewers used. - 25,000 form feeds (or vertical tabs, or carriage returns) through the plain-text path killed R.
normalize_text()collapsed space and tab only;flatten_ws()had the full six-character set 180 lines away. The first pass now collapses every whitespace byte\smatches except the newline the second pass bounds. strip_boilerplate()ran two unbounded, newline-crossing regexes over the whole document ([\s\S]*?to the privacy paragraph's end,[^.]*to the glossary sentence's end); the line cap never bounded them and 44 KB of period-free text after "this information may include", standard boilerplate in every real report, overflowed the stack, frombricklayer_parse_siu(),bricklayer_siu_resolve_so()and the network pathbricklayer_fetch_parse_siu(). Both passes arefind()scans now; no regex in either SIU file crosses a line without a bound, and the hygiene test asserts it over the sources.- The 2000-character line split is said, and placed better. A line longer than the cap is split at the last sentence end inside the window when there is one, else at the last space; every split is counted and
bricklayer_parse_siu(),bricklayer_siu_text()andbricklayer_siu_resolve_so()warn how many lines were split, because a field spanning a split can come back incomplete. 0.5.8 split at any space, silently. - Two of the new regex-free passes were quadratic (
strip_elements()rescanned to the end from every unclosed<script, 40 minutes at the 2 MiB cap;strip_tags()rescanned for>from every bare<). Both are linear, and the whole SIU path now polls for a user interrupt through a hook the host installs (rmbl::check_interrupt()here; rmorie's and morie's bindings install their own). bricklayer_siu_iso_date()errors past 4096 bytes instead of returning"";small_int()reads up to nine digits (SO #1234567is 1234567 again, not 1); zero-padded numeric entities (é) decode; named entities are case-sensitive, as in HTML and in 0.5.7.
From the 0.5.8 diff reviewrmbl_pbkdf2_sha256()andrmbl_mann_kendall(), the two LinkingTo kernels that poll for an interrupt, raise it from a frame that holds none of their buffers; 0.5.8 raised from inside the poll, over livestd::vectors.rmbl_interrupt_pending()never raises now;rmbl_interrupt_raise_unbarriered()does, after the kernel returned. The header documentsrmbl_mann_kendall()'s NA-on-interrupt contract.capsule_drift()'s identifier heuristic gates on distinct values (20 or more), not on a hundred rows a side: an 80-row id column is an identifier again, a 5-level factor on ten rows still gets its verdict.- The exact Mann-Kendall memo evicts its oldest table instead of flushing all 32.
mcar_test()'slog_p_valueis documented;download_data(mode =)says what it accepts;bricklayer_json_to_json()nameskeep_vec_namesandquiet_vec_names;MORIE_SERVICES_URLis documented and a mirror without a.jsonsuffix finds its.sig;bricklayer_download(timeout = 1e18)is an error, not an undefined cast; a single-label host name is reported as a local address;bricklayer_llm_ask()'s help says an own endpoint is taken when configured, not probed, andbricklayer_llm_status()'s example switches the local route off.- An endpoint of your own (
MORIE_LLM_BASE_URL) may be a literal private LAN address over plain http (10/8,172.16/12,192.168/16,fc00::/7: an LM Studio or vLLM box on your own network, the documented use), through the same per-call relaxation the loopback host has; the diff review (D4) found it refused by two independent rules with no way through. Link-local addresses and host names over plain http stay refused, and a redirect off such an address meets the full policy.bricklayer_services()documentsMORIE_SERVICES_URL;bricklayer_parse_siu()andbricklayer_siu_text()document the 2 MiB cap and the line split;capsule_drift()documents the identifier gate;bricklayer_download()documents itsheadersandtimeoutranges. - The services cache serves only when it is newer than the document bundled with the package: a cache earns its place by being newer, the sam...
0.5.8
The third review of the hardening work (2026-10-06) found seven things,
every one of them a fix that had landed at the site a reproducer named and
not at its siblings. This release fixes each at every site of its kind and,
where the siblings can be listed, puts the list in the tests
(test-review3.R, test-src-hygiene.R), so a new one cannot appear
unnoticed.
Language-model routes and the signed services document
bricklayer_llm_ask()tries an endpoint of your own, then a local Ollama server, then the hosted MORIE tier.MORIE_LLM_BASE_URL(withMORIE_LLM_API_KEY,MORIE_LLM_MODEL) names any OpenAI-compatible server;OLLAMA_HOST/OLLAMA_BASE_URL(defaulthttp://localhost:11434,offto skip) a local Ollama, whose first listed model is the default unlessOLLAMA_MODELsays otherwise.route = "own" | "ollama" | "hosted"insists on one.bricklayer_llm_status()andrmoriebricklayer doctorreport all three, in order;agent_bundle()follows the same order.- The hosted tier's addresses come from a signed document, not from the package.
bricklayer_services()readshttps://rmorie.com/.well-known/morie-services.jsonand its detached ML-DSA-44 signature, verifies it against the public key pinned in the package, caches it for a day undertools::R_user_dir(), refuses a document older than the one it holds, and falls back to the cached copy, then the copy bundled ininst/services/, then a document with every service switched off. The endpoint, the model list, the sign-in address and whether the tier is open can now change without a package release; a document that does not verify is ignored.MORIE_HOSTED_BASE_URL,MORIE_HOSTED_AUTH_URL,MORIE_HOSTED_MODELandMORIE_DATA_URLstill override. The hosted tier is a last resort; keys are personal and issued on request at https://rmorie.com/access, and every hint in the package says so. The GitHub and emailed-code sign-ins keep working. - Plain http on the loopback host is admitted for the local routes only. The URL policy that guards every other address refuses
http://and loopback twice over; the own-endpoint and Ollama calls setoptions(rmoriebricklayer.allow_loopback = TRUE)for their own duration, which admits exactly the loopback host (localhost,127.0.0.0/8,::1), nothing else private, and a redirect off it meets the ordinary rules (test-llm-routes.R). - SIU core: every string-to-integer conversion on a regex capture goes through one bounded helper (
small_int());std::stoion a tag such asSO #4444444444444444threwstd::out_of_rangeand aborted the process (found by the libFuzzer target; the input is in the fuzz corpus). Ported to the rmorie and morie copies. bricklayer_siu_text()andbricklayer_parse_siu()killed R on 25 KB of whitespace. libstdc++'s regex executor recurses once per character a repeated atom consumes, so\s+over a long run overflowed the C stack, which notryCatch()and no exception barrier can see. Every pass that runs over a whole document (newline normalisation, script and style removal, tags, entities, whitespace) is a plain loop now, byte-identical to the regexes it replaced (the test keeps the old pipeline as an R reference and compares); the text every field extractor sees has its spaces collapsed and no line longer than 2,000 characters, so no remaining regex can recurse further than that (and MSVC's regex, which has a step budget instead of a stack, stays inside it -- the one time it did not, the public entry points now answer "no match" instead of throwing); the input cap of every SIU entry point is 2 MiB (a report page is a few hundred KB). The guard is a test that enumerates every one-string SIU entry point from the namespace and runs ten hostile inputs through each in a subprocess. XMSS key generation polls for Ctrl-C with the throwing check like the other ten sites.- One attestation still verified two documents. 0.5.7 wrote a level whose names were all empty as an array, so
{"k":{"":"X"}}and{"k":["X"]}shared a digest. The names attribute, present or absent, is now the one thing that tells an object from an array at every level, empty names included, and a named atomic vector is an object too ({"a":1}is not[1]). Attestations of documents whose fields are named vectors change digest; capsules verified against a stored digest should be re-attested. - Every network byte goes through the compiled transport.
bricklayer_download()gated with the validator and then transported with base R'surl(): no DNS check, no pin, no redirect re-validation, no size cap on the package's own documented pipeline;bricklayer_json_from_json()fetched a bare URL with no gate at all;download_data()wrappeddownload.file(). All three usesrc/rmbl_fetch.cpp, with the live progress bar driven by a callback from libcurl. The transport itself now dropsAuthorization,CookieandProxy-Authorizationon a cross-host redirect, refuses a redirect from https to plain http even when the http option is set, checks a URL before opening the destination and writes beside it, moving the body into place only on a 2xx (a refused URL or a 404 no longer deletes a pre-existing file). The redirect rule is one pure function with its own.Callentry and tests. A single-label hostname (metadata,instance-data) is refused by shape, without DNS. A test lists every network open inR/and allows exactly the two base-R fallbacks a standalone capsule bundle needs. - The interrupt flag. The barrier cleared
rmbl_kernel_interruptedonly on the normal path; it is cleared on entry and on every exit now, and a kernel called with no barrier active (a sibling package's own.Call) re-raises the pending interrupt instead of eating it and returningNA.rmbl_pbkdf2_sha256()in the C API returns an int (0, or -1 for an argument it refuses); an interrupt inside it is raised, never a zero key. capsule_drift()'s identifier heuristic applies its ratio test from a hundred rows a side: a five-level factor on ten rows gets a verdict again.mahalanobis_outliers()carrieslog_p_value, the fifth chi-square site;log_p_valueis documented at every site.drift_psi()accepts anyepsin (0, 1) again (0.5.7 refused values above 0.01, a new hard error on a previously accepted argument) and the help explains what the floor does to the index.- The Mann-Kendall exact-distribution memo is bounded; hostnames may carry underscores;
Makevars.win's libcurl fallback link line is wired in;rmbl_digest_equal()gets the same value barrier as the ML-KEM select.
0.5.7
A release about measuring what the 0.5.6 README could only describe. Every claim in the new "Security posture" table is a check that runs in CI.
- Constant-time verification, every primitive.
inst/ctcheck/runs ML-KEM, ML-DSA, SLH-DSA, HQC (v5 and round 4), XMSS, the CTR_DRBG, HMAC, PBKDF2, keyed BLAKE2b, SHA-2, SHA-3 and the digest comparison under valgrind memcheck with their secrets marked undefined (ctgrind), with GCC and Clang; a branch or memory address that depends on a secret fails theconstant-timeworkflow. The run found and this release fixes: ML-KEM'sByteEncodebranched on every bit it packed (the secret key included); the hex encoder used a table indexed by the secret nibble (every HMAC, PBKDF2 and key output); ML-DSA's norm check returned at the first offending coefficient; PBKDF2 round-tripped each MAC through hex and branched on the characters; XMSS measured its hex-encoded seeds withstrlen(). The values the specifications publish (rejected samples, the challenge, the hints, R) are declassified at one named line each. - Secrets are wiped. The same harness scans the dead stack after each operation for copies of the seed, signing key or password. Every scheme left at least one; all temporaries now clear on every return path (
rmbl_ct::Guard), and the scan is part of the gate. - Wycheproof. 1,247 Project Wycheproof vectors (ECDSA P-256/SHA-256, P-384/SHA-384, RSASSA-PKCS1-v1_5 2048/SHA-256) run in the test suite. They found that the DER parser accepted BER (long-form lengths below 128, lengths with leading zeros, non-minimal tags, end-of-contents octets), that
ECDSA-Sig-Valuewas accepted with extra elements, padded or negative integers, and that a PKCS#1 block was checked field by field rather than against RFC 8017's exact DigestInfo bytes. All three are fixed; every valid vector verifies, every invalid and every legacy "acceptable" one is refused. - Fuzzing.
inst/fuzz/holds libFuzzer targets (ASan + UBSan) for the DER parser and RSA arithmetic, the decimal-to-double conversion (compared bit for bit with the C library), and the SIU report parsers, with seed corpora; thefuzzworkflow runs them on every change and for longer weekly. SECURITY.md: threat model, what each check catches and does not, reporting. The README's "Security posture" is now that table.rmbl_hmac_sha256_raw()joins the C API (the hex variant is a wrapper).
The second review of 0.5.6 (2026-10-06) asked for each fix's neighbours. Every item below has a test intests/testthat/test-review2.R.- Every
.Callentry point has the exception barrier, not two of 110:src/rmbl_barrier.cpp, generated frominit.cbyinst/scripts/gen_barrier.R, wraps each entry so a C++ exception is an R error raised after every destructor has run. SevenNULLs tohawkes_rescaled, a 2 GBshake, mismatchedtheil_senlengths (a 40 MB out-of-bounds read), a 4 TBsen_slopes, a 2^31 reservoir and an INT_MAX PBKDF2 are all plain errors now; the quadratic loops and the RSA arithmetic check for Ctrl-C; the RSA public exponent is capped at 64 bytes (1 KB took 1.2 s, 1 MB would have taken 20 minutes, from a certificate);shakeoutput and PBKDF2 iterations are capped; every string argument goes through onermbl_str0(); the DER tree is converted underR_UnwindProtect, so an allocation failure cannot skip its destructor either. - SSRF gate rewritten, in the transport.
src/rmbl_fetch.cppparses the authority as a URL parser does (#@and?@no longer hide a host, any userinfo is refused), reads IPv4 literals withinet_aton's grammar (127.1,0177.0.0.1,2130706433,0x7f000001), IPv6 withinet_pton(mapped, NAT64, link-local, unique-local), refuses local names by suffix (localhost,.local,.internal,.localdomain,.home.arpa), resolves every hostname and tests every address, pins the connection to those addresses (CURLOPT_RESOLVE, so DNS rebinding cannot answer differently), follows redirects itself one checked hop at a time (anhttpsfetch cannot be bounced tohttpor to a private host), and caps a chunked download.bricklayer_fetch(),.rmbl_read_json(), the CRL/OCSP fetch, the Wayback resolver and the hosted-model client all go through it..rmbl_check_public_url()calls the same C check. verify_capsule():provenance_fileis contained like every other path (a symlinkeddata_provenance.jsonverified tampered data as intact);data_not_syntheticis a required row that always exists and reads the sidecar, the manifest and the provenance; amanifest.jsonin the capsule is checked by default andmanifest_consistent/script_sha256are required whenever they apply; asha256orfilenamethat is not a single string, an unreadable file or a directory are failed rows, not R errors.manifest_digest()is injective again: the canonical encoder writes an empty key as""(0.5.6 renamed it to its index, so{"":"X"}and{"1":"X"}shared a digest and one attestation verified both). A list with a names attribute is an object and is sorted, empty names included: a manifest built with unnamed elements in a named list gets a different digest from 0.5.5's, which never canonicalised such a level.make_manifest()writesmeta$syntheticonly when it isTRUE, so a real-data manifest keeps its pre-0.5.6 digest, andmake_synthetic_csv()sets the flag in-process.- JSON: string assembly is linear (250,000 escapes took 16 minutes); the reformatter refuses NUL and lone surrogates as the parser does; parser and encoder share one 100-level cap (the encoder used to hit the C stack first);
bricklayer_json_base64_dec()refuses characters outside the alphabet instead of deleting them; whole numbers beyond 2^53 warn on the decimal branch too;bigint_warn = FALSEandduplicate_keys = "keep"are what the package's own API readers use; the gzip inflate cap is 64 MiB. capsule_drift(): a character column of dates is compared as dates (two disjoint years drift); a column with more thanidentifier_levelsdistinct values, or one per five rows, is an identifier -- itsunseen_shareis reported anddriftedisNA(the 0.5.6 PSI over the union of categories fired on every id and date column); PSI for the rest is over the reference's categories with unseen mass pooled. The result gainstype,unseen_shareandnotecolumns.drift_homogeneity()returnsp_value = NAwithmethod = "inapplicable: ..."when the categories outnumber half the observations (every table with those margins has the same statistic), draws its Monte Carlo p-value under a localseedand restores the caller's RNG, and likedrift_chisq()refuses negative, NA and infinite counts. All chi-square sites reportlog_p_valuenext top_value: a double has no number for exp(-2536), and the log says how far below zero the tail is.cramers_v()drops an unused level (it returned NaN) and takes aseedtoo.parse_bands(): trailing units with punctuation ("18 to 24 (years)", "5 to 9 years, inclusive") parse again; cue words are matched as words ("overnight" is not "over") in the leading slot as well as the tail ("under 18 to 24" is NA, not 18-24); a label with two numbers that is not a range is NA ("more than 10 - 20"); thousands separators are removed only from correctly grouped numbers ("12,34,567+" is NA); a label with a decimal point is on a decimal scale; the dash family is normalised byte-wise, so an en dash parses in a C locale.band_values()guards the open-lower floor as it guards the cap, and its default floor ismin(0, upper), so "under 0" has its midpoint inside the band.trend_test()reportsslope_ci_clamped = TRUEexactly when a rank is at an end, warns whenexact = TRUEcannot be honoured (n > 8), and memoises the tied exact null;count_trend()refuses an all-zero series;step_change()andcount_trend()refuse infinite values astrend_test()does;morans_i()refuses negative weights and a constant variable;gini(),top_share(),lorenz()andhill_tail_index()refuse empty input;hurwitz_zeta()returns the Bernoulli values at non-positive integers (zeta(0, 1) = -1/2); the Benford first digit of a subnormal is read from the shortest round-tripping decimal (1e-310 is 1, not 9).- Result changes from 0.5.6 that NEWS did not state:
morans_i()excludes self-neighbours from a weight matrix;make_manifest()no longer changes the digest of a real-data manifest (0.5.6 did); the OTIS full-verification workflow runs on demand only.
The coverage pass that followed (every branch below was untested at 96.2%) found five defects that no test had reached; the CRAN checks and the entry-point sweep found two more. - Revocation checking worked only on paper.
revocation_fetch()ran every certificate through the single-file-path guard, so a parsed path was refused before anything was fetched; the CRL/OCSP fetch refused plainhttp, which is what RFC 5280 distribution points and RFC 6960 responders use (the answers are signed, the transport adds nothing); and a delegated OCSP responder's certificate was never found, because the search for the[0]certificates element landed onnextUpdateinside the SingleResponse first and the signed answer was reported as unverifiable. All three are fixed; the client is now tested end to end against recorded OpenSSL responses (good, revoked, unknown, SHA-384 without an embedded certificate, a delegated responder, an ECDSA responder,tryLater, and every malformed shape), with the two network calls behind one-line seams. The fixtures are documented intests/testthat/x509-fixtures.txt. top_share()summed from the start for every fraction: a million fractions over a million values was 10^12 additions with no interrupt check (the entry-point sweep found it as the one hang). It is one prefix sum now, interruptible.manifest_restore_seed()padded a record naming only the main generator kind withNA...
0.5.6
A security and correctness release from a full external review of 0.5.5 (every item below was reproduced against the installed package before it was fixed, and each has a test that fails on 0.5.5).
verify_capsule()fails when it cannot check. A provenance file that lackedresource$sha256,resource$filenameor the wholeresourceblock -- or was exactly{}-- verified a capsule with ALTERED data asok = TRUE, because an absent field appended no row andall()of the remaining rows was still true. Every check that cannot be made is now a FAILED row with the reason,okrequires the three required rows (provenance_readable,data_present,data_sha256) to exist and pass, a pinned script digest is checked whenever one exists, a non-CSV file with a recorded row count or schema is a failed check, a<data>.syntheticsidecar is a failed check, and the result has a class and a print method that shows what was skipped.- The DER parser no longer aborts R. A 22-byte file made
cert_parse()(and every OCSP, timestamp and X.509 path that parses network bytes) throwstd::bad_allocthroughstd::terminate, killing the session past anytryCatch(): a length field could be read as 2^64-1 and the bound checks wrapped. Lengths are bounded without wrapping, the node count is capped, a child must advance the cursor, and no C++ exception can cross the R boundary (rmbl_asn1.cpp). The same discipline forRf_error()over live C++ objects in the RSA, BLAKE2b, SHAKE and digest entry points. - ML-DSA signing cannot loop forever. A length-correct but corrupt secret key (s1/s2 outside +-eta) is refused in words, and the rejection loop is capped. The seven unguarded FIPS 204 arithmetic probes that read past a 1-byte allocation are removed (nothing called them); the two other unreachable
.Callregistrations are gone too.pack_sig()usesmemmove()for its aliased copy; thepoly_chknorm()comment says what the code does. - Every
.Callentry that reads a string checks it is one (rmbl_str0());random_bytes()prefersgetrandom(2)where glibc has it; XMSS key generation can be interrupted. - Paths and URLs are checked before use.
resource$filename, bundle paths and the pipeline's download target must be plain relative paths inside the capsule (.., absolute paths, drive letters and symlinks out are refused). Every URL the resolvers return and every URLbricklayer_download(),friendly_download()anddownload_data()fetch must be https (plain http only withoptions(rmoriebricklayer.allow_http = TRUE)), neverfile://(an offline test passesallow_file = TRUE), and never a loopback, link-local, private or metadata address. The Wayback availability API is queried over https; the snapshot URL it returns passes the same gate. libcurl transfers are bounded: 5 redirects pinned to http(s), 2 GiB to a file, 64 MiB into memory, and the"available": truecheck reads the literal value. - JSON. A repeated key is an error (
duplicate_keys = "keep"restores jsonlite's behaviour); nesting is capped at 200 levels; a lone surrogate escape is an error rather than the text"NA"; an integer above 2^53 warns once unlessbigint_as_char = TRUE.bricklayer_json_unserialize()rebuilds data types only by default and refuses functions, calls, expressions, namespaces and S4 objects (trusted = TRUEfor your own session's JSON).manifest_canonical()sorts a level with an empty-string key, so a re-serialised manifest keeps its digest.json_gzip_decode()refuses a member whose ISIZE trailer declares more than 512 MiB and checks the trailer after inflating. - Synthetic data say so.
make_synthetic_csv()writes a<path>.syntheticsidecar and returns a classed object;make_manifest()andwrite_manifest_json()setmeta$syntheticfromBRICKLAYER_SYNTHETICthemselves, andrecord()marks every comparisonINFOon a synthetic manifest, whatever the analysis script did. - The reference pipeline stops on a digest mismatch (exit 7) unless
--allow-digest-driftis given, and then records the override in the manifest. Its subprocess command line is fully quoted. capsule_bundle()hashes dotfiles andcapsule_bundle_verify()lists them; only the bundle's own file is exempt from "unlisted".- The hosted-tier endpoints taken from
MORIE_DATA_URL/MORIE_HOSTED_BASE_URLmust be https and public before a bearer key is sent; the credentials file is created0600before the key is written; a DRBG reseeds itself from the operating system in a forked or restored process; the launcherinstall_cli()writes quotes its library path. - Statistics. Chi-square p-values are the upper tail (they underflowed to exactly 0 in
drift_chisq(),drift_homogeneity(),benford_test(),mcar_test()and the Mahalanobis screen); the KS p-value uses the theta-function series for small t (two million-row columns differing in one value were flagged as drifted);drift_psi()no longer collapses to one bin on a constant or mostly-constant reference (a 1000x rescale reported PSI = 0);drift_homogeneity()gives a Monte Carlo p-value below five expected counts, andcapsule_drift()reports a PSI and the unseen-category share for categorical columns, so two disjoint date columns are flagged; a one-category table reports df 0 and NA.morans_i()honours the magnitudes of a weight matrix. The exact Mann-Kendall p-value enumerates the observed values, ties kept.eb_rates()refuses negative, missing and infinite counts.parse_bands()strips thousands separators, reads "ages 18-24", and leaves a bare "-5", inverted bounds, "100-200-300" and "15 to 19 and over" as NA;expand_bands()errors (or warns, withdrop_unparsed = TRUE) instead of dropping units;band_values()refuses a cap below a band's lower bound.count_trend()uses t quantiles in its quasi-Poisson branch;trend_test()reportsslope_ci_clampedwhen the pairwise slopes do not reach the requested confidence and refuses infinite values. Benford digits are read from the decimal rendering (2e-300 was digit 1). - Packaging.
graphics,grDevicesandtoolsare declared;R CMD checkfails CI on a WARNING; aconfigurescript probes pkg-config / curl-config for libcurl on Unix; an ASAN + UBSAN job runs the suite; DESCRIPTION names every scheme the package implements and states that none has been audited; the key and signature help pages carry a Security section; a FIPS 202 known-answer test covers SHA-3 and SHAKE directly; the LinkingTo consumer test calls all 45 published kernels; the stale CHANGELOG.md is gone (NEWS.md is the record).
0.5.5
- XMSS: an index never signs twice.
capsule_sign()signed two messages at index 0 when called twice with the same key object (the key is an R value, so it cannot advance the caller's copy), and the help said it refused. It now remembers, for the session, the highest index each key has signed with and refuses that index or an earlier one (one key object signed twice, or a stale copy), retiring the index before the signature exists (RFC 8391; NIST SP 800-208). A damagednext_index(NA, negative, fractional) is refused in words. Across sessions, save the latestkey_stateafter every signature. - Hawkes processes: events at the same instant no longer excite each other. The intensity sums over strictly earlier events, as the definition of a Hawkes process (a simple point process) has it; the exponential recursion, the sum-of-exponentials recursion and the exponential and Lomax densities counted a tie as an excitation at lag 0.
core_hawkes_fit()warns when times are tied, andcore_hawkes_jitter()spreads times recorded to a resolution (daily dates) uniformly across their interval, as Filimonov & Sornette (2015) recommend, with the splitmix64 uniforms rmorie and morie use for the TPS fits (the same numbers on daily data). The goodness-of-fit KS test uses the exact distribution only without tied residuals (the exact computation with ties ran for minutes and exhausted memory). core_hawkes_fit()is fast for the Lomax kernel: the projected BFGS now restarts its curvature estimate when a parameter reaches or leaves its bound (as L-BFGS-B does). A Lomax fit to exponential-like data drifts to the bound onalpha(its exponential limit), and the matrix learned withalphafree mis-scaled the remaining steps: 1,164 iterations and 196 s on 9,839 events, now 64 iterations and 3 s to the same optimum. The same restart ends the gamma kernel's stall on its sum-of- exponentials and truncated likelihoods (2,000 iterations to a worse optimum on 2,118 tied events; now 2.8 s to the exact route's maximum), so"auto"keeps"soe"for gamma.core_hawkes_fit()also returnsat_bound, the parameters whose estimate lies on the box:convergedalone did not show that day-dated times had driven a shape to its wall. A fit whose event raten / horizonlies outside what the bounded baseline can reach (e^-15to100 e^15per unit time;horizon = 1e308) is refused with a message to express the times in another unit, instead of returning its start.core_hawkes_nll()is the likelihoodcore_hawkes_fit()maximises, on the same parameters: its Lomax was a power law with exponentalpha(a Lomax of shapealpha - 1), so its values and a fit of it disagreed with the fit and with the documented density; it now uses the shared core for every kernel, keeping its documented sentinel outside the feasible region. A non-finitehorizonis refused by every Hawkes function (Infreturned a list that looked like a fit).- SIU reports: counts in every layout. Investigators, forensic investigators, subject and witness officials and civilian witnesses are read from 2020-on English pages ("Witness Officials ( WO )", the singular "Civilian Witness ( CW )"), French pages ("Agents témoins ( AT )", numbered "AT no 1" or "AT n o 1", "Nombre d'enquêteurs de l'UES assignés : 3") and the legacy (2005-2011) layouts ("Witness Officer #2", "Three SIU investigators and two forensic investigators", "l'agent témoin n o 1", "Cinq enquêteurs"); "No civilian witnesses were identified" is 0, and with no subject-official section the resolver's rules decide (a bare "SO" no longer counts as one). Checked against the reviewed corpus on the 29 pages of the round-8 test: witness officials 27 of 29 (16 before), civilian witnesses 24 of 25 (14), investigators 29 of 29 (20), subject officials 20 of 20 (17).
- SIU
police_service: a service named as the one that notified the SIU ("the OPS contacted the SIU to report that police officers with the VPD ...") is passed over for the subject officials' service (drid 2010: Vancouver Police Department, not Ottawa); the OPP has one French name ("la Police provinciale", a legacy header's detachment) and one English name ("OPP Sioux Lookout"). - SIU dates: a notification that dates the incident relative to itself ("during his arrest two hours prior", "the day before", "deux heures plus tôt", "la veille") gives the incident date (drid 648: 2019-09-12, not an earlier break-in); legacy headers ("Incident date:", "Date de l'incident :") are read; the French narrative's "du 9 février 2022" is read and interview and team dates are passed over. Legacy pages are recognised by language. French abbreviated months ("5 janv. 2023"), upper-case accented months ("3 AOÛT 2017") and "March 3 , 2020" are read.
- SIU dates: the incident date is the first dated sentence of the narrative that is not the notification or the SIU's own work (drid 670: "... contacted the SIU to report a serious injury. On July 8, 2019 at about 3:50 p.m., CKLPS were called" is 2019-07-08), "of " counts ("Just before 4:00 a.m. of January 7, 2020"), and a narrative without a date gives way to the director's analysis ("On December 4, 2020, the Complainant rolled his SUV"); drids 670, 820, 1200 and 4600 were empty. "The SIU was notified of the incident by ... on October 27, 2017" dates the notification, and French "1 er septembre 2016" and "L'UES a été avisée ... le" are read.
- SIU fields:
specific_injuriescomes from the investigation, never from the mandate's definition of a serious injury, with whole words ("stab" is not "constables", "arm" not "firearm"), more body parts, and French injuries ("fracture de l'épaule droite"); Frenchcharges_recommendedandrelevant_legislation("Code criminel") are read.À,«,», and the other capital and French entities decode. bricklayer_siu_resolve_so(): the spelled-out plural ("Two subject officials were designated"), "no official was a subject official" and witness officials with no subject official named resolve, as the help describes.bricklayer_siu_resolve_so()reads French reports: the "Agent(s) impliqué(s)" roster ("AI no 1", a lone "AI"), legacy ordinals ("agent impliqué n o 3", "l'agent(e) impliqué(e) n o 1"), "aucun agent impliqué" and witness officials (AT) alone; the privacy, definition and legal paragraphs ("des agents impliqués et témoins", "les agents impliqués sont invités ...") are not read as cues. It returned NA on every French page.bricklayer_parse_siu()falls back to these rules for a French report with no roster, as it does for an English one (drid 3658: 9, as its English twin).- Inputs:
kem_keygen(512.5),kem_keygen(c(512, 768)),pqc_keygen(2.5)and every other whole-number argument refuse a fractional value instead of truncating it to another setting;bricklayer_fetch_siu(1.5)no longer fetches drid 1;bricklayer_parse_siu()refuses a path that does not exist (it parsed the string as an empty report);bricklayer_fetch_parse_siu(lang = "xx")is an argument error. - Argument errors in words where R's internals spoke:
bricklayer_json_base64url_dec(character(0))(now empty, asbricklayer_json_base64_dec()),bricklayer_json_unserialize()of a number,capsule_bundle(character(0)),yoy()of an infinite value, HyperLogLog registers beyond 64 (distinct_count(1e300)), andbricklayer_fetch_parse_siu()with an invalid drid, which is now an argument error before any network instead of a message andNULL. core_hawkes_jitter(horizon = )spreads each event over[t, min(t + resolution, horizon)], so a time dated on the horizon stays inside the window (it was pushed past it, and the fit then refused the times); the fit's refusal names the span and the remedy.- The
?mahalanobis_outliersexample contradicted itself: its "ordinary on each variable" row had the largest weight, and its classical-versus-robust comparison printedFALSE. The row is now short and heavy within both ranges, and the comparison uses a cluster of bad rows. - A count argument refuses
TRUE(it was read as 1:drbg_generate(d, TRUE)gave one byte),Infand values beyond the integer range in words, with no coercion warning;hqc_sizes(Inf)no longer warns before refusing. bricklayer_fetch()gives the HTTP status of a server that answered ("answered HTTP 404"; it said "could not reach");friendly_download(attempt_wayback = FALSE)(or"") says the fallback is switched off instead of claiming the Wayback Machine has no snapshot.data pullrefuses an--outpath it cannot write before downloading the table.- CLI exit codes: a
bundlewith no language-model route exits 1; an empty prompt or request, a malformed--emailand an unknown option tohelpexit 2. bricklayer_json_to_json()is byte-identical to jsonlite'stoJSON()where jsonlite does not pass an option on:raw = "js"is written with jsonlite's defaults (new Uint8Array([35]), an array, even underauto_unbox, where it wrote the number 35),raw = "hex"is unpadded ("f"), aDateepoch never gets a decimal point, the mongo$dateand$binaryobjects andkeep_vec_namesobjects stay on one line underpretty, andkeep_vec_namesdoes not reach the columns of a data frame. 18,000 random objects now encode identically (30 in 3,000 differed).bricklayer_json_serialize()is no longer called lossless: like jsonlite'sserializeJSON()it rounds doubles todigits(8);digits = I(17)round-trips them exactly.?bricklayer_llm_loginsays a token is checked with the gateway before it is stored.- CI checks on Intel macOS (
macos-15-intel), where r-universe's check had failed. hqc_compress_key()stores an HQC secret key as its seed alone: for v5 the specification's compressed formatdk = seed_KEM(32 bytes), for round 4 the 96 to 112 bytes its key generation draws (round 4 defines no compressed format).hqc_decapsulate()takes such a key, re-der...