drmSEM 0.5.0
This release closes the cyclic / feedback-graph milestone (roadmap §0.5) and
ships the dev-line surface accumulated since 0.2.0. The DAG-only restriction is
lifted for declared feedback motifs, with an equilibrium estimand and a pure-R
fixed-point propagator; undeclared cycles remain a hard error. The feature
highlights below are grouped by area. Consistent feedback estimation (IV/2SLS or
a joint likelihood), full sigma-separation, distributional feedback equilibria,
and the joint bivariate fit remain engine-dependent and are carried forward to
the live-drmTMB lane (see docs/memory/CODEX_HANDOFF.md).
Sampler and propagation fixes
drm_sample_family()and effect propagation now match currentdrmTMBparameterization for the common sampler families in live recovery tests. Default fitted dpars such assigmaare carried into prediction engines even when no explicitsigma ~ ...formula is declared, and lognormal nodes now usemu = meanlog,sigma = sdlog, with mean mediation propagatingexp(mu + sigma^2 / 2)(OQ-1, V-57..V-60).
DAG plot: faithful legend + readable edges
plot.drm_sem()now builds its legend from the components actually drawn
(sourced from the same style function as the edges, so the two can never
drift) instead of always listing all seven distributional components — the
hero/landing-page DAG previously showednu/hu/sd(.)/rho12swatches for
paths that did not exist. Covariance rows are added only for the classes truly
present.- Parallel paths between one pair (e.g. a
muand asigmaarrow on the
same edge) are now fanned onto separate arcs instead of overlapping into a
single line, and alayout =matrix (optionally row-named) can be supplied for
a fixed, crossing-free diagram. A node-fill legend (endogenous response vs
exogenous predictor) is drawn, andhugets a distinct linetype so it no longer
relies on colour alone to separate fromzi(colour-blind safety). - The legend construction is now a tested pure helper (
drm_path_legend()), so a
regression that re-introduces phantom legend entries fails CI. - Composite measurement edges:
plot.drm_sem(show = "all")now draws each
[drm_composite()] construct's indicators pointing into the construct as
steel-blue arrows (indicators shown as distinctly-filled nodes), so a formative
measurement model reads apart from the structural paths (OQ-15).
Outcome functionals across the effect API (OQ-11)
- All three effect functions now report the effect on a chosen functional of the
outcome distribution, not just the mean:target = "mean"/"p_gt"/
"p_zero"/"var"/"quantile"(new, with aprobargument).target
already rodedirect_effects()/total_effects(); it now also rides
indirect_effects()(effect = "controlled"), where every leg reports the
contrast on the functional and the mean-/distribution-mediated split still
closes (indirect = mean_mediated + distribution_mediated). This is where
distribution-mediated paths earn their keep — a path intosigma/zi/nucan
move a tail probability or quantile while leavingE[Y]nearly unchanged. - The
"quantile"target reports theprob-quantile of the simulated outcome
(kernel-validated: a path intosigmashifts the upper quantile but not the
median). direct_effects()/total_effects()gainfunctional = c("simulate", "analytic")."analytic"evaluates the functional in closed form from the
predicted parameters (no Monte-Carlo noise) for the gaussian and
poisson families — exactvar/p_gt/p_zero/quantile. Other families
abort with a pointer back to"simulate"(theirsigma↔dispersion scale is the
OQ-1 open item); analytic needs mean mediation (method = "gcomp").- Fix: the functional engine now honours the mediator-propagation mode
("mean"vs"distribution") instead of always simulating the mediator, so the
controlled decomposition is non-degenerate for a non-meantarget.effect = "natural"remains mean-only (the cross-world functional contrast is open,
OQ-8/OQ-11); a feedback SEM stays mean-only (the equilibrium response).
Validation wave 2 harness + newcomer docs
inst/validation/generate.R+ avalidationarticle now provide the wave-2
coverage/calibration harness (effect-CI coverage against a known-effect
linear-Gaussian DGP, and model-selection recovery rate), mirroring the OQ-6
calibration pattern (cached.rds+ a vignette that renders with or without the
cache). Full replicate runs happen in the live lane; see
docs/design/12-coverage-calibration.md.- The README quick-start and the intro vignette now open with a concrete
biological question and show illustrative (clearly-marked, engine-free)
paths()andindirect_effects()output so a newcomer can see the
component-labelled anddistribution_mediatedrows and learn how to read them.
Simulation-based recovery grid (validation wave 1)
- A campaign of numerical-recovery tests on real
drmTMBfits now exercises
the machinery end-to-end, not just for finiteness (V-45..V-73; see the new
docs/design/11-validation-matrix.md). The effect decomposition is recovered
across the family×link grid (gaussian, poisson, nbinom2, binomial,
beta_binomial, beta, Gamma, lognormal) — mean-mediated equals the fitted-coef
product / apredict_parameters()do-contrast, total = direct + indirect
closes, anddistribution_mediatedmatches the Jensen-gap magnitude from
fitted params (the V-7 live-fit follow-up). Each family'sdrm_sample_family()
mean and variance are checked againstdrmTMB::simulate()(closing OQ-1
gaps), outcome functionals (p_zero/var/p_gt) are recovered, and the
standardizationsigma_Epipeline, composite-as-response, feedback equilibrium
vs the fitted reduced form, and natural NDE/NIE are validated on live fits. A
nonlinear feedback fixed point is added at the kernel tier (V-73). Tweedie /
zero_one_beta inflation / studentnusamplers stay flagged for the live lane.
Wave 2 (effect-CI coverage, d-sep Type-I/power, model-selection recovery rate)
is the calibration layer, tracked inCODEX_HANDOFF.md.
Standardization: GLM mean-path sigma_E (OQ-4)
- The
latentstandardization of amupath on a constant-variance link now
divides bysqrt(Var(eta) + sigma_E^2), adding the link's theoretical
latent-scale error variance — logitpi^2/3, probit1, cloglogpi^2/6
(Grace et al. 2018; piecewiseSEM'slatent.linear). This corrects the earlier
mild over-standardization of GLM mean paths. Identity-linkmuand non-mu
components (sigma/zi/sd(*)) are unchanged; the log-link families'
mean-dependent variance term remains deferred. Validated in closed form (V-44).
Interop (graph interchange)
- Graph interchange, not a fitting bridge. A new pure-R interop layer
(R/interop.R) translates a drmSEM component-labelled graph to and from the
neighbouring ecosystems' text formats. drmSEM still never fits its own
likelihoods, and lavaan/brms fitting interop stays out of the 0.x scope. as_lavaan(sem)(andas_lavaan(dag)) emits a lavaan model-syntax string:
oney ~ x1 + x2regression per endogenous node (the mean structure) and one
y1 ~~ y2line per declared covariance edge (covariances()).- Honesty: lavaan syntax cannot express a distributional-component path (an
arrow intosigma,zi,nu,hu,sd(group),rho12).as_lavaan()
therefore collapses to the mean structure and reports every dropped non-mu
path — both as adroppedattribute and via a one-timeclimessage. A non-mean
path is never silently misrepresented as a lavaan mean regression. from_lavaan(syntax)parses lavaan syntax back into a drmSEM graph skeleton:
~regressions become per-response node formulas in adrm_dag(), and~~
lines becomecovary()declarations. Reflective measurement (=~) lines are
ignored with a warning (reflective measurement needs a joint likelihood,
out of 0.x scope). Pure string parsing — nothing is evaluated or fitted, so
from_lavaan(as_lavaan(sem))round-trips the directed mean structure and the
covariance edges.as_dot(sem)(andas_dot(dag)) exports the component-labelled DAG as a
Graphviz DOT string: one labelled edge per typed edge, with non-mean paths
dashed/greyed. Unlike lavaan, DOT keeps every component path.
Feedback / cyclic motifs (0.5.0, grammar + equilibrium engine)
drm_cycle("y1", "y2")declares a feedback motif;drm_sem()/drm_psem()
gain afeedback =argument that accepts it. Cycles remain a hard error
unless declared — a declared motif is condensed into one topological layer,
so the DAG check still rejects every undeclared cycle.cycles(sem)lists the
declared motifs.- Honest fitting. Node-wise ML of a declared cycle is inconsistent under
simultaneity, sodrm_sem()warns: consistent estimation (IV/2SLS or a
joint likelihood) is an engine capability, not something drmSEM fakes. - d-separation drops independence claims among a motif's nodes (DAG
d-separation does not hold across a cycle; full sigma-separation is deferred). - Equilibrium total effects (0.5.x).
total_effects()now reports the
equilibrium response of a feedback SEM, iterating the mean-propagation map
to its fixed point (themediationcolumn reads"equilibrium"); if the
feedback diverges (no stable equilibrium, spectral radius>= 1) the estimate
isNAwith a warning — never a fabricated number.direct_effects()(the
controlled direct effect, which does not traverse the cycle) also works. The
mean/distribution decomposition through a cycle is out of scope, so
indirect_effects()/path_effects()refuse a feedback SEM and point to
total_effects(). The internalpropagate_fixedpoint()carries a
spectral-radius / max-iter stability guard; closed-form tests confirm it
recovers the linear reduced form(I − B)⁻¹ Γ(V-42) and that the equilibrium
total effect equals the reduced-form total effect of the exposure (V-43).
Design:docs/design/10-cyclic-feedback.md. - New vignette "Feedback cycles: reciprocal causation and the equilibrium
effect (drm_cycle)" works a reciprocalactivity ⇄ stresspair: why a
cycle breaks the DAG machinery (simultaneity bias in fitting; an equilibrium,
not a path product, for effects), the opt-indrm_cycle()/feedback =
grammar andcycles(), the equilibrium estimand ((I − B)⁻¹ Γ= the walk sum,
generalized to the fixed point; stability needsρ(B) < 1), and the
total_effects()equilibrium output (mediation = "equilibrium",target = "mean"only,NAon divergence) withindirect_effects()/path_effects()
refusing a feedback SEM. It is prominently honest that node-wise ML of a
declared cycle is inconsistent and drmSEM does not fake consistency.
Engine-dependent chunks are illustrative-only (eval = has_engine, FALSE).
Effect decomposition: paired Monte-Carlo and honest framing
- Bug fix (intervals).
indirect_effects()now computes the
controlled-direct, mean-mediated, and distribution-mediated legs from one
shared coefficient draw per replicate (drm_decomp_legs()), mirroring the
natural-effect branch. Previously the three legs were drawn independently, so
with the defaultseed = NULLthemean_mediated/distribution_mediated
intervals subtracted unrelated parameter draws and were inflated (not a
valid paired contrast). Point estimates are unchanged; the reported intervals
are now common-random-numbers (paired) contrasts that isolate the propagation
mode. - New asserted tests lock the shipped decomposition path (not only the
kernels): the additive identityindirect = mean_mediated + distribution_mediated(V-36), the lognormal Jensen-gap closed form and its
sign flip throughdrm_decomp_legs()(V-37), the linear-outcome zero (V-38), a
two-mediator chain (V-39), seed reproducibility (V-40), and an end-to-end
live-fit check that the distribution-mediated channel is real and the
decomposition closes (V-41). - Honest framing. Documentation now describes
distribution_mediatedas a
Jensen-gap / interventional-mediation term — non-zero only when a higher
moment of the mediator responds to the exposure and the outcome is curved
in that mediator — identified under the usual mediation assumptions plus a
correctly specified mediator distribution. The novelty is positioned as
implementational (a mediator'ssigma/zi/shape as a first-class causal
target), with the estimand credited to the interventional/distributional
mediation literature (Pearl; Imai et al.; VanderWeele 2015; Vansteelandt &
Daniel 2017).
Bivariate nodes (0.4, grammar layer)
drm_pair()declares a bivariate (joint two-response) node — two response
formulas, two families, an optionalrho12 ~ xresidual-correlation model (a
directed path into therho12component), and an auto-detected higher-level
corpairedge where the two responses share a grouping level. It is the
bivariate counterpart ofdrm_node().drm_expand_pair()bridges a pair onto the shipped covariance-edge grammar
(twodrm_node()sub-nodes +covary()edges), so the residual (rho12) and
higher-level (corpair) arcs flow throughcovariances()/basis_set()/
dsep()unchanged — the documented hook point for the 0.4 joint fit.rho12()andcorpairs()accessors report the declared residual and
higher-level correlation edges (of adrm_pairor adrm_sem), kept separate
frompaths(). TheestimateisNA: drmSEM never fits its own
likelihoods, so reading a fitted correlation back needs a live bivariate
drmTMBfit (the 0.4 engine deliverable). The declaration is never a
fabricated estimate. Seedocs/design/07-bivariate-covariance-edges.md.plot(sem)now draws covariance edges as double-headed arcs — solid grey
for a residualrho12, dashed grey for a higher-levelcorpair— so the three
edge classes (directed path, residual covariance, higher-level covariance) are
visually distinct. A newshow = c("all", "paths")argument toggles the arcs
off ("paths"draws the directed structural edges only).- New vignette "Bivariate nodes: two responses, one correlation (
drm_pair)"
walks an animal-personality example (activity & boldness) throughdrm_pair(),
print(),rho12(),corpairs(), anddrm_expand_pair(), motivating the
difference between a directedactivity -> boldnesspath and the residual
rho12/ higher-levelcorpaircovariance arcs. It is emphatic that the
correlation estimates areNAby construction — declaring a bivariate node
is not fitting one, and the joint bivariatedrmTMBfit is the 0.4 engine step.
Seedocs/design/07-bivariate-covariance-edges.md.
Path-specific effects (OQ-5)
path_effects(effect = "natural")reports each mediator's cross-world natural
indirect effect, with anidentifiedcolumn flaggedFALSEunder a
recanting witness (another mediator that is a descendant offromand an
ancestor of this one), after Avin, Shpitser & Pearl (2005). The detection is
pure graph logic; the cross-world computation reuses the validated
drm_natural_targetkernel.
Latent constructs (0.3)
drm_composite()now reports reliability (Cronbach's alpha, an
internal-consistency measure for reflective indicator sets — unclamped,NA
for a single indicator) and acceptsstandardize = TRUE(mean-0 / sd-1 score).
A newsummary.drm_composite()shows the loadings table, first-PC variance, and
reliability. A composite may now be used as a response (a node modelling the
construct), not only a predictor.- New vignette "Latent constructs: composites and reliability" walks through
formative vs reflective-flavoured (PCA) constructs, construct quality, and
construct-as-response; reflective measurement-model SEM stays out of scope
(needs a joint likelihood → 0.4), documented indocs/design/09-latent-variables.md.
Calibration (OQ-6)
- The live-drmTMB lane generated
inst/calibration/calibration-results.rds
(drmTMB0.1.3.9000, Git SHA17b1321, 14,400 calibration replicates). All
five pre-specified acceptance checks pass, so V-17 is promoted to validated
for the OQ-6 mean-only / distributional / cross-link calibration grid only;
broader Fisher's C / d-sep settings remain claim-scoped until separately
tested.