Skip to content

drmSEM 0.5.0

Latest

Choose a tag to compare

@itchyshin itchyshin released this 08 Jun 11:32
461159b

drmSEM 0.5.0

This release closes the cyclic / feedback-graph milestone (roadmap §0.5) and
ships the dev-line surface accumulated since 0.2.0. The DAG-only restriction is
lifted for declared feedback motifs, with an equilibrium estimand and a pure-R
fixed-point propagator; undeclared cycles remain a hard error. The feature
highlights below are grouped by area. Consistent feedback estimation (IV/2SLS or
a joint likelihood), full sigma-separation, distributional feedback equilibria,
and the joint bivariate fit remain engine-dependent and are carried forward to
the live-drmTMB lane (see docs/memory/CODEX_HANDOFF.md).

Sampler and propagation fixes

  • drm_sample_family() and effect propagation now match current drmTMB parameterization for the common sampler families in live recovery tests. Default fitted dpars such as sigma are carried into prediction engines even when no explicit sigma ~ ... formula is declared, and lognormal nodes now use mu = meanlog, sigma = sdlog, with mean mediation propagating exp(mu + sigma^2 / 2) (OQ-1, V-57..V-60).

DAG plot: faithful legend + readable edges

  • plot.drm_sem() now builds its legend from the components actually drawn
    (sourced from the same style function as the edges, so the two can never
    drift) instead of always listing all seven distributional components — the
    hero/landing-page DAG previously showed nu/hu/sd(.)/rho12 swatches for
    paths that did not exist. Covariance rows are added only for the classes truly
    present.
  • Parallel paths between one pair (e.g. a mu and a sigma arrow on the
    same edge) are now fanned onto separate arcs instead of overlapping into a
    single line, and a layout = matrix (optionally row-named) can be supplied for
    a fixed, crossing-free diagram. A node-fill legend (endogenous response vs
    exogenous predictor) is drawn, and hu gets a distinct linetype so it no longer
    relies on colour alone to separate from zi (colour-blind safety).
  • The legend construction is now a tested pure helper (drm_path_legend()), so a
    regression that re-introduces phantom legend entries fails CI.
  • Composite measurement edges: plot.drm_sem(show = "all") now draws each
    [drm_composite()] construct's indicators pointing into the construct as
    steel-blue arrows (indicators shown as distinctly-filled nodes), so a formative
    measurement model reads apart from the structural paths (OQ-15).

Outcome functionals across the effect API (OQ-11)

  • All three effect functions now report the effect on a chosen functional of the
    outcome distribution
    , not just the mean: target = "mean" / "p_gt" /
    "p_zero" / "var" / "quantile" (new, with a prob argument). target
    already rode direct_effects() / total_effects(); it now also rides
    indirect_effects() (effect = "controlled"), where every leg reports the
    contrast on the functional and the mean-/distribution-mediated split still
    closes (indirect = mean_mediated + distribution_mediated). This is where
    distribution-mediated paths earn their keep — a path into sigma/zi/nu can
    move a tail probability or quantile while leaving E[Y] nearly unchanged.
  • The "quantile" target reports the prob-quantile of the simulated outcome
    (kernel-validated: a path into sigma shifts the upper quantile but not the
    median).
  • direct_effects() / total_effects() gain functional = c("simulate", "analytic"). "analytic" evaluates the functional in closed form from the
    predicted parameters (no Monte-Carlo noise) for the gaussian and
    poisson families — exact var/p_gt/p_zero/quantile. Other families
    abort with a pointer back to "simulate" (their sigma↔dispersion scale is the
    OQ-1 open item); analytic needs mean mediation (method = "gcomp").
  • Fix: the functional engine now honours the mediator-propagation mode
    ("mean" vs "distribution") instead of always simulating the mediator, so the
    controlled decomposition is non-degenerate for a non-mean target. effect = "natural" remains mean-only (the cross-world functional contrast is open,
    OQ-8/OQ-11); a feedback SEM stays mean-only (the equilibrium response).

Validation wave 2 harness + newcomer docs

  • inst/validation/generate.R + a validation article now provide the wave-2
    coverage/calibration harness (effect-CI coverage against a known-effect
    linear-Gaussian DGP, and model-selection recovery rate), mirroring the OQ-6
    calibration pattern (cached .rds + a vignette that renders with or without the
    cache). Full replicate runs happen in the live lane; see
    docs/design/12-coverage-calibration.md.
  • The README quick-start and the intro vignette now open with a concrete
    biological question and show illustrative (clearly-marked, engine-free)
    paths() and indirect_effects() output so a newcomer can see the
    component-labelled and distribution_mediated rows and learn how to read them.

Simulation-based recovery grid (validation wave 1)

  • A campaign of numerical-recovery tests on real drmTMB fits now exercises
    the machinery end-to-end, not just for finiteness (V-45..V-73; see the new
    docs/design/11-validation-matrix.md). The effect decomposition is recovered
    across the family×link grid (gaussian, poisson, nbinom2, binomial,
    beta_binomial, beta, Gamma, lognormal) — mean-mediated equals the fitted-coef
    product / a predict_parameters() do-contrast, total = direct + indirect
    closes, and distribution_mediated matches the Jensen-gap magnitude from
    fitted params (the V-7 live-fit follow-up). Each family's drm_sample_family()
    mean and variance are checked against drmTMB::simulate() (closing OQ-1
    gaps), outcome functionals (p_zero/var/p_gt) are recovered, and the
    standardization sigma_E pipeline, composite-as-response, feedback equilibrium
    vs the fitted reduced form, and natural NDE/NIE are validated on live fits. A
    nonlinear feedback fixed point is added at the kernel tier (V-73). Tweedie /
    zero_one_beta inflation / student nu samplers stay flagged for the live lane.
    Wave 2 (effect-CI coverage, d-sep Type-I/power, model-selection recovery rate)
    is the calibration layer, tracked in CODEX_HANDOFF.md.

Standardization: GLM mean-path sigma_E (OQ-4)

  • The latent standardization of a mu path on a constant-variance link now
    divides by sqrt(Var(eta) + sigma_E^2), adding the link's theoretical
    latent-scale error variance — logit pi^2/3, probit 1, cloglog pi^2/6
    (Grace et al. 2018; piecewiseSEM's latent.linear). This corrects the earlier
    mild over-standardization of GLM mean paths. Identity-link mu and non-mu
    components (sigma/zi/sd(*)) are unchanged; the log-link families'
    mean-dependent variance term remains deferred. Validated in closed form (V-44).

Interop (graph interchange)

  • Graph interchange, not a fitting bridge. A new pure-R interop layer
    (R/interop.R) translates a drmSEM component-labelled graph to and from the
    neighbouring ecosystems' text formats. drmSEM still never fits its own
    likelihoods, and lavaan/brms fitting interop stays out of the 0.x scope.
  • as_lavaan(sem) (and as_lavaan(dag)) emits a lavaan model-syntax string:
    one y ~ x1 + x2 regression per endogenous node (the mean structure) and one
    y1 ~~ y2 line per declared covariance edge (covariances()).
  • Honesty: lavaan syntax cannot express a distributional-component path (an
    arrow into sigma, zi, nu, hu, sd(group), rho12). as_lavaan()
    therefore collapses to the mean structure and reports every dropped non-mu
    path — both as a dropped attribute and via a one-time cli message. A non-mean
    path is never silently misrepresented as a lavaan mean regression.
  • from_lavaan(syntax) parses lavaan syntax back into a drmSEM graph skeleton:
    ~ regressions become per-response node formulas in a drm_dag(), and ~~
    lines become covary() declarations. Reflective measurement (=~) lines are
    ignored with a warning (reflective measurement needs a joint likelihood,
    out of 0.x scope). Pure string parsing — nothing is evaluated or fitted, so
    from_lavaan(as_lavaan(sem)) round-trips the directed mean structure and the
    covariance edges.
  • as_dot(sem) (and as_dot(dag)) exports the component-labelled DAG as a
    Graphviz DOT string: one labelled edge per typed edge, with non-mean paths
    dashed/greyed. Unlike lavaan, DOT keeps every component path.

Feedback / cyclic motifs (0.5.0, grammar + equilibrium engine)

  • drm_cycle("y1", "y2") declares a feedback motif; drm_sem() / drm_psem()
    gain a feedback = argument that accepts it. Cycles remain a hard error
    unless declared — a declared motif is condensed into one topological layer,
    so the DAG check still rejects every undeclared cycle. cycles(sem) lists the
    declared motifs.
  • Honest fitting. Node-wise ML of a declared cycle is inconsistent under
    simultaneity, so drm_sem() warns: consistent estimation (IV/2SLS or a
    joint likelihood) is an engine capability, not something drmSEM fakes.
  • d-separation drops independence claims among a motif's nodes (DAG
    d-separation does not hold across a cycle; full sigma-separation is deferred).
  • Equilibrium total effects (0.5.x). total_effects() now reports the
    equilibrium response of a feedback SEM, iterating the mean-propagation map
    to its fixed point (the mediation column reads "equilibrium"); if the
    feedback diverges (no stable equilibrium, spectral radius >= 1) the estimate
    is NA with a warning — never a fabricated number. direct_effects() (the
    controlled direct effect, which does not traverse the cycle) also works. The
    mean/distribution decomposition through a cycle is out of scope, so
    indirect_effects() / path_effects() refuse a feedback SEM and point to
    total_effects(). The internal propagate_fixedpoint() carries a
    spectral-radius / max-iter stability guard; closed-form tests confirm it
    recovers the linear reduced form (I − B)⁻¹ Γ (V-42) and that the equilibrium
    total effect equals the reduced-form total effect of the exposure (V-43).
    Design: docs/design/10-cyclic-feedback.md.
  • New vignette "Feedback cycles: reciprocal causation and the equilibrium
    effect (drm_cycle)"
    works a reciprocal activity ⇄ stress pair: why a
    cycle breaks the DAG machinery (simultaneity bias in fitting; an equilibrium,
    not a path product, for effects), the opt-in drm_cycle() / feedback =
    grammar and cycles(), the equilibrium estimand ((I − B)⁻¹ Γ = the walk sum,
    generalized to the fixed point; stability needs ρ(B) < 1), and the
    total_effects() equilibrium output (mediation = "equilibrium", target = "mean" only, NA on divergence) with indirect_effects() / path_effects()
    refusing a feedback SEM. It is prominently honest that node-wise ML of a
    declared cycle is inconsistent and drmSEM does not fake consistency.
    Engine-dependent chunks are illustrative-only (eval = has_engine, FALSE).

Effect decomposition: paired Monte-Carlo and honest framing

  • Bug fix (intervals). indirect_effects() now computes the
    controlled-direct, mean-mediated, and distribution-mediated legs from one
    shared coefficient draw per replicate
    (drm_decomp_legs()), mirroring the
    natural-effect branch. Previously the three legs were drawn independently, so
    with the default seed = NULL the mean_mediated / distribution_mediated
    intervals subtracted unrelated parameter draws and were inflated (not a
    valid paired contrast). Point estimates are unchanged; the reported intervals
    are now common-random-numbers (paired) contrasts that isolate the propagation
    mode.
  • New asserted tests lock the shipped decomposition path (not only the
    kernels): the additive identity indirect = mean_mediated + distribution_mediated (V-36), the lognormal Jensen-gap closed form and its
    sign flip through drm_decomp_legs() (V-37), the linear-outcome zero (V-38), a
    two-mediator chain (V-39), seed reproducibility (V-40), and an end-to-end
    live-fit check that the distribution-mediated channel is real and the
    decomposition closes (V-41).
  • Honest framing. Documentation now describes distribution_mediated as a
    Jensen-gap / interventional-mediation term — non-zero only when a higher
    moment of the mediator responds to the exposure and the outcome is curved
    in that mediator — identified under the usual mediation assumptions plus a
    correctly specified mediator distribution. The novelty is positioned as
    implementational (a mediator's sigma/zi/shape as a first-class causal
    target), with the estimand credited to the interventional/distributional
    mediation literature (Pearl; Imai et al.; VanderWeele 2015; Vansteelandt &
    Daniel 2017).

Bivariate nodes (0.4, grammar layer)

  • drm_pair() declares a bivariate (joint two-response) node — two response
    formulas, two families, an optional rho12 ~ x residual-correlation model (a
    directed path into the rho12 component), and an auto-detected higher-level
    corpair edge where the two responses share a grouping level. It is the
    bivariate counterpart of drm_node().
  • drm_expand_pair() bridges a pair onto the shipped covariance-edge grammar
    (two drm_node() sub-nodes + covary() edges), so the residual (rho12) and
    higher-level (corpair) arcs flow through covariances() / basis_set() /
    dsep() unchanged — the documented hook point for the 0.4 joint fit.
  • rho12() and corpairs() accessors report the declared residual and
    higher-level correlation edges (of a drm_pair or a drm_sem), kept separate
    from paths(). The estimate is NA: drmSEM never fits its own
    likelihoods, so reading a fitted correlation back needs a live bivariate
    drmTMB fit (the 0.4 engine deliverable). The declaration is never a
    fabricated estimate. See docs/design/07-bivariate-covariance-edges.md.
  • plot(sem) now draws covariance edges as double-headed arcs — solid grey
    for a residual rho12, dashed grey for a higher-level corpair — so the three
    edge classes (directed path, residual covariance, higher-level covariance) are
    visually distinct. A new show = c("all", "paths") argument toggles the arcs
    off ("paths" draws the directed structural edges only).
  • New vignette "Bivariate nodes: two responses, one correlation (drm_pair)"
    walks an animal-personality example (activity & boldness) through drm_pair(),
    print(), rho12(), corpairs(), and drm_expand_pair(), motivating the
    difference between a directed activity -> boldness path and the residual
    rho12 / higher-level corpair covariance arcs. It is emphatic that the
    correlation estimates are NA by construction — declaring a bivariate node
    is not fitting one, and the joint bivariate drmTMB fit is the 0.4 engine step.
    See docs/design/07-bivariate-covariance-edges.md.

Path-specific effects (OQ-5)

  • path_effects(effect = "natural") reports each mediator's cross-world natural
    indirect effect
    , with an identified column flagged FALSE under a
    recanting witness (another mediator that is a descendant of from and an
    ancestor of this one), after Avin, Shpitser & Pearl (2005). The detection is
    pure graph logic; the cross-world computation reuses the validated
    drm_natural_target kernel.

Latent constructs (0.3)

  • drm_composite() now reports reliability (Cronbach's alpha, an
    internal-consistency measure for reflective indicator sets — unclamped, NA
    for a single indicator) and accepts standardize = TRUE (mean-0 / sd-1 score).
    A new summary.drm_composite() shows the loadings table, first-PC variance, and
    reliability. A composite may now be used as a response (a node modelling the
    construct), not only a predictor.
  • New vignette "Latent constructs: composites and reliability" walks through
    formative vs reflective-flavoured (PCA) constructs, construct quality, and
    construct-as-response; reflective measurement-model SEM stays out of scope
    (needs a joint likelihood → 0.4), documented in docs/design/09-latent-variables.md.

Calibration (OQ-6)

  • The live-drmTMB lane generated inst/calibration/calibration-results.rds
    (drmTMB 0.1.3.9000, Git SHA 17b1321, 14,400 calibration replicates). All
    five pre-specified acceptance checks pass, so V-17 is promoted to validated
    for the OQ-6 mean-only / distributional / cross-link calibration grid only;
    broader Fisher's C / d-sep settings remain claim-scoped until separately
    tested.