Skip to content

Releases: itchyshin/drmSEM

drmSEM 0.5.0

Choose a tag to compare

@itchyshin itchyshin released this 08 Jun 11:32
461159b

drmSEM 0.5.0

This release closes the cyclic / feedback-graph milestone (roadmap §0.5) and
ships the dev-line surface accumulated since 0.2.0. The DAG-only restriction is
lifted for declared feedback motifs, with an equilibrium estimand and a pure-R
fixed-point propagator; undeclared cycles remain a hard error. The feature
highlights below are grouped by area. Consistent feedback estimation (IV/2SLS or
a joint likelihood), full sigma-separation, distributional feedback equilibria,
and the joint bivariate fit remain engine-dependent and are carried forward to
the live-drmTMB lane (see docs/memory/CODEX_HANDOFF.md).

Sampler and propagation fixes

  • drm_sample_family() and effect propagation now match current drmTMB parameterization for the common sampler families in live recovery tests. Default fitted dpars such as sigma are carried into prediction engines even when no explicit sigma ~ ... formula is declared, and lognormal nodes now use mu = meanlog, sigma = sdlog, with mean mediation propagating exp(mu + sigma^2 / 2) (OQ-1, V-57..V-60).

DAG plot: faithful legend + readable edges

  • plot.drm_sem() now builds its legend from the components actually drawn
    (sourced from the same style function as the edges, so the two can never
    drift) instead of always listing all seven distributional components — the
    hero/landing-page DAG previously showed nu/hu/sd(.)/rho12 swatches for
    paths that did not exist. Covariance rows are added only for the classes truly
    present.
  • Parallel paths between one pair (e.g. a mu and a sigma arrow on the
    same edge) are now fanned onto separate arcs instead of overlapping into a
    single line, and a layout = matrix (optionally row-named) can be supplied for
    a fixed, crossing-free diagram. A node-fill legend (endogenous response vs
    exogenous predictor) is drawn, and hu gets a distinct linetype so it no longer
    relies on colour alone to separate from zi (colour-blind safety).
  • The legend construction is now a tested pure helper (drm_path_legend()), so a
    regression that re-introduces phantom legend entries fails CI.
  • Composite measurement edges: plot.drm_sem(show = "all") now draws each
    [drm_composite()] construct's indicators pointing into the construct as
    steel-blue arrows (indicators shown as distinctly-filled nodes), so a formative
    measurement model reads apart from the structural paths (OQ-15).

Outcome functionals across the effect API (OQ-11)

  • All three effect functions now report the effect on a chosen functional of the
    outcome distribution
    , not just the mean: target = "mean" / "p_gt" /
    "p_zero" / "var" / "quantile" (new, with a prob argument). target
    already rode direct_effects() / total_effects(); it now also rides
    indirect_effects() (effect = "controlled"), where every leg reports the
    contrast on the functional and the mean-/distribution-mediated split still
    closes (indirect = mean_mediated + distribution_mediated). This is where
    distribution-mediated paths earn their keep — a path into sigma/zi/nu can
    move a tail probability or quantile while leaving E[Y] nearly unchanged.
  • The "quantile" target reports the prob-quantile of the simulated outcome
    (kernel-validated: a path into sigma shifts the upper quantile but not the
    median).
  • direct_effects() / total_effects() gain functional = c("simulate", "analytic"). "analytic" evaluates the functional in closed form from the
    predicted parameters (no Monte-Carlo noise) for the gaussian and
    poisson families — exact var/p_gt/p_zero/quantile. Other families
    abort with a pointer back to "simulate" (their sigma↔dispersion scale is the
    OQ-1 open item); analytic needs mean mediation (method = "gcomp").
  • Fix: the functional engine now honours the mediator-propagation mode
    ("mean" vs "distribution") instead of always simulating the mediator, so the
    controlled decomposition is non-degenerate for a non-mean target. effect = "natural" remains mean-only (the cross-world functional contrast is open,
    OQ-8/OQ-11); a feedback SEM stays mean-only (the equilibrium response).

Validation wave 2 harness + newcomer docs

  • inst/validation/generate.R + a validation article now provide the wave-2
    coverage/calibration harness (effect-CI coverage against a known-effect
    linear-Gaussian DGP, and model-selection recovery rate), mirroring the OQ-6
    calibration pattern (cached .rds + a vignette that renders with or without the
    cache). Full replicate runs happen in the live lane; see
    docs/design/12-coverage-calibration.md.
  • The README quick-start and the intro vignette now open with a concrete
    biological question and show illustrative (clearly-marked, engine-free)
    paths() and indirect_effects() output so a newcomer can see the
    component-labelled and distribution_mediated rows and learn how to read them.

Simulation-based recovery grid (validation wave 1)

  • A campaign of numerical-recovery tests on real drmTMB fits now exercises
    the machinery end-to-end, not just for finiteness (V-45..V-73; see the new
    docs/design/11-validation-matrix.md). The effect decomposition is recovered
    across the family×link grid (gaussian, poisson, nbinom2, binomial,
    beta_binomial, beta, Gamma, lognormal) — mean-mediated equals the fitted-coef
    product / a predict_parameters() do-contrast, total = direct + indirect
    closes, and distribution_mediated matches the Jensen-gap magnitude from
    fitted params (the V-7 live-fit follow-up). Each family's drm_sample_family()
    mean and variance are checked against drmTMB::simulate() (closing OQ-1
    gaps), outcome functionals (p_zero/var/p_gt) are recovered, and the
    standardization sigma_E pipeline, composite-as-response, feedback equilibrium
    vs the fitted reduced form, and natural NDE/NIE are validated on live fits. A
    nonlinear feedback fixed point is added at the kernel tier (V-73). Tweedie /
    zero_one_beta inflation / student nu samplers stay flagged for the live lane.
    Wave 2 (effect-CI coverage, d-sep Type-I/power, model-selection recovery rate)
    is the calibration layer, tracked in CODEX_HANDOFF.md.

Standardization: GLM mean-path sigma_E (OQ-4)

  • The latent standardization of a mu path on a constant-variance link now
    divides by sqrt(Var(eta) + sigma_E^2), adding the link's theoretical
    latent-scale error variance — logit pi^2/3, probit 1, cloglog pi^2/6
    (Grace et al. 2018; piecewiseSEM's latent.linear). This corrects the earlier
    mild over-standardization of GLM mean paths. Identity-link mu and non-mu
    components (sigma/zi/sd(*)) are unchanged; the log-link families'
    mean-dependent variance term remains deferred. Validated in closed form (V-44).

Interop (graph interchange)

  • Graph interchange, not a fitting bridge. A new pure-R interop layer
    (R/interop.R) translates a drmSEM component-labelled graph to and from the
    neighbouring ecosystems' text formats. drmSEM still never fits its own
    likelihoods, and lavaan/brms fitting interop stays out of the 0.x scope.
  • as_lavaan(sem) (and as_lavaan(dag)) emits a lavaan model-syntax string:
    one y ~ x1 + x2 regression per endogenous node (the mean structure) and one
    y1 ~~ y2 line per declared covariance edge (covariances()).
  • Honesty: lavaan syntax cannot express a distributional-component path (an
    arrow into sigma, zi, nu, hu, sd(group), rho12). as_lavaan()
    therefore collapses to the mean structure and reports every dropped non-mu
    path — both as a dropped attribute and via a one-time cli message. A non-mean
    path is never silently misrepresented as a lavaan mean regression.
  • from_lavaan(syntax) parses lavaan syntax back into a drmSEM graph skeleton:
    ~ regressions become per-response node formulas in a drm_dag(), and ~~
    lines become covary() declarations. Reflective measurement (=~) lines are
    ignored with a warning (reflective measurement needs a joint likelihood,
    out of 0.x scope). Pure string parsing — nothing is evaluated or fitted, so
    from_lavaan(as_lavaan(sem)) round-trips the directed mean structure and the
    covariance edges.
  • as_dot(sem) (and as_dot(dag)) exports the component-labelled DAG as a
    Graphviz DOT string: one labelled edge per typed edge, with non-mean paths
    dashed/greyed. Unlike lavaan, DOT keeps every component path.

Feedback / cyclic motifs (0.5.0, grammar + equilibrium engine)

  • drm_cycle("y1", "y2") declares a feedback motif; drm_sem() / drm_psem()
    gain a feedback = argument that accepts it. Cycles remain a hard error
    unless declared — a declared motif is condensed into one topological layer,
    so the DAG check still rejects every undeclared cycle. cycles(sem) lists the
    declared motifs.
  • Honest fitting. Node-wise ML of a declared cycle is inconsistent under
    simultaneity, so drm_sem() warns: consistent estimation (IV/2SLS or a
    joint likelihood) is an engine capability, not something drmSEM fakes.
  • d-separation drops independence claims among a motif's nodes (DAG
    d-separation does not hold across a cycle; full sigma-separation is deferred).
  • Equilibrium total effects (0.5.x). total_effects() now reports the
    equilibrium response of a feedback SEM, iterating the mean-propagation map
    to its fixed point (the mediation column reads "equilibrium"); if the
    feedback diverges (no stable equilibrium, spectral radius >= 1) the estimate
    is NA with a warning — never a fabricated number. direct_effects() (the
    controlled direct effect, which does not traverse the cycle) also works. The
    mean/distribution decomposition through a cycle is out of scope, so
    indirect_effects() / path_effects() refuse a feedba...
Read more

drmSEM 0.2.0

Choose a tag to compare

@itchyshin itchyshin released this 06 Jun 19:32
39c7c36

drmSEM 0.2.0

Second release. Post-0.1 work: a unified effect-API surface, first-class
covariance-edge and composite-construct grammars, per-mediator and per-component
path attribution, finalized standardization conventions, and a scaffolded
Fisher's C calibration study. The any-component d-separation calibration remains
experimental until its (compute-heavy) study is run; everything else is
CI-validated against a live drmTMB or kernel-validated by closed-form tests.

Documentation

  • New vignette "Covariance edges, composites, and path attribution" walks
    through covary()/covariances(), drm_composite()/loadings(), and
    path_effects(). Every exported function now carries an example.

Path-specific effects (OQ-5, per-mediator)

  • path_effects(object, from, to, through=) decomposes the indirect effect into
    a per-mediator contribution: an inclusion effect (T({Mj}) - direct, only
    Mj responds) and an exclusion effect (T(all) - T(all \\ Mj), Mj's
    marginal given the rest), plus total_indirect and an explicit
    interaction_remainder. The pieces sum to the total only in the additive case;
    the remainder is reported, never forced to zero. Model-based attribution, not a
    nonparametric path-specific identification claim. The cross-world natural
    variant is the OQ-5 follow-up.
  • path_effects(by = "component") splits each mediator's effect into a
    mean_channel and one channel per non-mean component (sigma_channel,
    zi_channel, ... -- the drop when that component is frozen at its reference
    value), plus a component_remainder for the part that does not separate cleanly
    under a nonlinear outcome.

Composite latent constructs (0.3, first increment)

  • drm_composite(name, indicators, weights, method = c("fixed", "pca")) declares
    a composite (formative) construct — a weighted sum or first-PC index of
    observed indicator columns. It is materialized as an ordinary column before
    fitting, so a node formula can use it as a predictor or response with no engine
    change.
  • drm_sem() / drm_psem() gain a composites = argument; loadings(sem)
    reports the indicator-to-construct loadings, kept separate from paths().
  • Reflective (measurement-model) latent variables remain out of scope (they need
    a joint likelihood); see docs/design/09-latent-variables.md.

Inference hardening (towards 0.2)

  • Analytic effect cross-checks are now asserted tests (test-analytic-effects.R,
    pure-R, no engine): the Gaussian identity-link mean-mediated effect equals the
    coefficient product a*b*w; a non-mean (sigma) path contributes exactly
    nothing to the mean channel and the distribution-mediated effect goes to zero
    when the outcome is linear in the mediator; the distribution-mediated effect
    matches the lognormal closed form (and flips sign) across a downstream
    nonlinearity; natural and controlled effects diverge under an exposure-mediator
    interaction by the predicted amount; and the Poisson Pr(Y>0) and Gaussian
    Var(Y) outcome-functional effects match their closed forms.
  • Standardization conventions finalized and documented (OQ-4; see
    docs/design/08-standardization.md and ?standardize): standardized
    coefficients are reported on the link scale only; factor predictors keep SD = 1
    (raw per-contrast effect, lavaan std.nox convention); the latent divisor is
    per-component, so sigma/zi paths standardize on their own link scale. A
    Gelman 2-SD opt-in for continuous-vs-factor comparability and a theoretical-
    variance term for GLM mean paths are noted as tracked refinements.

Covariance edges: rho12 and corpair (OQ-14, grammar layer)

  • covary(y1, y2, level = ) declares a covariance edge between two responses
    — a residual correlation (rho12, within-observation, level = NULL) or a
    higher-level random-effect correlation (corpair, between-unit, a grouping
    level). Covariance edges are double-headed arcs: they carry no direction and
    no mediated effect.
  • drm_sem() / drm_psem() gain a covariances = argument that takes covary()
    declarations and validates them against the node records.
  • covariances(sem) reports residual and higher-level edges separately, kept
    out of paths() (which stays directed-only, including any x -> rho12 path).
  • basis_set() / dsep() are now covariance-aware: a declared rho12 /
    corpair edge between y1 and y2 drops the y1 _||_ y2 | predictors
    independence claim (Shipley's bidirected-edge rule).
  • This is the pure-R grammar/d-separation layer. drm_pair() (joint bivariate
    fitting), rho12() / corpairs() accessors that read a live fit, and
    double-headed-arc plotting need a bivariate drmTMB fit and remain on the
    roadmap (OQ-14).

Unified effect-API surface (OQ-12)

  • direct_effects(), total_effects(), and indirect_effects() now share one
    argument vocabulary: uncertainty = c("parametric", "none", "bootstrap"),
    nsim (inner distributional realizations), and population = c("conditional", "marginal"). total_effects() selects mediation with method = c("gcomp", "simulate").
  • The previous mediation, draw, and n_sim arguments are deprecated
    aliases
    — they still work but emit a deprecation warning, and the new
    argument wins when both are supplied. B (the number of uncertainty
    replicates) is unchanged. No simulation kernel changed.
  • Not-yet-implemented choices fail fast with a pointer to the tracking question:
    uncertainty = "bootstrap" (refit bootstrap, OQ-10) and population = "marginal" (marginalizing over the random-effect distribution, OQ-9).
  • direct_effects() gains target / threshold, so a controlled direct effect
    can be read on an outcome functional (p_gt, p_zero, var) as well as the
    mean, matching total_effects().