Skip to content

Releases: ehrlinger/TemporalHazard

TemporalHazard 1.2.17

Choose a tag to compare

@ehrlinger ehrlinger released this 28 Oct 13:03
219e21b
  • hzr_deciles() ranked a multiphase time_windows fit by the wrong
    window's coefficients (#629).
    For the horizon ranking it read each
    subject's stored design, which time_windows had expanded at that
    subject's exit time, so a subject who exited in one window was ranked
    with that window's coefficient even when the horizon falls in another.
    On two 400-subject examples, 12 and 85 subjects landed in the wrong risk
    group, and mean_survival was wrong in every group, with no warning. The horizon
    design is now built at the horizon through predict(), as hzr_gof()
    and the single-distribution path already do. A caller's event_time is
    handled the same way: the design is expanded at event_time whenever it
    differs from the stored times, for H(event_time) and, on a
    left-truncated fit, for the H(entry) subtracted from it, so the two use
    the same window. Expected events at the default event_time, each
    subject's own exit time, are unchanged.

  • On a multiphase time_windows fit saved by 1.2.9 or earlier,
    predict(newdata = ) and hzr_gof() could swap a phase formula's own
    covariates for the global ones (#658).
    Such a fit has no record of which
    phases took the global design, so it was read from the stored design's
    column names. A phase formula can use columns named like the
    window-expanded global ones (x1_w1, x1_w2) that hold other covariates,
    and that phase was read as inheriting. predict(newdata = ) was then off
    by up to 77% relative in one example, and hzr_gof()'s curve moved, with
    no warning. A phase is now read as inheriting only when its stored design
    equals the global design, expanded at the stored times, in values as well
    as names. Fits saved with the record, and every fit made by a current
    version, are unchanged.

  • hzr_deciles() miscounted expected events when a caller's event_time
    disagreed with a row's time_lower (#656).
    Both cases were silent.
    An event_time at or below a left-truncated row's entry dropped the
    entry correction, H(entry), so the row counted H(event_time) expected
    events where none are defined; that is now an error naming the first such
    row. And a row with time_lower equal to its stored time, which the
    likelihood reads as having no entry, was read as truncated once
    event_time was later than that time, losing H(time). Truncation is now
    decided as the likelihood decides it: status 0 or 1 with
    0 < time_lower < time at the stored time. Calls that leave event_time
    at its default, and hzr_gof(), are unchanged.

  • hzr_bootstrap() pooled replicates with no finite maximum (#654). A
    resample with separation, such as no events at one level of a factor, has
    a likelihood that keeps rising as a coefficient, the intercept or a
    phase's log_mu runs to infinity. The refit stops where the slope falls
    below the optimizer's tolerance and reports convergence with a small
    gradient, so it passed every check and its estimate was pooled. In one
    example 12 of 40 replicates ran out to between -7 and -15 against -1.4 on
    the full data, and the coefficient's bootstrap sd was 5.0 where the other
    28 give 0.51, with no warning. Each replicate is now tested by stepping
    further out from its estimate: along each coefficient, and along the
    direction its standard errors, or failing those its Hessian, say is
    flattest, which is the one a separation along a combination of
    parameters takes. When the log-likelihood does not fall, the replicate
    fails with the reason
    "likelihood flat or still rising (no unique finite maximum)", it leaves
    the summary, and a warning gives the count. A replicate whose
    design is rank-deficient, such as a factor level absent from the resample
    or a column that is an exact combination of others, is dropped the same
    way: its estimates are not determined by the data. The fit being
    bootstrapped is tested the same way first, and warned about if it fails,
    as a multiphase phase with covariates but few events of its own can. The
    step is sized on the whole linear predictor, intercept included, and the
    tolerance scales with the mean weight, so a covariate far from zero or
    very small weights do not make a genuine maximum read as flat.
    The warning also says what the summary is now conditional on: only the
    resamples in which the model is identified, which understates the
    uncertainty and moves the other parameters' intervals too. On avc with
    a constant phase carrying mal, 17 of 30 replicates were dropped and the
    constant.mal interval narrowed from about [-19, 12] to [-0.1, 1.8].
    When 20% or more of the replicates are dropped, the warning says to treat
    the summary as unreliable.

  • A Conservation of Events fit could report wrong standard errors when its
    full-information variance could not be computed (#652).
    The variance from
    the search holds the conserved phase's log_mu fixed, so it treats that
    parameter as known, and it was taken at the starting value of log_mu
    rather than the solved one. When the full-information Hessian could not be
    computed or inverted, the fit fell back to that variance and reported a
    standard error for every other parameter, while the warning named only the
    conserved log_mu. On avc with two phases, one standard error was 0.17
    of its full-information value and another 1.3 times it. Such a fit now
    reports no standard errors, warns that none are reported, and records why
    in degraded_causes; vcov() returns NA. The estimates are unchanged.
    The conserved log_mu is also now counted as estimated, as it is when the
    recompute succeeds, so the fit's AIC no longer comes out 2 lower than the
    same fit's with a successful recompute.

  • Behaviour change: hzr_bootstrap(seed = ) now sets the session's random
    number stream, as set.seed() does, and no longer restores it on exit (#653).
    Restoring the
    caller's stream (#611) meant writing .Random.seed into the global
    environment, which CRAN policy does not allow. The replicates for a given
    seed are unchanged. What changes is the stream after the call: it continues
    from where the bootstrap's draws left it, so a script that seeded once at
    the top and draws after a seeded bootstrap now gets different numbers from
    those draws. Save .Random.seed before the call and restore it after, or use
    withr::with_preserve_seed(), to keep your own stream. withr moves from
    Imports to Suggests.

  • A . inside a function call in a formula was dropped without a word.
    terms() writes out . only where it is a term of its own, so
    Surv(time, status) ~ log(abs(.)) reached the design as ~ 1 and the
    model was fitted with no covariates. The same held for a phase's
    hzr_phase(formula = ). Beside a bare ., as in ~ . + log(.), terms()
    substituted the sum of the columns, giving one covariate log(x1 + x2).
    hazard() now stops on . anywhere but as a term of the formula
    operators, naming the term, as lm() stops on these formulas. ~ .,
    ~ . - age, ~ (.)^2 and ~ .:age are unchanged.

  • A formula helper that reads an outside vector by position is now refused
    when rows at time 0 are dropped (#590).
    hazard() drops a row at time 0
    before fitting, and builds the design on the rows that remain. A helper
    such as function(a) a + g[seq_along(a)], with g aligned to every row
    given, then paired the remaining rows with the first values of g, with
    no warning. In one example all 48 of 48 rows were paired with the wrong
    value; in another, an alternating treatment arm read this way fitted its
    coefficient with the sign reversed. The existing check reads only the
    variables the formula names, so it could not see g inside the function.
    hazard() now also evaluates the formula's variables on every row in
    order and again with the remaining rows first, as the fit sees them, and
    refuses a variable that then gives a remaining row a value other than its
    own, naming it. Terms computed from the rows as a set, such as scale(),
    poly(), ns() and factors, pass and still use the remaining rows. A
    term that depends on the order of the rows, such as cumsum(age), or is
    random, such as jitter(age), is now refused when rows other than the
    last are dropped, where it was fitted before. A helper that depends on
    how many rows it is given, such as tail(g, length(a)) or
    a * mean(g[seq_along(a)]), is not caught. Put such a vector in data,
    where it is dropped with its rows.

  • hzr_bootstrap(id =) resamples patients, not rows (#667). Data with
    several rows per patient, such as time-varying covariates in
    counting-process form (Surv(start, stop, event) or time_lower =), were
    resampled row by row, so a replicate could hold some of a patient's
    segments and not others, or one of them twice. id names the patient,
    as a column of data or a vector with one value per row; each replicate
    then draws patients with replacement and keeps every row of each one
    drawn, and fraction counts patients. This is SAS %HAZBOOT with
    _TVC=1, _IDVAR=. The default, id = NULL, resamples rows as before, and
    the same seed gives the same replicates as before. A fit with entry
    times and no id now draws a warning that its rows may not be patients.

TemporalHazard 1.2.16

Choose a tag to compare

@ehrlinger ehrlinger released this 14 Oct 17:15
fc76a7b
  • The SAS-parity data resolvers no longer fall back to a hard-coded study
    path; set HAZARD_MAZE_DATASETS and HAZARD_ACHALASIA_DIR (once, in
    ~/.Renviron) to run the volume-gated parity tests and
    tools/release-suite.R. The repository is public, and a study's folder name
    says which study it is.

TemporalHazard 1.2.15

Choose a tag to compare

@ehrlinger ehrlinger released this 12 Oct 22:10
db4934d

Bug fixes

  • A multiphase fit no longer selects a start that ran away to an
    underflow (#600).
    For an early phase with m = 0 and nu < 0,
    hzr_decompos() computed the CDF as 1 - exp(-x), which rounds to
    exactly 0 once x is below about 1e-16, while the density, formed
    directly, stayed positive. Where that happened the cumulative hazard
    vanished and the hazard did not, so the likelihood had a false peak that
    rose without bound in mu. On the avc data, the m-fixed model started
    from t_half = 0.5, nu = 1, mu = 1 with the default five starts
    climbed it to log(mu) 130 and a log-likelihood of +6380.86, and returned
    that start as the fit with converged = TRUE; three of the other four
    reach -214.875. The CDF is now -expm1(-x), as the m > 0, nu < 0
    branch already had it, and the same call returns -214.875. The two forms
    round differently everywhere, not only near 0, so other fits with an
    m = 0, nu < 0 phase can change in their last digits (by about 1e-15
    relative); fits without one are unchanged.

Tests

  • Two tests now pin the speed-up from 1.2.14's CoE gradient cache (#631).
    One counts the per-phase work and checks that a call matching the cache
    key does none, and that changing any one of theta, the times or the
    entry times alone forces it. The other runs a real CoE fit and checks that
    the second gradient evaluation at each point reuses the first's cache and
    does no work. The existing test compared gradients, which a recomputed
    value matches too, so it passed with caching switched off, with theta,
    the times or the entry times dropped from the key, and with the CoE
    wrapper handing its second evaluation a fresh cache. The new tests fail on
    each of those.

TemporalHazard 1.2.14

Choose a tag to compare

@ehrlinger ehrlinger released this 12 Oct 02:04
b4aa29e

Performance

This release removes the slowdown 1.2.13 recorded as a known limitation
(#631). Every estimate, standard error, prediction and bootstrap replicate is
bit-identical to 1.2.13's; only the time to compute them changes. The
package's own R CMD check --as-cran now takes about 400 seconds, where
1.2.13's took about 730.

  • Multiphase fits are about twice as fast again (#631). Since #578,
    each evaluation of a phase formed log(1 - G) and the hazard on the log
    scale, which only a "hazard" phase reads, and formed some of it four
    times over. A "cdf" phase now skips that work, and a "hazard" phase
    with m and nu both positive forms each piece once. On the cabgkul
    data a fixed-shape three-phase fit takes 1.0 seconds against 1.75, and a
    5-replicate bootstrap of it 3.4 seconds against 8.1.

  • Fits under Conservation of Events are a further 20 to 25% faster
    (#631).
    The gradient under CoE evaluates the score twice at each point
    (#565), and the second evaluation now reuses the first's per-phase
    quantities instead of recomputing them. hzr_log1pexp() forms each
    branch only where it is taken, where it used ifelse(), which formed
    both over every element; its values, attributes and NAs are unchanged
    for every input.

Tests

  • The repeated-events parity test of the cardioversion job now expects the
    $untranslated row for its PROC HAZPRED, which stands inside a
    %MACRO definition and has been refused since #625. Its two fits had
    failed since that change. The test needs the study datasets, which no CI
    runner has, so the failure was not reported. The package's behaviour is
    unchanged.

TemporalHazard 1.2.13

Choose a tag to compare

@ehrlinger ehrlinger released this 11 Oct 12:09
eac3dc1

Breaking changes

  • hzr_bootstrap(seed = ) no longer resets your random number stream
    (#611).
    It seeded the global generator with set.seed() and left it
    there, so a script that called set.seed() once at the top lost that seed
    at its first seeded bootstrap, and every draw after it followed from the
    bootstrap's seed instead. The seed now holds for the call only, through
    withr::local_seed(), and your stream is restored when the call returns or
    fails. A draw after hzr_bootstrap(seed = k) no longer follows from
    k
    : a script that relied on that must call set.seed(k) itself. The
    replicates for a given seed are unchanged, and equal those of
    set.seed(k) followed by seed = NULL. seed must now be NULL or a
    single whole number: 1.7, c(1, 2) and TRUE used to act silently as
    seed = 1, and NA failed with a message that did not name seed.
    withr moves from Suggests to Imports; it depends only on base R.
    This restores the 1.0.2 behaviour, which 1.0.3 removed.

  • Estimates and selections from earlier versions may change when
    refitted.
    Several fixes change what a fit or a screen returns, not only
    what it reports: multiphase fits under Conservation of Events, which could
    stop short of the maximum (#565); fits with a "hazard" phase far past
    saturation (#578); multiphase fits whose search passed a very small
    t_half (#574); multiphase stepwise selections and hzr_bootstrap()
    select-mode frequencies, whose candidate refits never started from the
    current model (#551); score-criterion selections at a step whose
    information cannot be formed, which now refit and test their candidates
    instead of stopping (#570); Wald selections, which no longer enter a
    candidate whose refit ended below the base (#538); and multiphase AIC
    selections over a candidate with missing values (#488). Refit models and
    rerun screens from earlier versions; the entries under Bug fixes give the
    details.

  • Some calls that returned a value now stop, or return something
    different.

    • hazard() refuses a control$maxit below 1 or not a number ("200"
      was accepted) (#541), and hazard(), hzr_evaluate() and predict()
      refuse a Weibull mu or nu too small to hold as a normal double,
      such as 1e-310, which 1.2.12 fitted and evaluated (#566).
    • summary()$log_lik is NA for an objective = "sas" fit that read
      an interval-censored row, whose objective is not a log-likelihood
      (#544).
    • hzr_decompos() returns a fourth element, log_surv (#578).
    • hzr_stepwise() refuses a data whose rows are not the fit's rows in
      the fit's order (#487, #515), and any data with a duplicated column
      name, whether or not that column is in the scope (#515).
    • hzr_stepwise() refuses a base model that did not converge under
      criterion = "aic", and under "wald" with direction = "backward"
      or "both"; select-mode hzr_bootstrap() refuses such a base up
      front, and fails a replicate whose base refit did not converge (#627).
    • hzr_deciles() refuses a multiphase fit whose design dropped rows, as
      hzr_gof() already did (#627).
  • Some returned objects have new elements, and one moves others. A
    Weibull fit's fit$fit gains log_scale before vcov, so vcov,
    rcond, pd, counts, message, rel_gradient,
    rel_gradient_reason and polish_code move position: read them by name
    (#566). hzr_decompos() returns log_surv (#578); a stepwise result's
    $steps, and as.data.frame() of it, gain n_rows (#519);
    attr(hzr_gof(), "summary") gains n_tallied (#492); hzr_evaluate()
    returns $objective, and summary() gains objective and
    objective_value (#544, #556).

Bug fixes

  • Standard errors computed numerically could be far too small, with no
    warning, when a covariate was far from zero (#598).
    Fits with left- or
    interval-censored rows, which have no analytic Hessian, take theirs
    from numDeriv::hessian(), as does the score criterion of
    hzr_stepwise(). Its default step is a tenth of each parameter's size,
    and an intercept that absorbs a covariate's offset is large, so the step
    spanned many standard errors. With the covariate offset by 100, a fit
    reached the same log-likelihood as the centred fit and reported SE(beta)
    at 0.035 (Weibull), 0.047 (exponential), 0.073 (log-logistic) and 0.52
    (lognormal) of the centred fit's; at an offset of 1000, an exponential
    SE was 1e-28 of it, a Weibull fit had no covariance, and a lognormal
    fit's standard errors were NA.

    • The step is now set from the curvature: 0.1 standard error along each
      direction of the Hessian, refined twice, with each direction's step
      capped at numDeriv's own first step in every parameter. The same fits
      now agree with the centred fit's standard errors at offsets of 100
      and 1000.
    • A Hessian that is not positive definite is still reported as found.
    • The remaining limit is double precision: at an offset of 1e4 the
      exact Hessian's reciprocal condition number is about 1e-16, so no
      Hessian on that scale can be inverted reliably. Centring the
      covariates avoids it.
  • A single-distribution fit (exponential, Weibull, lognormal or
    loglogistic) could stop at a false maximum, far below the best
    log-likelihood, and report converged = TRUE with no warning (#531).

    On the avc data an exponential fit with ~ age + mal from an ordinary
    start stopped at a log-likelihood of -80896.5 against a best of -394.8.
    hazard() now warns, with class "hzr_possible_false_maximum", when such
    a fit reports convergence with a relative gradient above 1e-3 and the
    continuation did not stop on its iteration limit or an unbounded direction
    (nlm() codes 4 and 5, which already warn). The warning says the fit may not
    be a maximum, names the relative gradient and suggests other starting
    values, or centring or rescaling the covariates. converged is not
    changed, and nothing is refused. hzr_bootstrap(), whose replicates run
    with their warnings suppressed, counts the replicates in which any fit
    meets the same rule (the base refit and every stepwise refit, not only
    the final fit) and warns once; their estimates are still pooled. The threshold sits
    above every good fit in the test suite (the worst at 6.8e-4) and below
    the false maxima found, but it does not separate every case. A good fit
    on badly scaled covariates (one multiplied by 1e4 or more) can exceed it,
    with a relative gradient of 1e-3 to 5e-3 and a log-likelihood up to 0.02
    short; there another start does not help, and centring or rescaling
    does. And a stuck fit can fall below it: an intercept-only Weibull on
    times near 1e-170, started at a shape of 1.5, stops there against 1.72
    with a relative gradient of about 5e-4 and raises nothing. The absence of
    the warning is not a guarantee. Multiphase fits are not flagged.

  • predict(se.fit = TRUE) could return wrong standard errors, including
    exactly 0, with no warning, when the fit's covariance was incomplete or not
    positive definite (#586).

    • When inverting the Hessian finds a variance that is not positive, the
      fit stores it as NA, the same mark a fixed parameter carries.
      predict() dropped such a parameter as if it were known. On a Weibull
      fit with mu's variance masked, survival standard errors of 0.0175,
      0.0345 and 0.0392 came back as 0.0221, 0.0221 and 0.0524. An NA
      variance now counts as fixed only where the fit's fixed_mask says
      so. Otherwise, a prediction that uses the parameter returns NA
      standard errors and limits, with a warning naming it.
    • A covariance with a positive diagonal can still be indefinite, which
      happens when the Hessian was not at a proper maximum. A negative
      quadratic form was then clamped to an se.fit of 0 with a zero-width
      interval, and a positive one was reported although it is not a
      variance. The covariance of the parameters a prediction uses is now
      checked for positive definiteness, and an infinite covariance among
      them is refused.
    • Parameters the prediction does not use, by its form, are left out of
      that check and of the sandwich. An infinite covariance between two of
      them no longer turns se.fit into NaN (#587).
    • When Conservation of Events could not recompute the conserved phase's
      variance, predict() keeps its standard errors but now warns that
      they leave that variance out and may be understated.
  • After rows at time 0 are dropped, messages are clearer (#484).

    • Row numbers in later error messages refer to the rows as given, not
      to their positions among the rows that remain: an NA status on row
      10 is reported as row 10, where it used to be reported as row 9.
    • A warning raised by a term of the formula, such as one a function in
      it raises, was shown twice, because the formula is evaluated again on
      the rows that remain. Only a warning that evaluation raises anew is
      shown now.
    • hzr_stepwise() given data with exactly the fit's rows plus the
      ones it dropped, which it could not confirm as the fit's data, now
      warns once that those rows were dropped and should be dropped from
      data. Before, it reported only a mismatched row count.
    • hzr_stepwise()'s note that it could not check the row order of
      data said, for a formula fit given data of another length, that
      the stored data frame "has 116 rows, not the fit's 116". It now
      compares data with the stored data frame, as the check does.
  • **Multiphase fits under Conservation of Events could stop short of the
    maximum and report converged = TRUE (#565). Multiphase estimates from
    earlier versions, fitted with conserve = TRUE (the default), may be
    wro...

Read more

TemporalHazard 1.2.12

Choose a tag to compare

@ehrlinger ehrlinger released this 26 Sep 12:52
b56df0d

TemporalHazard 1.2.12 is a GitHub-only release, not submitted to CRAN. It carries all the work since v1.2.9, including the never-tagged 1.2.10 and 1.2.11. It has passed R CMD check --as-cran with the PDF manual on ubuntu (devel and release), Windows (devel and release) and macOS (check-release run 36238494826).

The full change log is too long for a release page (about 176 kB). Read it in NEWS.md at v1.2.12; it is also attached below as NEWS-1.2.12.md. The breaking changes are reproduced here, because they can change the results of existing code.

Breaking changes

  • A g3 phase with alpha fixed at 1 is now fitted as PROC HAZARD fits it,
    so its reported tau, gamma, eta and mu can change (#415).
    At
    alpha = 1 the g3 form is (t/tau)^(gamma*eta), so tau is confounded
    with mu, and gamma with eta. With them free the fit walked a ridge
    and reported converged = TRUE at an arbitrary point on it, warning only
    about standard errors. PROC HAZARD re-expresses the phase before fitting
    (SETG3_ignore_tau(), setg3.c:313-315 and 380-425). tau is held at
    1, and the product is carried by one parameter: gamma, unless gamma is
    the one you fixed. Under constraint = "eta_gamma" (FIXGE2) both are
    held, at gamma = 2 and eta = 1. hazard() now does the same, with mu
    rescaled to match. It warns and records it in fit$fit$boundary
    (mechanism "g3_alpha_one", warning class "hzr_g3_alpha_one", which
    inherits "hzr_boundary").

    The likelihood is unchanged, and a test checks the held fit against
    survival::survreg()'s Weibull. But the coefficients are reported in the
    held parameterisation. A phase such as
    hzr_phase("g3", tau = 5, gamma = 3, alpha = 1, eta = 1, fixed = "shapes")
    now reports tau = 1 and a mu smaller by a factor of 5^3. The hold is
    announced only when it changes a value or what is fixed, and a free
    alpha that starts at 1 is not held, by SAS or here. What PROC HAZARD does
    next is not mirrored: it rewrites a gamma * eta start at or below 2
    (setg3.c:870-921), and R has never done that for any g3 phase.

  • Under constraint = "eta_gamma" (FIXGE2), a fixed alpha above 1 is
    now an error, and a free one started at 1 or above starts at 2/3, as in
    PROC HAZARD (#418).
    With gamma * eta = 2 the g3 form needs
    gamma * eta / alpha > 2, that is alpha < 1. PROC HAZARD stops on a
    fixed alpha above 1 (SETG31040, setg3.c:843-847), and hazard()
    now does too; it used to fit. A free alpha started at or above 1 is
    rewritten to 2/3 (setg3.c:848-850), and hazard() now does the same,
    with a warning and a fit$fit$boundary record (mechanism
    "g3_fixge2_alpha_start"). A fixed alpha of exactly 1 takes the hold
    above. Whether a constrained gamma is heading for infinity is still not
    detected (see Known limitations).

  • A hzr_translate_sas() job PROC HAZARD refuses now warns loudly, and
    says so in $untranslated
    (#359). When SETG3 sets an error, the
    procedure exits in shape() before results(), so the job produces
    nothing. The translation recorded that as an untranslated row and emitted a
    hazard() chunk with nothing to mark it, and a reader who rendered past the
    callout got a converged fit standing in for a job with no result. The
    emitted document now carries a warning() immediately above the fit,
    naming the SETG3 code, its cause and the PARMS operands that produced
    it, and the row is recorded as before. The fit is still emitted: a rendered
    document completes, and the reader is told what it stands in for.

    Some of these still fail further down, and the warning says which.
    Several SETG3 refusals fire precisely because a shape value is out of
    range, and the same value is out of range for hzr_phase(), which will not
    build the phase. Rather than name a list of codes here, which drifted once
    already, the warning itself is derived by trying to construct the phase: it
    says hzr_phase() accepts the shape only when it does, and otherwise says
    the document stops at that check. A test executes every SETG3 class's
    emitted chunks and requires the message and the outcome to agree, so the
    two cannot diverge again. In every case the warning is emitted in its own
    chunk above the fit, naming the SETG3 code and the operand, so the
    cause is stated before hzr_phase() refuses; the render then stops there
    with hzr_phase()'s own message.

    Be aware of where that warning does and does not appear. When a chunk
    errors, Quarto writes no output document, and knitr collects warnings
    into the document rather than printing them, so the warning does not
    reach the render console either. What a reader has in that case is the
    emitted .qmd itself, where the warning() naming SETG3910 sits
    immediately above the failing fit, and the $untranslated row on the
    translated job.

    The refusal is raised only where it is PROC HAZARD's. With FIXGE2 or
    FIXGAE2 and no WEIBULL, SAS reaches SETG3 down a path the setg3.c
    trace does not model, so the trace's verdict is not used there. Only
    SETG3's entry refusals are raised as refusals on that path (see the next
    entry).

  • More hzr_translate_sas() jobs that PROC HAZARD refuses, or fits
    differently, now warn loudly
    (#358, #403, #421). Each was already recorded
    as an untranslated row, but nothing in the rendered document said so, and a
    reader met a converged fit with no sign that PROC HAZARD would not have
    produced it. Each now emits the fit, a warning() above it naming the
    cause, and the row:

    • a PARMS operand PROC HAZARD rejects with a syntax error: a value its
      lexer does not read as a number (NU=1E-3, NU=2.), a value keyword
      with no = NUMBER, a spaced operand that is invalid even joined, or a
      keyword outside its grammar (FIXG1);

    • a MAXITER= or CONDITION= value that its lexer does not read as a
      number, or no value at all: MAXITER '=' NUMBER and
      CONDITION '=' NUMBER (hazard_y.y:63-64) have no form without a
      number, so MAXITER=, MAXITER = and a bare MAXITER are each a
      syntax error and the job does not run;

    • a template's ? placeholder in PARMS, which PROC HAZARD's lexer also
      rejects. It was filled from SAS's default and fitted; it now asks to be
      filled in;

    • a model this translation cannot emit:

      • FIXMNU1 on an active early phase, which PROC HAZARD fits with
        |M*NU| = 1; this translation does not mirror that constraint;
      • DELTA other than 0 on an active early phase;
      • FIXTAU with no TAU written, which PROC HAZARD fixes at 0.75 of the
        longest follow-up;
      • FIXGE2 or FIXGAE2 without WEIBULL. That path is not modelled
        here, so the warning says the translation cannot tell whether PROC
        HAZARD refuses the job or which model it fits;
    • SETG3's entry refusals, on every path;

    • SETG1's refusals for an early phase (#424): THALF fixed at a value
      that is not positive (SETG1910); M and NU both fixed on a case no
      model takes (SETG1940, SETG1950, SETG1960, and SETG1920 and
      SETG1930 under FIXMNU1); and DELTA fixed outside [-1, 1]
      (SETG1900, SETG1901). They were fitted with no row, or warned for the
      wrong reason, that the model was not mirrored. Most of these values are
      out of range for hazard() as well, so the fit still fails after the
      warning;

    • an early phase PROC HAZARD may not fit (#424). With NU=0 and M free,
      SETG1 selects its limiting case, and these jobs fitted with no row.
      What the binary does next was measured on two datasets, and the warning
      says only what was seen. For M=0 NU=0 FIXNU it produced no result on
      both. For M=1 NU=0 and M=-1 NU=0, with or without FIXNU, it stopped
      on a domain error (DLG1980) on one dataset and fitted on the other, so
      the warning says PROC HAZARD may print no estimates on your data. In
      neither case does it say PROC HAZARD fits another model;

    • a phase variable that is not a name to PROC HAZARD's lexer (#440):
      AGE*SEX, LOG(AGE), B SEX, 1AGE. A phase variable must be a NAME,
      [_A-Z][_A-Z0-9]* (hazard_l.l:39, phasevar : NAME at
      hazard_y.y:213), so PROC HAZARD rejects such a job at parse. The phase
      parser passed the text through as a column name, with no row and no
      warning, so EARLY AGE=0.1, AGE*SEX=0.2; emitted a fit on a column
      called AGE*SEX and, under SELECTION, a screen offering it as a
      candidate. The operand is now left out of the model, and out of theta
      with it, so the emitted formula has one starting value per term. This
      holds with and without SELECTION, and for every spacing of the
      operand. A name PROC HAZARD accepts, including _X1 and a word that is
      a keyword elsewhere (E, EARLY), does not warn, and neither does a
      macro reference, which SAS expands before its lexer runs.

      Parentheses follow the lexer rather than a rule of thumb, and the
      verdicts are checked against the HAZARD binary (C-Version 4.4.4). ) is
      whitespace to it (hazard_l.l:32), and ( returns no token but
      switches it to its PROC-line state (hazard_l.l:56). So a last item
      LOG() or LOG() = 0.2 is the variable LOG, which PROC HAZARD fits
      and the translation now keeps. LOG(X), AGE(1), LOG() /I, and every
      item after a ( in the same statement are rejected. A ( also clears
      PROC HAZARD's syntax-error flag, so a job whose phase statements carry
      one may run despite an earlier error: the binary runs
      EARLY AGE*SEX, LOG(); and fits AGE alone. The translation does not
      reproduce which variables survive, and says so in the warning.

    • a v...

Read more

TemporalHazard 1.2.9

Choose a tag to compare

@ehrlinger ehrlinger released this 05 Sep 18:25
fe57e60

Bug fixes

  • The phase-identifiability warning no longer names a cause that did not
    occur.
    variation is the relative range of a phase's contribution across
    the observed times, so it collapses for every phase when those times carry
    nothing that separates them. The saturated scan fired anyway: with
    time = rep(2, 20) it told a constant phase that its shape parameters were
    unidentified, when a constant phase has none, and blamed a short half-life.
    A single-row fit produced the same text.

    That case is now reported in its own words, under a condition that takes two
    things together: the observed times' own relative range is below the
    tolerance, and no phase's contribution varies across them. Neither half is
    sufficient. The range alone would condemn times bunched far from the origin,
    where a steep phase can be fully identified; the phase measures alone would
    condemn well-spread times whenever a single phase saturates. Because the
    range is relative, near-ties reach the condition as exact ties do --
    c(100, 100.000001) produced the old wording verbatim. Per-phase saturation
    and absence over well-spread times are reported exactly as before, and the
    condition is now caught even when every phase carries covariates, where the
    per-phase measures are withheld and previously nothing was said at all.

    Two related corrections. A phase that has not started is reported as absent
    even in a left-truncated fit, since that test rests on the share and not on
    how the times are spread. And when the counting-process entry times or
    interval bounds supply enough evaluation points, the degenerate-times verdict
    is withheld -- and "enough" is counted rather than assumed. With the event
    times tied, each added point supplies one more functional of the parameters,
    so one more than the number of added points must reach the number of free
    parameters. The package's default two-phase model has five, and pinning a
    phase's shapes lowers the bar because those parameters are no longer free.
    Points that add nothing do not count: a zero entry time, a bound equal to an
    event time -- both of which Surv(type = "interval") synthesises for every
    row -- or a bound the objective never reads, since time_upper is live only
    for left-censored and interval rows. Counting any of those silenced this
    warning on data that needed it while changing no likelihood value.

    The message also no longer names shape parameters for a model that has none.
    A single constant phase is now silent, since one functional determines its
    mu exactly; two constant phases still warn, since only their sum is
    determined, but the wording says the phases cannot be told apart rather than
    naming shapes they do not have.

    Where the times are NOT degenerate, the per-phase saturation message is
    reported regardless of the bounds, so adding a start column no longer
    removes a correct diagnostic (#211).

    Known limitation, unchanged by this work and stated rather than implicit: the
    per-phase measures are taken over the event times alone, and the count of
    evaluation points is deliberately conservative -- it does not credit the
    extra functional a zero entry time makes separately observable, so a fit can
    be warned about while being identified. That cuts both ways.
    A phase can be flat across the event times while its shape is identified
    through the bounds, so the saturated message can overstate what is lost; and
    the counting rule above is a local identification argument for tied event
    times, so a fit can clear it and still have a flat direction the guard does
    not see, in which case nothing is said at all. Tracked in #228.

  • The SAS nomogram parser no longer deletes a whole-year time column. The
    PROC PRINT observation counter is identified as a leading column running
    exactly 1..n, which rested on no measurement column ever being a gapless
    1-based integer run -- true of the corpus in hand, not of SAS listings. A
    counter-suppressed nomogram on a whole-year grid prints YEARS as 1.0000, 2.0000, 3.0000, and the parser deleted the time key. Because the remaining
    columns then matched the header count, the shape check that would have caught
    it passed: the listing parsed, and the comparison ran against a table with no
    time column.

    The header now decides: a leading column the header names as a counter is
    dropped whatever values it carries, which also fixes the reverse case -- page
    two of a paginated listing starts at 41, and requiring a 1..n run left that
    counter in the data. The labels are matched case-insensitively, after the
    leading underscore the parser strips, so SAS's _N_ is recognised. A 1..n
    run under a name the parser does not know is now kept and warned about rather
    than silently dropped: the listing cannot distinguish a counter spelled a new
    way from a measurement that happens to run 1..n, and an extra column is
    visible where a deleted one is not (#212).

  • A phase named total is now rejected rather than silently losing its
    contribution.
    total is the key the multiphase cumulative-hazard
    accumulator is stored under, so phases = list(total = hzr_phase("cdf"), ...)
    overwrote that phase's own contribution vector with the summed hazard. The fit
    returned normally; the phase then reported a contribution share of exactly 1
    and the identifiability diagnostic read it as dominating the model. The name is
    reserved across the decomposition and prediction output as well, where it
    labels the total column. hazard() and hzr_theta_names() -- which applies
    the same validation -- now stop with an explanatory message, and the
    reservation is documented on the phases argument (#214).

  • objective = "sas" data defects are reported as data defects. The two
    conditions the SAS objective imposes -- no left-censored rows, and a positive
    width on every interval-censored row -- are pure functions of the data, but
    were checked inside the objective. The optimizer's per-start handler caught
    them and reported "produced no usable fit from N starts", which reads as a
    convergence problem and invites raising n_starts: a remedy that cannot work,
    because the condition is identical at every start. hazard() now checks both
    before any optimization, and the interval-width message reports the offending
    row in the data rather than its position among the interval rows. The guards
    inside the objective and gradient are unchanged, since the gradient is
    reachable without hazard() (#213).

  • Behaviour change: because those preconditions are properties of the data
    and not of the fit, they are now checked whenever objective = "sas" is
    specified, including fit = FALSE. hazard(..., fit = FALSE, objective = "sas") on left-censored or zero-width-interval data previously returned a
    hazard object carrying an objective it could never be fitted with; it now
    stops. An NA in status under objective = "sas" is also reported by
    argument and row, rather than as a bare missing value where TRUE/FALSE needed.

    Note this reaches only the codes hazard() is given. On the vector
    interface a survival::Surv() object is unclassed without translating its
    codes, so they mean something else on arrival, and what goes wrong depends on
    type: under type = "left" a left-censored row is coded 0 and is fitted
    as right-censored; under type = "interval" it is coded 2 and is read as
    an interval of zero width, which this guard then rejects; and a genuine
    interval row, coded 3, matches no branch of the likelihood and contributes
    nothing. The formula interface translates and is guarded. That asymmetry is a
    pre-existing defect of the vector path, tracked in #226.

TemporalHazard 1.2.8

Choose a tag to compare

@ehrlinger ehrlinger released this 02 Sep 20:11
f790c73

New features

  • New hzr_theta_names() returns the names of a multiphase theta
    vector, in the order theta requires, before any fit runs. Use it to check
    a hand-written starting vector against the specification it belongs to:

    phases <- list(early = hzr_phase("cdf"), late = hzr_phase("g3"))
    stopifnot(length(theta0) == length(hzr_theta_names(phases)))
    setNames(theta0, hzr_theta_names(phases))

    theta is positional and its entries are not on a common scale -- the late
    phase logs mu and tau but carries gamma, alpha and eta naturally --
    and wrapping the wrong element in log() produces a fit, not an error.
    A comment describing the order is therefore not enough for a template that
    ships to many studies, which is what prompted this: the order is a property
    of the phase specification and changes the moment an author adds, removes or
    retypes a phase.

    The function is a thin wrapper over the naming the optimizer itself uses,
    not a second implementation of it. hazard()'s optimizer, the score test's
    re-expansion and this function now all go through one internal helper, so
    the order documented here cannot drift from the order a fit produces --- a
    test asserts the two are identical for three different phase
    specifications. Phase validation is shared too, so unnamed phases get the
    same phase_1 / phase_2 labels a fit will give them.

  • hzr_stepwise()'s $steps frame gains a stat_type column, saying
    what the stat on that row is and so which reference distribution
    recomputes its p-value: "score_q" (chi-square on df), "wald_z"
    (standard normal) or "wald_chisq" (chi-square on df).

    df could not tell these apart. A scalar Wald is reported as a z, not as
    its square, so it and a score Q are both recorded at df = 1 while calling
    for different distributions. The gap was widest on a candidate rescued by
    the Wald fallback added in 1.2.7: that row carries a Wald z under
    criterion = "score", where every neighbouring row carries a Q. The
    selection was right and p_value was right, but a reader recomputing a
    p-value from stat the way the neighbouring rows permit got an answer wrong
    by dozens of orders of magnitude. Under criterion = "score" the entry
    rows reading "wald_z" are the ones the fallback rescued --- filter on
    action == "enter" & stat_type == "wald_z". Drop rows are always
    Wald-tested under that criterion, following SAS, so they read "wald_z"
    whether or not the fallback fired.

  • hzr_bootstrap() reports Wald fallbacks in select mode, through
    $n_wald_fallback_replicates and $n_wald_fallbacks, with a warning when
    either is non-zero. Every replicate runs under suppressWarnings(), so a
    run in which the fallback fired throughout previously reported nothing at
    all -- and the bootstrap is where a wholesale substitution matters most,
    since those entries drive the pooled selection frequencies.

  • The nomogram parser accepts lines = as well as path =. A multi-fit
    listing needs each nomogram attributed to the fit whose block contains it,
    not to the file. Previously a caller who had already split the listing by
    fit had to write each block back out to a temporary file to parse it, which
    is a workaround rather than an interface. Passing the lines directly now
    works, and is what makes the multiple-nomogram warning actionable.

Bug fixes

  • objective is now recorded on the fit, so refits keep the estimand.
    hazard(objective = "sas") was accepted and acted on by the optimizer, but
    the choice was never stored on the object. Everything that rebuilds a
    hazard() call from fit$spec therefore reverted to
    objective = "likelihood": every hzr_stepwise() candidate refit, the
    Wald-fallback refit and each accepted move, plus the score test's numeric
    Hessian and gradient.

    Selecting on a "sas" base fit consequently differenced delta_logLik,
    aic and delta_aic across two different estimands, and the score
    statistic was computed against a likelihood the fit had not been fitted to.
    On the esophagectomy reference the two objectives differ by about 22
    log-likelihood units -- larger than most single-variable effects -- and the
    run produced a full $steps table with no error and no warning.

    fit$spec$objective now records it, and one accessor feeds every consumer
    so they cannot drift apart. A fit that carries no objective predates the
    argument and is read as "likelihood", which is what it was.

    Note the asymmetry this had created: hzr_bootstrap() refit mode
    re-evaluates the stored call, which did carry objective = "sas", so it was
    already consistent; select mode goes through the scope refit and was not.

    Following from this, objective cannot be overridden per refit. Passing it
    through hzr_stepwise()'s ... is accepted when it restates the base fit's
    own objective and refused when it conflicts, with a message saying why:
    a candidate refit under a different objective would make delta_logLik,
    aic and delta_aic differences between two estimands rather than between
    two models. It is refused rather than quietly ignored, because a full
    $steps table that silently disregarded an explicit argument is the failure
    this entry is about.

  • A candidate that neither criterion could test is now reported as such.
    criterion = "score" declines a candidate whose observed information is
    indefinite at beta = 0 -- which happens when the effect is large -- and
    then refits it and tests it by Wald instead. That rescue can converge,
    producing a perfectly good point estimate, while its Hessian is singular:
    no standard error, so the Wald test cannot be computed either.

    The fallback dropped that case silently. The row kept the score's reason,
    information_indefinite, which describes the first of two independent
    failures and says nothing about the second, and no counter moved. A
    strongly predictive variable vanished and the screen rendered as an honest
    "nothing met slentry" -- the same failure this package has twice fixed
    elsewhere, where a clean-looking screen and an honest null result cannot be
    told apart.

    Such rows now report fallback_no_variance, distinct from
    information_indefinite (the rescue errored or did not converge, and is
    listed in $criteria$refit_failures). hzr_stepwise() and
    hzr_bootstrap() both warn on either, saying the candidate was tested by
    neither criterion and which mechanism applied. The
    information_indefinite prose claimed "that refit also failed", which was
    wrong for the new case and sent readers to an empty refit_failures; both
    it and the bootstrap warning now describe the two mechanisms separately.

    No behaviour changed in what gets selected: a candidate that could not be
    tested is still not entered. What changed is that the run says so.

  • fit$fit$weak now distinguishes "no ridge" from "not checked". The
    weak-identification diagnostic introduced in 1.2.7 returned NULL both when
    a fit had been examined and found well identified and when it could not be
    examined at all -- most importantly when no Hessian was available, which
    happens on an install without the suggested numDeriv package and on fits whose
    rows are left- or interval-censored, where the analytic Hessian declines by
    design. NEWS offered fit$fit$weak as the programmatic check, so on those
    installs it certified as well identified a fit nothing had looked at.

    The field now takes three values: a list when a ridge was found, NULL when
    the fit was examined and is well identified, and NA when the check could
    not run. Test it with is.list(fit$fit$weak) rather than !is.null().
    summary() prints a note in the NA case saying the check did not run, and
    a fit imported with hzr_read_outhaz() -- which has no R Hessian to examine
    -- now reports NA rather than reading as certified clean.

  • A fit with more than one flat direction says so. The detector reported a
    single direction, which invited reading every parameter it did not name as
    identified. It now reports n_directions, the number of distinct
    near-flat directions found, and the warning says when there is more than
    one. Distinctness is counted over the parameters spanning each direction,
    not over eigenvectors: a ridge between two parameters clears the gate twice,
    once on the flat direction and once on its stiff partner, so counting
    eigenvectors would report one ridge as two.

  • A rescued candidate that goes on to win is no longer fitted twice. The
    Wald fallback refits each candidate it rescues, and the acceptance step then
    refit the winner again with identical arguments. The two fits were
    bit-identical, so this was cost rather than incorrectness -- but it doubled
    the price of every accepted fallback entry, against a criterion whose whole
    advantage is that it does not refit per candidate. The rescuing fit is now
    kept and reused, as the Wald path already did with its candidate fits.

  • DELTA is not implemented, and two comments said it was absorbed. The
    headers of R/decomposition.R and R/argument_mapping.R both stated that
    the C DELTA parameter's time transformation
    B(t) = (exp(delta * t) - 1) / delta is "absorbed by decompos()". It is
    not absorbed; it is unimplemented, and delta = 0 is assumed. DELTA
    enters the C reference in three separate places -- it builds rho from
    B(t_half) rather than t_half, it replaces the time argument with
    B(t), and delta * t enters the log-density additively so the density
    carries a factor of exp(delta * t) -- and R computes the delta = 0
    branch of all three.

    The comment was the harmful part. It made the omission look deliberate and
    safe, so a reader looking for exactly this discrepancy was told to stop
    looking, while ...

Read more

TemporalHazard 1.1.0

Choose a tag to compare

@ehrlinger ehrlinger released this 12 Jun 19:10
289346b

On CRAN: https://CRAN.R-project.org/package=TemporalHazard (published 2026-06-12) · DOI 10.32614/CRAN.package.TemporalHazard

install.packages("TemporalHazard")

Highlights

  • Analytic Hessians for all five families (exponential, Weibull, log-logistic, log-normal, multiphase); validated against numDeriv.
  • Full-information variance for Conservation-of-Events fits — conserved-phase SE recovered from the unconstrained Hessian at the optimum; every hz.death.AVC SE now matches SAS HAZARD.
  • predict.hazard() improvements — type = "hazard" for multiphase, conf.type survival CL transform (log-log / logit-for-HAZPRED), per-phase decompose + se.fit confidence limits.
  • hzr_deciles() matches the SAS deciles.hazard macro exactly.
  • SAS HAZPRED parity fixtures (Group A): patient-specific predictions, stratified calibration, KM/Nelson-Aalen life tables, decile-of-risk tables.
  • Docs — package overview page, rfsrc-style math in core help files, baseline-distribution + stepwise option explanations; pkgdown math now renders.
  • Robustness — Hessian-inversion hardening, parse-tree multiphase formula preprocessor, hzr_bootstrap() weighted-path fix.

See NEWS.md for the full list.

TemporalHazard 1.0.3

Choose a tag to compare

@ehrlinger ehrlinger released this 30 May 15:22
5525158

Bug fixes / CRAN compliance

  • hzr_bootstrap() no longer touches .GlobalEnv directly. The 1.0.2 oldseed/on.exit()/assign(".Random.seed", ...) save-restore wrapper violated CRAN policy and has been removed. When seed is supplied the function now simply calls set.seed(seed); @param seed documents that the caller's RNG state is not restored on exit.

Other changes

  • All seven vignettes received audience-grounding prose and section-level lead-ins (no naked code blocks under headers). No code, math, or numeric results were touched.
  • inst/CITATION now templates the version via meta$Version so citation("TemporalHazard") self-reports the current version.
  • RoxygenNote synced to 8.0.0 to match the installed roxygen2.

Installation

install.packages("TemporalHazard")