Repository navigation
TemporalHazard 1.2.13
Breaking changes
-
hzr_bootstrap(seed = )no longer resets your random number stream
(#611). It seeded the global generator withset.seed()and left it
there, so a script that calledset.seed()once at the top lost that seed
at its first seeded bootstrap, and every draw after it followed from the
bootstrap's seed instead. The seed now holds for the call only, through
withr::local_seed(), and your stream is restored when the call returns or
fails. A draw afterhzr_bootstrap(seed = k)no longer follows from
k: a script that relied on that must callset.seed(k)itself. The
replicates for a givenseedare unchanged, and equal those of
set.seed(k)followed byseed = NULL.seedmust now beNULLor a
single whole number:1.7,c(1, 2)andTRUEused to act silently as
seed = 1, andNAfailed with a message that did not nameseed.
withrmoves fromSuggeststoImports; it depends only on base R.
This restores the 1.0.2 behaviour, which 1.0.3 removed. -
Estimates and selections from earlier versions may change when
refitted. Several fixes change what a fit or a screen returns, not only
what it reports: multiphase fits under Conservation of Events, which could
stop short of the maximum (#565); fits with a"hazard"phase far past
saturation (#578); multiphase fits whose search passed a very small
t_half(#574); multiphase stepwise selections andhzr_bootstrap()
select-mode frequencies, whose candidate refits never started from the
current model (#551); score-criterion selections at a step whose
information cannot be formed, which now refit and test their candidates
instead of stopping (#570); Wald selections, which no longer enter a
candidate whose refit ended below the base (#538); and multiphase AIC
selections over a candidate with missing values (#488). Refit models and
rerun screens from earlier versions; the entries under Bug fixes give the
details. -
Some calls that returned a value now stop, or return something
different.hazard()refuses acontrol$maxitbelow 1 or not a number ("200"
was accepted) (#541), andhazard(),hzr_evaluate()andpredict()
refuse a Weibullmuornutoo small to hold as a normal double,
such as1e-310, which 1.2.12 fitted and evaluated (#566).summary()$log_likisNAfor anobjective = "sas"fit that read
an interval-censored row, whose objective is not a log-likelihood
(#544).hzr_decompos()returns a fourth element,log_surv(#578).hzr_stepwise()refuses adatawhose rows are not the fit's rows in
the fit's order (#487, #515), and anydatawith a duplicated column
name, whether or not that column is in the scope (#515).hzr_stepwise()refuses a base model that did not converge under
criterion = "aic", and under"wald"withdirection = "backward"
or"both"; select-modehzr_bootstrap()refuses such a base up
front, and fails a replicate whose base refit did not converge (#627).hzr_deciles()refuses a multiphase fit whose design dropped rows, as
hzr_gof()already did (#627).
-
Some returned objects have new elements, and one moves others. A
Weibull fit'sfit$fitgainslog_scalebeforevcov, sovcov,
rcond,pd,counts,message,rel_gradient,
rel_gradient_reasonandpolish_codemove position: read them by name
(#566).hzr_decompos()returnslog_surv(#578); a stepwise result's
$steps, andas.data.frame()of it, gainn_rows(#519);
attr(hzr_gof(), "summary")gainsn_tallied(#492);hzr_evaluate()
returns$objective, andsummary()gainsobjectiveand
objective_value(#544, #556).
Bug fixes
-
Standard errors computed numerically could be far too small, with no
warning, when a covariate was far from zero (#598). Fits with left- or
interval-censored rows, which have no analytic Hessian, take theirs
fromnumDeriv::hessian(), as does the score criterion of
hzr_stepwise(). Its default step is a tenth of each parameter's size,
and an intercept that absorbs a covariate's offset is large, so the step
spanned many standard errors. With the covariate offset by 100, a fit
reached the same log-likelihood as the centred fit and reported SE(beta)
at 0.035 (Weibull), 0.047 (exponential), 0.073 (log-logistic) and 0.52
(lognormal) of the centred fit's; at an offset of 1000, an exponential
SE was 1e-28 of it, a Weibull fit had no covariance, and a lognormal
fit's standard errors wereNA.- The step is now set from the curvature: 0.1 standard error along each
direction of the Hessian, refined twice, with each direction's step
capped atnumDeriv's own first step in every parameter. The same fits
now agree with the centred fit's standard errors at offsets of 100
and 1000. - A Hessian that is not positive definite is still reported as found.
- The remaining limit is double precision: at an offset of 1e4 the
exact Hessian's reciprocal condition number is about 1e-16, so no
Hessian on that scale can be inverted reliably. Centring the
covariates avoids it.
- The step is now set from the curvature: 0.1 standard error along each
-
A single-distribution fit (exponential, Weibull, lognormal or
loglogistic) could stop at a false maximum, far below the best
log-likelihood, and reportconverged = TRUEwith no warning (#531).
On theavcdata an exponential fit with~ age + malfrom an ordinary
start stopped at a log-likelihood of -80896.5 against a best of -394.8.
hazard()now warns, with class"hzr_possible_false_maximum", when such
a fit reports convergence with a relative gradient above 1e-3 and the
continuation did not stop on its iteration limit or an unbounded direction
(nlm()codes 4 and 5, which already warn). The warning says the fit may not
be a maximum, names the relative gradient and suggests other starting
values, or centring or rescaling the covariates.convergedis not
changed, and nothing is refused.hzr_bootstrap(), whose replicates run
with their warnings suppressed, counts the replicates in which any fit
meets the same rule (the base refit and every stepwise refit, not only
the final fit) and warns once; their estimates are still pooled. The threshold sits
above every good fit in the test suite (the worst at 6.8e-4) and below
the false maxima found, but it does not separate every case. A good fit
on badly scaled covariates (one multiplied by 1e4 or more) can exceed it,
with a relative gradient of 1e-3 to 5e-3 and a log-likelihood up to 0.02
short; there another start does not help, and centring or rescaling
does. And a stuck fit can fall below it: an intercept-only Weibull on
times near 1e-170, started at a shape of 1.5, stops there against 1.72
with a relative gradient of about 5e-4 and raises nothing. The absence of
the warning is not a guarantee. Multiphase fits are not flagged. -
predict(se.fit = TRUE)could return wrong standard errors, including
exactly 0, with no warning, when the fit's covariance was incomplete or not
positive definite (#586).- When inverting the Hessian finds a variance that is not positive, the
fit stores it asNA, the same mark a fixed parameter carries.
predict()dropped such a parameter as if it were known. On a Weibull
fit withmu's variance masked, survival standard errors of 0.0175,
0.0345 and 0.0392 came back as 0.0221, 0.0221 and 0.0524. AnNA
variance now counts as fixed only where the fit'sfixed_masksays
so. Otherwise, a prediction that uses the parameter returnsNA
standard errors and limits, with a warning naming it. - A covariance with a positive diagonal can still be indefinite, which
happens when the Hessian was not at a proper maximum. A negative
quadratic form was then clamped to anse.fitof 0 with a zero-width
interval, and a positive one was reported although it is not a
variance. The covariance of the parameters a prediction uses is now
checked for positive definiteness, and an infinite covariance among
them is refused. - Parameters the prediction does not use, by its form, are left out of
that check and of the sandwich. An infinite covariance between two of
them no longer turnsse.fitintoNaN(#587). - When Conservation of Events could not recompute the conserved phase's
variance,predict()keeps its standard errors but now warns that
they leave that variance out and may be understated.
- When inverting the Hessian finds a variance that is not positive, the
-
After rows at time 0 are dropped, messages are clearer (#484).
- Row numbers in later error messages refer to the rows as given, not
to their positions among the rows that remain: anNAstatus on row
10 is reported as row 10, where it used to be reported as row 9. - A warning raised by a term of the formula, such as one a function in
it raises, was shown twice, because the formula is evaluated again on
the rows that remain. Only a warning that evaluation raises anew is
shown now. hzr_stepwise()givendatawith exactly the fit's rows plus the
ones it dropped, which it could not confirm as the fit's data, now
warns once that those rows were dropped and should be dropped from
data. Before, it reported only a mismatched row count.hzr_stepwise()'s note that it could not check the row order of
datasaid, for a formula fit givendataof another length, that
the stored data frame "has 116 rows, not the fit's 116". It now
comparesdatawith the stored data frame, as the check does.
- Row numbers in later error messages refer to the rows as given, not
-
Multiphase fits under Conservation of Events could stop short of the
maximum and reportconverged = TRUE(#565). Multiphase estimates from
earlier versions, fitted withconserve = TRUE(the default), may be
wrong. Under Conservation of Events one phase's scale is re-solved from
the other parameters at every step. The optimizer was given the score at
that solved scale without the term for how the scale moves with the other
parameters, so it followed a direction that was not the gradient of the
log-likelihood it was maximizing. On theavcdata, a two-phase model
with six covariates stopped at a log-likelihood of -184.462 where SAS
reaches -182.659. The optimizer now receives the full gradient, and that
fit reaches -182.659 from the default start. Refit any multiphase model
fitted with Conservation of Events;conserve = FALSEfits are not
affected. On that fit the relative-gradient test recorded in
fit$fit$rel_gradienthad failed (0.311 against a limit of about 6e-6),
with no warning.
A fit can now run towards the point where the conserved phase leaves
the model, where before it stopped well short, and it can still stop
beside that point with the gradient test not met. When the conserved
phase's share of the cumulative hazard falls below
control$phase_share_tol, the identifiability warning reports the phase
as it did, and the fit also carries a"coe_phase_vanished"record in
fit$fit$boundary, with that share. The record raises no second
warning. -
In a two-way
hzr_stepwise()screen, a variable frozen bymax_move
is now frozen in the state it ends the iteration in (#580). A variable
that reachedmax_moveby entering was frozen at once, but the
iteration's protected variables had been fixed before its forward step,
so the backward step that followed could still drop it: the trace read
FROZENthenDROP, and$scope$frozennamed a variable the final
model did not contain, with no warning. It happened whenever the freeze
fell on an entry, which for a variable that enters and is dropped every
iteration is every evenmax_move, the default 4 included. The freeze
now takes effect at the end of the iteration, so such a variable is
frozen out and$scope$frozenagrees with the final model. The final
model is unchanged. The "Known limitation (the frozen set)" section of
?hzr_stepwiseis replaced by one describing the rule. PROC HAZARD's
MOVE rule differs again (outsideNOSTEPWISEit counts only exits, so
it freezes a variable only as it leaves the model); that is recorded,
not adopted. -
hzr_stepwise(criterion = "score")now refits and Wald-tests two more
kinds of candidate it could not score (#570). The score criterion refits
a candidate whose score test breaks down and tests it by Wald, but only for
two reasons. Two others were declined untested:nuisance_singular, where
the current model's information matrix cannot be formed or inverted, and
information_nonpositive, where the candidate's own observed information
at zero is not positive. Neither says the candidate is unusable, and a
refit tests it. The first is a fault of the score test at the current fit
and applies to every candidate at that step, so the screen stopped having
tested nothing; with the second, a weaker variable could enter in the
untested one's place. It is reached by ordinary fits. Onavcwith a
Weibull base and the scope~ age + mal + com_iv, the score screen
enteredcom_ivand then stopped, wherecriterion = "wald"enters
com_iv,malandage: withcom_ivin the model, the numeric
information the score test needs could not be formed, though the fit
itself was sound. The score screen now reaches the Wald selection, with
four refits to Wald's six. Inhzr_bootstrap()select mode on the same
base (five replicates, scope~ age + mal), four replicates stopped
untested andageandmalwere selected in one and three; none stops
now, and both are selected in all five.$criteria$n_wald_fallbacks
counts these refits as before. A step whose information cannot be formed
now refits every candidate it would have scored, which is what
criterion = "wald"does at every step. A candidate that is constant,
non-numeric or not a single column is still declined without a refit. The
warning for a candidate tested by neither criterion now covers every
reason that is refitted, inhzr_stepwise()and inhzr_bootstrap(). -
A fit whose covariance mostly failed is no longer reported as examined
for a weakly identified direction (#570).fit$fit$weakisNULLwhen
the check ran and found no ridge, andNAwhen it could not run. When an
ill-conditioned fit had a finite variance for fewer than two of its
parameters, there was nothing to examine, yetweakwasNULLand
weak_direction_checkwas missing fromfit$degraded. It is nowNA,
with the cause recorded. -
The multiphase score was wrong for an early or late phase with a very
smallt_half(#574). Fits whose search passed through such a point
may have been steered wrongly. The derivative with respect to
log_t_halfis taken by finite differences, and the step had a fixed
lower limit. Belowt_half = 1e-4the step no longer shrank with
t_half, and below about6e-10it was larger thant_halfitself. At
log_t_half = -23.28withnu = 0.2931andm = 120, the derivative
came back at 0.27 of its value. The error was under 1e-5 fort_half
down to about1e-7(log_t_half = -16), 0.05% atlog_t_half = -18
and 3% at-20. The derivative is now differenced inlog_t_half
itself, so the step keeps its proportion tot_half. Estimates with a
fittedt_halfof1e-4or more are unaffected beyond the optimizer's
own tolerance. At_halftoo small to step at all (below about
3e-321) now givesNaNfor this derivative, where it gave a number.
A"hazard"phase far past saturation was still wrong after this fix,
because its value was; that is #578, below. -
A
"hazard"phase far past saturation had the wrong cumulative hazard
and hazard (#578). Fits with such a phase from earlier versions may be
wrong. The phase's cumulative hazard,-log(1 - G(t)), was formed from
1 - G. AsGneared 1 that lost its digits, and onceGrounded to 1
it was clamped: the cumulative hazard came back as 708.396 (the log of
the smallest double) and the hazard,g / (1 - G), near1e290, where
both are of order 1 to 100. Atnu = 1,m = 1,t_half = exp(-40)and
t = 1the cumulative hazard is 40 and the hazard 1. The derivative
with respect tolog_t_halfwas noisy from aboutt_half = exp(-30)and
exactly 0 fromexp(-34)at that shape.hzr_decompos()now returns a
fourth element,log_surv, equal tolog(1 - G(t))but computed from
each case's own log-scale terms, and the"hazard"phase's cumulative
hazard and hazard are formed from it. Where1 - Gcan be formed, the
values agree with the old ones to rounding.G,gandhkeep their
names and meaning. The hazardhof any phase type is also formed
without a cancellation that lost it for anuvery close to 0 (at
nu = -1e-18,t = 2,t_half = 1,m = 1it was 0 or 1 where it is
about 5e17); fitted values ofnunear1e-16occur. Fornu = 0and
mbelow about -52,Gwas 0 at every time, includingt_halfwhere it
is 1/2, because1 - 2^mrounded to 1; it is now computed without that
rounding.The clamp could also create an optimum. The hazard near
1e290added
about 668 to the log-likelihood per event, and a fit could converge on
that: one fit in this package's tests reported a log-likelihood of
+60479 with 120 events, with its hazard phase below the first observed
time, and was the one fit here that theunbounded_phaserecord (#444)
caught. It now ends inside its data at an ordinary log-likelihood. A
multiphase fit with a"hazard"phase that reported a positive or
implausibly large log-likelihood in an earlier version should be refitted.
Theunbounded_phaserecord now reports the phase's remaining mass,
1 - G(t_min), fromlog_surv, so a small mass is no longer printed as
0; where the mass itself underflows, its log is printed. Its third
sentence no longer states that the objective is unbounded above. -
A Wald stepwise entry no longer tests a refit that ended below the
current model (#538). The AIC fix below (#490) left two paths open.
Undercriterion = "wald", and in the Wald test thatcriterion = "score"falls back on, such a refit still got a Wald p-value, from a fit
that did not converge. At a strictslentryit was rejected as if tested,
with every counter at 0. Onavc,opmosin the constant phase refit
11.2 log-likelihood units below its base, got p = 0.069, and at
slentry = 0.05was rejected with no warning. The defaultslentry = 0.30
entered it, and the existing check on an entered model warned. Such a
candidate is now refused under the reasonloglik_below_base, as under
AIC, andhzr_stepwise()warns that it declined it without testing it.
It is not listed in$criteria$wald_untested_entries, which is for
entries with no variance. -
hazard()now refuses acontrol$maxitbelow 1 (#541). It was
accepted without a word, and what happened depended on the model. A
Weibull fit or a single-phase multiphase fit returned its starting values
withconverged = TRUE: on theavcdata a Weibull fit withmaxit = 0
reported a log-likelihood of -1425.16 as converged, where the fit reaches
-223.55. A multiphase fit with acdfand aconstantphase ignored the
limit and optimised anyway.control$maxitmust now be a single finite
number of at least 1 (a fraction is truncated, as PROC HAZARD truncates
MAXITER), and anything else stopshazard(), whether or not
it fits, with a pointer tohzr_evaluate()for an evaluation at given
values.hzr_stepwise()checks it once, before any refit. -
print(),summary()and thehzr_stepwise()trace no longer call an
objective = "sas"fit's objective a log-likelihood (#544, #556). Where
the fit reads an interval-censored row (one of positive weight, not
dropped by a phase design), the objective anobjective = "sas"fit
reaches is PROC HAZARD's interval-mean-hazard objective, not a
log-likelihood.print()showed it aslog-lik:,summary()returned it
aslog_lik, and the stepwise trace's final line aslogLik = ..., so a
reader comparing it with another model's log-likelihood, or taking an LR
or AIC from it, got a finite and plausible wrong number. On one fit the
printedlog-lik:was -160.02, where the log-likelihood at the same
estimates is -118.94. Such a fit now printsSAS objective:, and
summary()'slog_likisNA, with the value in the new
objective_valueand its kind inobjective. The stepwise trace reads
SAS objective = ..., AIC from it = ..., and a screen withcriterion = "aic"warns (classhzr_stepwise_sas_objective) that each entry is
decided on-2 * (SAS objective) + 2k, which is not an AIC, and each
removal on a Wald statistic from the SAS objective's curvature. Without
interval-censored rows
the two objectives agree, and nothing changes. The newsummary()fields
come last, so no existing element moves.logLik()andAIC()have
no method for ahazardfit and still stop. -
A Weibull fit whose scale cannot be represented now says so, and is
read throughlog(mu)(#566). The Weibull scalemuis the baseline at
x = 0, so with a covariate far from zero, such as a calendar year, it
can leave the range a number can hold at a sound fit. It was then reported
asInf, as 0, or as a value so small that most of its digits were lost.
WithInf,predict()returned survival 0 and cumulative hazardInf
with no warning, where the same model with the covariate centered gave
0.217, andhzr_gof()reported an expected count ofInf.hazard()now warns, with class"hzr_unrepresentable_scale", and
says to centre or rescale the covariates to reportmuitself. The
fit still reportsconverged = TRUE: it is sound, and only its scale
cannot be reported.- The fit keeps
log(mu), which always holds a number, with its
standard error.predict(), and through ithzr_gof()and
hzr_deciles(), read it, and agree with the centered fit of the same
model.summary()shows it as alog(mu)row undermu.coef()
showsmuas it is, andvcov()givesNAfor its row and column,
with the reason in the fit'sdegraded_causes. hzr_evaluate()andpredict()refuse athetagiven with such a
mu, which carries nolog(mu)to read. They already refused amu
of 0 orInf; a subnormalmu(one too small to hold as a normal
double, such as1e-310) was evaluated, and is now refused too. This
includes an intercept-only fit, whose parameterspredict()did not
check before.hzr_bootstrap()warns with the number of replicates whosemu
cannot be represented; the summary ofmuis not usable there.- A finite
muwhose product with a large time overflowed gave a
cumulative hazard ofInffrompredict().predict()now computes
it on the log scale. - Where
muis finite but its variance overflows or underflows,
predict(se.fit = TRUE)returned a wrong standard error: 187 times too
large in one case measured, 75% too large in another. It now reads the
variance oflog(mu), an ordinary number, and agrees with the
centered fit.hazard()warns about such a fit too, with the same
class, andvcov()givesNAformu, because its variance there is
Inf, 0 or too small. - The same screen applies to any parameter's variance, and a negative
variance is now named as one:predict(se.fit = TRUE)had reported a
standard error of 0 for it. Underdecompose = TRUEeach phase is
screened on its own parameters, so a bad variance in one phase
withholds that phase's and the total's standard errors, not the
others'. Before, a phase with an infinite variance got a standard
error of 0, and the total the other phases' alone, with no warning. - The Weibull likelihood is computed on the log scale. Formed directly,
mu^nu,(mu * t)^nuandt^(nu - 1)underflow or overflow where the
log-likelihood is an ordinary number. On a left-censored row whose
cumulative hazard was below the smallest normal double,
hzr_evaluate()returned -743.341 with no warning, against -743.265.
An event at a time of 1e-180 gave-Inf, against -137.056. A
zero-weight event row whosemu^nufell below what a double holds
turned the whole log-likelihood into-Inf. Wheremu * tormu^nu
fell below the smallest normal double,hzr_evaluate()returned a
log-likelihood that was not the model's (23084.15 where the fit's own
was 22972.56); it now computes it.
-
Multiphase stepwise refits now start from the model they extend
(#551). Multiphase selections, andhzr_bootstrap()select-mode
frequencies, from earlier versions may be wrong. Each candidate refit
inhzr_stepwise()started from the phase specifications' default
values, not from the current model's estimates. The candidate model
contains the current one, so from the current estimates, with the new
coefficient at 0, a refit cannot end below the current log-likelihood.
From the default start it often did, while reportingconverged = TRUE.
Onavcwith the SAS reference base model (early and constant phases,
Conservation of Events), five of nine first-step candidates ended below
the base,agein the early phase by 25.1 log-likelihood units. Every
multiphase entry, whether scored by Wald, AIC or the score criterion's
Wald fallback, was therefore tested against a model not at its optimum.
An accepted step then carried that model forward. Each multiphase refit
is now fitted twice and the higher log-likelihood kept: once from the
current estimates, matched by parameter name with a new coefficient at 0,
which cannot end below the current model, and once from the default start
as before, because the likelihood has several optima and that start
sometimes reaches a higher one. The default start now holds each fixed
shape at the current model's value. It took the phase specification's
value, so a model fitted with a userthetathat set a fixed shape was
refit with that shape moved, and the candidate was credited with the
gain.$fit$refit_startrecords which start
won and$fit$refit_objectivesboth results. Fixed shapes keep their
fixed values, the first of severalcontrol$n_startsis the start
itself, and Conservation of Events runs as before. Single-distribution
refits already started from the current estimates. Each candidate now
costs two fits. A multiphase model whose global formula lists more
covariates than its phase formulas use can be refit (it could not once
refits were given a start), and athetapassed tohzr_stepwise()
through...is refused with a message that says why.hazard()now
rejectsdist = NAwith its own message; it passed the check and
failed later with a base-R error.
This applies to every path that refits a multiphase model:hzr_stepwise()
under each criterion,hzr_bootstrap()select mode, and the code
hzr_translate_sas()emits for a SAS stepwise job. -
hzr_gof()with a customtime_gridnow says how many patients its
totals leave out (#492). With a custom grid, a patient is counted in
the observed and expected tallies only if their exit time is a grid
point. A grid such asseq(0, max, length.out = 50)holds almost none,
so on theavcdata the totals covered 1 of 68 events and printed a
Conservation ratio of 0.227, with no warning.hzr_gof()now warns with
the number of patients left out, as it already did on the default grid.
The"summary"attribute gainsn_tallied, the number counted, and
print()adds a note when it is belown. A patient with case weight 0
adds nothing to either tally, so one left off the grid is not reported.
Code that checks the summary's names, or runs withoptions(warn = 2)
over a custom grid, sees the change. -
A Conservation of Events fit whose likelihood is higher with the
conserved phase switched off is now recorded and warned about (#261).
Under Conservation of Events the conserved phase'slog_muis solved so
that it absorbs the events the other phases leave. Where the other phases
already account for every event there is nothing to solve, and the
objective fell back, with no warning, to thatlog_mu's starting value.
The objective was therefore discontinuous there, and a fit could stop
short of a higher likelihood: the reproduction in #261 reported -206.743
where -206.669 was available with the constant phase's scale at zero.
PROC HAZARD stops the run at that point (SETCOE1200,consrv.c). After
such a fit,hazard()now scores the fit's own log-likelihood with the
conserved phase's scale sent to zero, both with the other parameters held
and with the other phases' scales rescaled to conserve the events again.
It records a finding only if one beats the reported value by more than
0.01. A finding goes infit$fit$boundarywith mechanism
"coe_no_events_left", carrying the higher point, and raises a warning
of classhzr_coe_no_events_left, which inheritshzr_boundary. The
estimates are unchanged.fit$fit$boundarycan hold several records, in
no guaranteed order, so select them bymechanism. -
A g3 phase under
constraint = "eta_gamma"whose likelihood is higher
at a much largergammais now recorded and warned about (#418). As
gammagrows, this phase tends to a corner law whose log-likelihood is
finite, so when the data prefer a sharp bend attauthe supremum can lie
atgamma = Inf. A fit could still stop at an ordinary-lookinggamma
with a standard error, reportconverged = TRUE, and warn about nothing.
On data drawn from the corner law, a fit stopped atgamma= 7.5 with the
likelihood 2.0 units higher toward infinity. After a converged fit with
gammabelow 1000,hazard()now searches the corner law with the other
phases held at their estimates. It records a finding only when the fit's
own log-likelihood, evaluated atgammaof 1e4, 1e6 or 1e8, beats the
reported one by more than 0.01, so a weak search can miss a case but
cannot report one that is not there. That proves the reportedgammais
not the maximum-likelihood estimate; it does not prove the supremum is at
infinity rather than at another largegamma. A finding goes in
fit$fit$boundarywith mechanism"g3_corner_supremum", carrying the
higher point, and raises a warning of classhzr_g3_corner_supremum,
which inheritshzr_boundary. The estimates are unchanged. PROC HAZARD
has no such check and would report the samegamma. On simulated data
this caught 12 of the 14 warning-free fits that stopped short. A fixed
tauoralphakeeps its value in the search and in the check. A fixed
gammais not examined, and nor are fits with left-censored or
interval-censored rows, or a supremum atgamma= 0. -
hzr_deciles()now counts entry times and case weights, as
hzr_gof()does (#491). Each subject's expected events were its
cumulative hazard at exit, and events were counted unweighted. A
left-truncated subject is at risk only from its entry time, so its
expected count is the cumulative hazard at exit minus that at entry, and a
weighted fit conserves weighted events. So a correctly specified fit
looked badly calibrated. Onavcwith entry times for 40% of subjects, a fit
that conserves events expected 107.7 events against 68 observed, where
hzr_gof()expected 68. A weighted fit reported 68 events where the
weighted count was 85.4. Both tallies now carry the entry-time correction
and the weights, theeventscolumn is weighted for a weighted fit, and
the group totals again sum to the observed events. For a weighted fit the
chi-square divides by the Poisson variance of the weighted count, and the
rates are per unit of weight, so neither changes when every weight is
multiplied by the same constant. A fit with neither entry times nor
weights is unchanged. -
An AIC stepwise entry no longer rejects, as if tested, a candidate whose
refit ended below the current model (#490). Entering a term gives a
model that contains the current one, so its log-likelihood cannot be lower
at the optimum, and a refit that ends below it did not converge, whatever
itsconvergedflag says. Its change in AIC was scored all the same,
came out positive, and the candidate was rejected with no warning and
every counter at 0. Onavc, with an early phase and a constant phase and
n_starts = 1,opmosin the constant phase refit to 11.2
log-likelihood units below its base and was rejected at an AIC change of
+24.5. Such a candidate is now refused, counted in
$criteria$n_uncomputable_scoresunder the reasonloglik_below_base,
andhzr_stepwise()warns that it declined it without testing it. More
starting points or iterations may let the refit converge.
hzr_bootstrap()in select mode muffles each replicate's warnings, so it
now warns itself when replicates completed after declining an entry this
way, or for being fitted on other rows (#488). Unless a later step of the
replicate tested and entered it, such a candidate counts as not selected,
so its pooled selection frequency may be understated. -
hzr_evaluate()no longer returns a curve at athetaits likelihood
cannot evaluate, and says when itslogLikis the SAS objective
(#503). Where the log-likelihood came back as-Infwith a warning,
thecurveasked for withtimeswas still computed, and its shape
functions have no such guard. At a g3log_tauof 800 the late phase
switched itself off there, socumulative_hazardwas finite and
plausible (0.019, 0.087 and 0.295 at times 0.1, 1 and 10). The curve's
hazardandcumulative_hazardare nowNAwhereverlogLikis
-Inf, and the warning, classhzr_evaluate_not_finite, says so. A
model built withobjective = "sas"scored its interval-censored rows
with PROC HAZARD's interval-mean-hazard term, and the result called that
value the log-likelihood. The result now carriesobjective, and
print()labels such a value as the SAS objective, not a
log-likelihood. -
hzr_stepwise()now says when a step changes the rows the model is
fitted on (#519). A multiphase fit drops every row where a variable in
the model is missing. So entering a variable with missing values shrank
the sample for every later step, and dropping it grew the sample back,
with no warning. In one screen of 310 rows, a complete variable entered
and then one missing on 60 rows did, and the final model rested on 250
rows. Such a step now warns fromhzr_stepwise(), with class
hzr_stepwise_sample_changed, naming the step, the variable and the row
counts before and after. Rows of weight 0 don't count as a change, since
they add nothing to the likelihood.$steps, and soas.data.frame()of
the result, gains ann_rowscolumn. Which variables enter is unchanged.
hzr_bootstrap()suppresses warnings in its select-mode replicates, so a
replicate whose sample changes is not reported. -
A stepwise refit of a single-distribution model now reports each
coefficient under its own name (#489). The refit started from the
current estimates with a zero appended for the new term, but
model.matrix()puts an interaction after every main effect, and
hazard()keeps the names of a namedtheta. With~ age * maland a
namedtheta,com_iventering was reported asbeta_agexmal = 0.974,
and the interaction's own coefficient had no name. A drop removed the
slot at the term's position among the terms, so dropping a term that
follows a factor removed one of the factor's coefficients and moved the
dropped term's name onto another column. The starting values are now
matched to the new design by column. A factor entering or leaving, and a
model withtime_windows, which used to stop with athetalength
error, now refit.hzr_stepwise()now counts a model's shape
parameters the wayhazard()does, ignoringcontrol$shape_param_count.
For a fit given a count that disagreed with its distribution, the score
screen used to stop with athetalayout error, and under
criterion = "wald"or"aic"the trace's "Final model" line reported
the wrong number of covariates. -
A translated
PROC HAZPREDjob now predicts at the grid SAS used
(#494).hzr_translate_sas()read only the firstDOloop of the
firstDATA <name>;step, and took the loop variable as the time.
Three wrong answers followed, none recorded. Covariates a
SET DESIGNbrought in were dropped, andpredict()evaluated them
at 0:hp.death.AVC.hm1lost its whole design.TIME YEARS;with
YEARS = MONTHS/12predicted at theMONTHSvalues, and
hp.death.AVC.hm2predicted at itsOPMOSloop, where SAS predicts
at month 6. A grid defined twice used the first definition, where SAS
uses the last, sohm1lost itsDIGITALrows. The grid is now
rebuilt from the job's own DATA steps, and thetimecolumn is the
variableTIMEnames. Fed to theHAZPREDbinary, the grids emitted
forhm1andhm2reproduce the survival SAS printed for them to the
listing's five decimals. A DATA-step statement the translator can't
compute leaves its variableNAin the grid, with a warning and an
$untranslatedrow. A grid whose rows it can't determine is refused. -
hzr_translate_sas()refuses aPROC HAZARDjob with noDATA=, and
aPROC HAZPREDblock with noDATA=,INHAZ=orOUT=(#497,
#498). The%HAZARDmacro finds its dataset only throughDATA=and
stops without it ("HAZARD not attempted",hazard.sas:11-15, :145-152);
%HAZPREDrequires all three options and stops the same way
(hazpred.sas:13-33, :153-163). So SAS runs neither job. The translator
refused a job with noDATA=only when a phase had covariates or it
carriedSELECTION(#311). Otherwise it emitted a fit that read the time
and event variables from whatever the session held. ADATA=with no
dataset name, which PROC HAZARD rejects as a syntax error, took the same
path. AHAZPREDblock with an option missing emittedpredict()
anyway: over the fitting rows with noDATA=, and from the job's own fit
with noINHAZ=. The same happened when one of the three had no dataset
name, such asINHAZ=orOUT=WORK., which PROC HAZPRED rejects as a
syntax error. Each is now recorded in$untranslated, and the fit or
prediction chunk is astop()that names the missing or invalid option.
None of the 39 public-corpus jobs that translate changes. -
A single-distribution fit started where the likelihood is not defined no
longer reportsconverged = TRUE(#486). The optimizer replaces a
non-finite log-likelihood with a large penalty, so from such a start every
trial point scores the same and it stops at once. For the exponential,
Weibull, lognormal and loglogistic fits that read as a converged fit with
log-likelihood-1e10and the starting values as estimates: for example
theta = 800for the exponential fit of the shippedavcdata. Such a fit
now reportsconverged = FALSE, no objective (NA) and no standard
errors, names the cause in the "Not done in this run" record, and warns
with class"hzr_infeasible_start". The multiphase path already recorded
such a start as infeasible and is unchanged.hzr_bootstrap()now counts a
replicate that ends there as a non-finite objective rather than at the
sentinel. -
A single-distribution fit stuck at a start where the log-likelihood is
finite but below-1e10no longer reportsconverged = TRUE(#512).
There every trial point outside the finite region scores better than the
start under the optimizer's penalty, and the gradient is too large for its
line search, so it stopped without moving.hazard(Surv(int_dead, dead) ~ 1, data = avc, dist = "weibull", theta = c(50, 50), fit = TRUE)read as
converged with log-likelihood-3.55e+196and the starting values as
estimates, and "Not done in this run: none". The exponential (theta = 400) and lognormal (theta = c(1, -300)) fits did the same. Such a fit is
now handled as #486's:converged = FALSE, no objective, the cause in the
"Not done in this run" record, and a warning of class
"hzr_start_past_penalty", which inherits"hzr_infeasible_start". The
test is where the optimizer ends, and a fit is flagged only when it also
fails SAS/C's relative-gradient test there, or when the score there cannot
be used (it errors, is not finite, or has the wrong length). So a start that far out which
the optimizer leaves still fits (the exponential fromtheta = 50reaches
-434.29). A heavily weighted fit that has converged to its true maximum
below-1e10is not called stuck, because it passes the test. But with
very large total weights (around 1e10), the optimizer can still run into
its penalty from an ordinary start, and such a fit is refused with a
warning (#513).hzr_bootstrap()counted such replicates as successes; they now
fail as a non-finite objective. -
A single-distribution fit that stopped on a zeroed score no longer
reportsconverged = TRUE(#518). The optimizer stops when the
log-likelihood stops changing, not when the gradient vanishes, and it
treats a score that is not finite as zero. From a far start it could stop
on those zeros: the loglogistic fit ofavcfromtheta = c(-1e5, 1)
read as converged at log-likelihood -4727490.72, and the only warnings
concerned the Hessian. A fit whose score is not finite, or cannot be
computed, at the estimates now reportsconverged = FALSE, with a
warning of class"hzr_unverified_convergence". The estimates,
log-likelihood and standard errors are still returned, as for any fit
that did not converge. The multiphase path is unchanged.
hzr_bootstrap()never readconverged, and it muffles each
replicate's warnings, so from that start all 10 replicates counted as
successes, pooling a first parameter with mean -43435 where the maximum
is about -2.07. A replicate that does not converge now fails, with the
reason "refit did not converge (converged = FALSE)", and bootstrapping a
fit that did not converge warns. A fit that
stops short of the maximum with a finite score, such as the lognormal
fromc(1e5, 1), can still reportconverged = TRUE; that case remains
open. -
A translated
PROC HAZPREDjob now draws its bands at the level SAS
uses: one standard error, unlessCLIMITS=names another (#493).
The emittedpredict()calls never setlevel, so they used
predict.hazard()'s 0.95 and every band was 1.96 times too wide, with no
error or warning.CLIMITS=was read and discarded. The default
CLIMITSin PROC HAZPRED is 0 (hazpred/stmtprc.c:14), and any value outside
(0, 1)gives a multiplier of one (hazpred/hzpp.c:8-9), so the calls
now carrylevel = 2 * stats::pnorm(1) - 1by default and the
CLIMITS=value when it lies in(0, 1). At the one-SE default the
multiplier is exactly 1 in both. For aCLIMITS=inside(0, 1)the
level matches, and the multiplier agrees to about 4e-4: R uses the exact
qnorm(), and SAS a rational approximation (hzd_calc_norinv.c:13-17),
giving 1.6452114 against 1.6448536 at 0.9.NOCLnow wins over a later
CLIMITS=, as it does inhzpp.c, whereCLIMITS=had switched the
bands back on. ACLIMITS=value SAS cannot read is recorded in
$untranslated. -
hzr_stepwise(criterion = "score")now refuses adatawhose rows are
not the base fit's rows in the fit's order (#487). The score test reads
each candidate fromdataby position and scores it against the fit's
stored rows, and it checked only the row count. The same rows sorted or
shuffled scored every candidate against the wrong observations and
entered a different variable, with no warning: onavc, a Weibull base
enteredcom_iv(p = 0.0001) from the original frame andmal
(p = 0.17) from the frame sorted by time. Every columndatashares with
the data frame given tohazard()must now match it; a column only
datahas, such as a candidate derived after the fit, is not compared.
The vector interface (hazard(time =, status =)) refits by pairing its
stored response withdataread by position, so there the Wald and AIC
criteria misread a reordered frame too, enteringcom_ivforopmos;
on that interface the comparison now runs under every criterion. So it
does for a fit withweights, on either interface: the refits reuse the
stored weights, in the fit's row order. Only an unweighted formula fit,
which rebuilds every per-row input fromdata, is left to the score
test's own comparison. Adataholding only columns added after the fit
shares none with that frame, so it cannot be compared, and is treated as
below; so is one whose shared columns have duplicate rows, since rows
reordered among duplicates leave those columns unchanged. Two fits store no
frame that can be compared: a vector-interface fit made without
data =, and one whosedataserved only to look names up and so has
another row count. For those,hzr_stepwise()looks for a column of
dataholding the fit's event times. In the fit's order, the screen
runs; in another order, it is refused, under every criterion. When the
fit's times have ties, a column holding them in order cannot show that
rows within a tie are in order, so it is not taken as proof on its own;
the fit's other per-row inputs are then checked as well (#515, below).
With no such column, or tied times those inputs cannot resolve, the order
cannot be checked, and the screen warns once, with class
hzr_score_rows_unverified. -
hzr_stepwise()now checks row order within tied event times against
the fit's status, weights and covariates (#515). Where it could only
checkdata's order against a column holding the fit's event times,
rows reordered within a tie left that column unchanged, so tied times
gave the same warning whether or not the rows had moved. With discrete
times onavc(14 distinct in 305 rows), a shuffle within ties moved 289
rows and took every criterion's first entry toopmos(fromcom_iv
under score and AIC, frommalunder Wald), under the same warning as
the aligned frame. Each other per-row input the fit stores (status,
interval bounds, weights, covariate design columns) is now looked for in
dataunder the column it came from: the one the call named
(status = dead,status = d$dead, or the event inSurv(time, event)),
or a covariate's own name. A column that merely holds the same values
does not count, so a look-alike cannot vouch for the order. The input's
own column holding the same values paired with the same times but in
another order means rows moved within a tie, and the screen is refused,
naming the column. The order is accepted, without a
warning, when the inputs found tell every row apart, or when all of them
are found: rows that could still be swapped are then identical in
everything the fit reads, and the screen's answer is unchanged. Otherwise
the warning remains, and now names the inputs to add todata. Every one
of these comparisons is now exact, as are the event-time column and the
columns compared with a stored frame. Underall.equal()'s tolerance,
rows whose times differed only by rounding, such as0.1 + 0.2and0.3,
could be swapped unseen and screened as if in order. Adatawith
duplicated column names is now refused: every check reads columns by
name, and of duplicates only the first is read. -
hzr_stepwise(criterion = "aic")no longer enters a variable on the
strength of its missing values (#488). A multiphase refit drops every row
where the candidate is missing, so the candidate's log-likelihood summed
fewer rows than the current model's, and the two AICs were compared as if
they covered the same data. A pure-noise variable missing on 60 of 310 rows
entered with a change in AIC near -58 while its own Wald p-value was 0.23,
with no warning. A candidate whose refit used different rows is now not
scored: it is counted in$criteria$uncomputable_reasonsas
rows_differ, a run that ends with no other candidate to test stops with
the untested-screen warning, and a run that completes warns that the entry
was declined. The Wald and score criteria never compared two fits and are
unchanged. -
predict()with anewdataof only atimecolumn now warns when the
model has covariates (#522). Such anewdataevaluates every covariate
at 0. For a Weibull fit ofSurv(int_dead, dead) ~ age + malon
na.omit(avc)that returned survival 0.7713 and 0.6908 at 12 and 60 months,
the prediction for a patient of age 0, with no warning; anewdatathat
gave some covariates but not all was already refused. The values are
unchanged, butpredict()now warns, once per call and with class
"hzr_predict_covariates_zero", naming the data variables to supply
(age, not the model-matrix columnscale(age)). This
covers the single-distribution fits, on both interfaces, and the
multiphase fits, whose covariates enter globally or through a phase
formula. A model without covariates does not warn, and?predict.hazard
documents the behaviour undernewdata. -
A translated
MAXITER=0job now evaluates the starting values, as
PROC HAZARDdoes, rather than fitting (#496).PROC HAZARDskips
its optimizer atMAXITER=0and prints the log-likelihood at the
starting values, after Conservation of Events has scaled everyMUby
one factor unlessNOCONSERVEis given.hzr_translate_sas()emitted
control = list(maxit = 0), andhazard()optimised anyway: one job
reported -198.370 withconverged = TRUEwherePROC HAZARDprinted
-295.609. The chunk is nowhzr_evaluate()at those starting values,
with the same scaling, and it reproduces the log-likelihood andMUE
thatPROC HAZARDprints with and withoutWEIGHT,LCENSORand
NOCONSERVE, and withICENSOR(#543). With fewer events than free
parameters it stops, asPROC HAZARDdoes. It is not a fit, so the job
warns and gains an$untranslatedrow.PROC HAZARDstill steps through aSELECTIONscreen, evaluating
each step at unfitted values; the translation does not, and says so. A value below 1 is
read the same way, asPROC HAZARDtruncates it to 0. A negative
MAXITER, whichPROC HAZARDignores, is no longer emitted:
hazard()had returned the starting values withconverged = TRUE. -
A translated
ICENSORstatement thatPROC HAZARDcannot parse now
warns (#495).ICENSORtakes exactlycount = timevar, andPROC HAZARDstops with a syntax error on anything else: a comma anywhere, a
missing=, or an extra name.hzr_translate_sas()removed a trailing
comma and fitted the job without a word, and it fitted a job with a
missing=without its interval-censored rows, with only an
$untranslatedrow to say so. Such a job now warns thatPROC HAZARD
does not run it and gains an$untranslatedrow. The fit uses the
namesPROC HAZARD's parser reads, the firstcount = timevaronce its
lexer has dropped the errors, soC3=TL,AGEfitsC3andTL, not a
variableTLAGE. A later(clearsPROC HAZARD's syntax error, and
the warning then says so instead. -
A translated
ICENSORjob now fitsPROC HAZARD's interval objective
(#543).PROC HAZARDaccumulates an interval-censored row as the
interval-mean hazard over (lower bound, time], whichhazard()calls
objective = "sas", andhzr_translate_sas()emitted the default
interval probability instead. Measured on the binary over a grid of 18
optimised fits, that moved the constant phase'sMUby up to 21% and the
reported objective by up to 291 units. The emitted call now passes
objective = "sas", which reproducesPROC HAZARD's estimates to the
precision it prints. A note above the fit says that the value it reports
isPROC HAZARD's objective, not a log-likelihood. Degenerate intervals
are resolved asPROC HAZARDresolves them before its fit: a lower bound
equal to the time makes the row an exact event, and a lower bound that is
missing, negative or after the time drops the row. The status chunk warns
with the number of rows each rule touched, and the data frame itself
keeps every row. -
A macro in a translated phase statement no longer hides an empty item
(#479).PROC HAZARDneeds a variable on each side of every,in
EARLY,CONSTANTandLATE, and stops with a syntax error otherwise.
hzr_translate_sas()skipped that check for a whole statement whenever
any item was a macro, soEARLY , &X;andEARLY SEX,, &X;fitted with
no warning and no$untranslatedrow. A macro can stand for an item,
but no plain variable fills an empty one, so these now warn and gain a
row, as their macro-free forms already did. An empty argument inside a
macro call, as in%F(A,,B), belongs to the macro and is not flagged.
A quoting function such as%STR()passes its argument through as text,
so%STR(A,,B)is judged asA,,B. -
?hazardno longer says a position-indexing helper behaves as it would
understats::lm(subset = )(#484). Whenhazard()drops a time-0
row, it builds the design on the retained rows. A term such as
hf(age), withhf <- function(a) a + g[seq_along(a)]andgoutside
data, then pairs the retained rows with the first elements ofg.
Every row after the first dropped one gets the wrong value, and only the
drop is warned about.lm(subset = )evaluates its terms on every row
before subsetting, so the same helper pairs correctly there; the page
wrongly implied the two agree. The advice is unchanged: put such a vector
indata. Evaluating terms aslm()does is #590. -
hzr_translate_sas()translated an early phase that starts atM=0as
a different model from the onePROC HAZARDfits (#471). For
PARMS ... M=0 NU=1, withMnot fixed,SETG1fixesMat 0 before the
fit, and it does the same for a negativeNU. The translation leftM
free, so the emittedhazard()call estimated a parameterPROC HAZARD
holds, and it could reach a different optimum with no warning. The
translation now fixesMasSETG1does, and records the rewrite in
$untranslated. Two related starting values are mirrored as well: with
NUfixed andMfree,SETG1startsMat 1, and withMfixed and
NU=0it startsNUat 1.M=0 NU=1,M=0 NU=0 FIXMand
M=0 NU=1 FIXMnow emit the same model, as they are the same job to
PROC HAZARD. On data drawn from that model, and from a start where the
HAZARDbinary converges, the emitted fit reproduces its estimates. -
hzr_translate_sas()now mirrorsSETG1's other early-phase rewrites
(#601). WithM<0andNU<0,SETG1flips signs into a valid
model:M's whenNUis fixed,NU's whenMis fixed, and both
otherwise. The translation kept both negative, and the emitted
hzr_phase()call stopped with an error when the document ran. With
NU=0, a nonzeroMand neither fixed,SETG1fixesNUat 0, and for
M>0it also flipsM's sign, which it does withNUfixed too. With
M=0andNU=0 FIXNUit startsMat 1. The translation emitted none
of these, so the "may not fit" and "no result" notes for those jobs
described a model the emitted call did not fit. It now emits the model
PROC HAZARDstarts from, and records each rewrite in$untranslated.
ForM=0 NU=0 FIXNU, whichPROC HAZARDcannot evaluate, the emitted
fit now stops with an error as well, where it used to fit a model
PROC HAZARDnever reaches. When a
macro reference inPARMScould carry a value or flag that changes
SETG1's case, the note says the rewrite holds only if it does not. -
A
PROC HAZPREDgrid is emitted only whenhzr_translate_sas()reads
every statement that can write it (#621, #625, #628). The grid is
rebuilt from the job's DATA steps under an allow-list: global statements
(TITLE,OPTIONS,LIBNAMEand the like),%LETand%PUT,
procedures whose output datasets are all named by an option or statement
the translator reads, DATA steps it translates, and aPROC SORTwith
onlyDATA=,OUT=and an ascendingBY. Anything else between a step
the grid uses and thePROC HAZPREDrefuses the grid, with an
$untranslatedrow naming the statement: aWHEREorNODUPKEYin that
sort, an%INCLUDE, a call of a macro the file does not define or whose
body writes the grid (with a DATA step, a sort in place, orPROC DATASETS), open-code%IFor%DO, and a macro call with no semicolon,
which swallows the statement after it. SAS macro scope is respected: a
step inside a%MACRO ... %MENDdefinition does not define the grid where
it stands, and aPROC HAZPREDinside a definition is refused, because
SAS runs it where the macro is called. A call before the grid's first
step cannot write a dataset the grid reads, but it may change session
state the grid depends on, such asOPTIONS OBS=: the grid is emitted,
and the call has an$untranslatedrow saying so. A statement the
translator cannot place outside any step, orOPTIONS OBS=itself,
refuses the grid wherever it stands. On the public corpus, four jobs gain
two such rows each, for an%INCLUDEand a macro call before the grid. -
A stepwise screen on a Weibull fit whose scale cannot be represented
failed every candidate refit, and a selection-modehzr_bootstrap()
pooled such replicates as successes (#566). The refit's warm start
copiedmu, which is 0,Infor subnormal on such a fit, andhazard()
refuses that as a starting value.hzr_stepwise()said so, but
hzr_bootstrap()did not read it: on 400 rows with a covariate near 1000,
20 replicates all counted as successes andz, a true effect, was
selected in 75% of them, against 100% for the same model with the
covariate centered. The warm start now takeslog(mu)from the fit,
moved into the range a number can hold, and carries the difference
through the coefficients. The same 20 replicates now select what the
centered fit selects, replicate for replicate. Separately,
hzr_bootstrap()now counts the selection-mode replicates in which a
candidate refit failed, for any reason, and warns once: such a candidate
was never tested. Those replicates are still pooled. The warning about a
muthat cannot be represented no longer says the other parameters are
unaffected without qualification. -
hzr_bootstrap()pooled replicates that stopped onnlm()code 4 or 5
and failed the relative-gradient test, with no count and no warning
(#622).
hazard()warns about such a fit, but replicates run with their warnings
suppressed, and the count for #531 leaves those codes out because
hazard()warns for them.hzr_bootstrap()now reads the base refit and
the final fit of each replicate and warns once with the number of
replicates affected. Their estimates are still pooled. A stepwise refit
that was not kept is not read, since the screen does not return it. -
hzr_translate_sas()code failed on R before 4.4 when a phase statement
named a variable that is not a syntactic R name, such as_X1: the fit
stopped with "object '_X1' not found". The emitted status chunk added its
column withtransform(), which renamed_X1toX_X1before R 4.4.
The chunk now assigns each derived column directly,
D[[".hzr_status"]] <- with(D, ...), so the column keeps its name on
every R version, including when the chunk runs again on data that already
carries.hzr_status(#609). -
A
PROC HAZPREDTIMEstatement with more than one variable now stops
the block, asPROC HAZPREDdoes (#628).TIMEtakes exactly one name, and a
second is a syntax error on which the procedure stops.hzr_translate_sas()
took the first name and emitted the predictions. It now emits astop()
and an$untranslatedrow, as it does for a multi-operandTIMEin
PROC HAZARD. -
A grid
DATAstep withSET;and no dataset name, or a numeric
LENGTHbelow 8 bytes, now refuses the grid (#628).SET;reads the most
recent dataset, and was read as noSETat all; a short numeric length
stores fewer digits than the emitted grid holds. -
A select-mode
hzr_bootstrap()replicate whose base refit did not converge
is now counted as failed (#627), with the reason "base refit did not converge
(converged = FALSE)". The rule that a replicate which does not converge
fails applied only to the final fit, so undercriterion = "aic"each
candidate warm-started from the base's unfinished point, reached the real
maximum and was credited with the base's shortfall. Under a forward Wald
screen a replicate that entered nothing failed while one that entered
something passed, so the replicates kept were biased towards selection. On 300 Weibull rows
with two pure-noise columns, a base that converged on the full data but not
on any resample gave 19 of 30 successful replicates with no warning, and
selected the noise columns in 12 and 8 of them. -
hzr_stepwise(criterion = "aic")now refuses a base fit that did not
converge (#627), ascriterion = "score"already did. A candidate's delta AIC is
measured against the base's log-likelihood, so a base that stopped short
entered pure noise at delta AIC -16.5 with no warning.criterion = "wald"
refuses one too unlessdirection = "forward": a Wald entry is tested at
the candidate's own converged refit, but a removal is tested on the base's
own estimates and variance. -
hzr_deciles()now refuses a multiphase fit that dropped the rows where a
phase covariate was missing (#627), with the same check and message as
hzr_gof(). It predicted each subject's cumulative hazard from the shorter
design, recycled onto the wrong subjects, and reported every subject
included: onavcwith half the covariate missing, 70 observed events
against 159 expected and p < 2e-16, with no warning. -
hzr_translate_sas()could emit a PROC HAZPRED grid in an order SAS
does not use, with no$untranslatedrow (#630). APROC SORTwith noDATA=
sorts SAS's most recent dataset,_LAST_. The translator took that to be
the dataset the job last named, butDATA;, anOUTPUTstatement with no
OUT=(PROC MEANS, SUMMARY, REG and others),OPTIONS _LAST_=and a macro
whose body does one of these all change it unseen. A grid of 12, 6, 1
followed byDATA; ... PROC SORT; BY MONTHS;was emitted as 1, 6, 12. A
PROC SORTwith noDATA=now refuses the grid when it sorts a dataset
the grid reads or stands between a step the grid uses and the PROC
HAZPRED,OPTIONS _LAST_=refuses it wherever it stands, andDATA;and
anOUTPUTwith noOUT=are recorded as writing an unnamed dataset. -
A grid sorted by a character variable could come out in an order SAS
does not use (#630). SAS compares character values in ASCII order and at the
column's fixed length, which its first assignment sets: afterG='B',
the values'AB'and'AA'are both stored as'A'and keep their input
order, where R sorted the full strings, and in the session's locale.
OPTIONS SORTSEQ=changed SAS's order unseen too. The emitted grid code
now stops when aPROC SORTit reproduces is by a character variable, and
OPTIONS SORTSEQ=orNOSORTEQUALSrefuses the grid. Numeric sorts are
unchanged; on the public corpus no emitted grid sorts by a character
variable. -
A DATA step between the grid and the PROC HAZPRED could rename or
delete the grid through a function, and the grid was still emitted (#630). Any
CALLroutine, and functions such asRENAME(),FDELETE(),SYSTEM(),
OPEN()andFETCH(), now refuse a grid the step stands inside. -
A
DATA PGM=statement between the grid and the PROC HAZPRED no longer
passes unseen (#630). It runs or stores a compiled DATA step program, which may
rewrite the grid, and now refuses a grid it stands inside.
Packaging and documentation
-
Articles on the pkgdown site put the table of contents on the left and use
the full width of the window, throughpkgdown/extra.css. The installed
vignettes are unchanged: the Quarto vignette engine renders them in its own
minimal format, which has no sidebar layout. -
DESCRIPTIONnow declares the Quarto command line tool in
SystemRequirements. The vignettes have always needed it to build; the
field makes that visible to installers and toR CMD check.
Known limitations
- Multiphase fits are about two to three times slower than in 1.2.12
(#631). The fixes that keep a saturated hazard phase's value (#578) and
difference the early-phaset_halfderivative on the log scale (#574)
made each evaluation of the phase decomposition about four times as
costly. One multiphase fit of 5,880 rows took 0.7 seconds in 1.2.12 and
1.7 seconds now, and a 20-replicatehzr_bootstrap()of it 7 seconds and
23. Estimates are not affected by the slowdown. It is to be removed in
1.3.0 without changing any value.