Repository navigation
Releases: ehrlinger/hvtiRdatabuild
Release list
hvtiRdatabuild 0.3.3
-
write_analysis_set()builds a set from registered data only, because an analysis set is
reused by other jobs. It refuses a working read of the study dataset, stops and writes
nothing.- A dataset with no dated version, including one registered before dated versions (a
manifest entry with noparquet), is refused before it is read and the message names
hvtiRutilities::update_manifest(). Run it once to convert a legacy registration; sets
already written read as before. - A registered dataset read as working because
HVTI_DATA=workingis set is refused with a
message that says to unsetHVTI_DATA. - In a final render or an
Rscriptrun, hvtiRutilities' own gate may refuse the read first,
with its own message. - Requires hvtiRutilities 1.5.3, the first release that signals a working read.
- A dataset with no dated version, including one registered before dated versions (a
-
The README points analysts to
vignette("registered-datasets", package = "hvtiRutilities"),
which explains dataset registration, version names and which version a job reads. It ships
with hvtiRutilities 1.5.12 and later.
hvtiRdatabuild 0.3.1
- The test of an analysis set over a dataset registered before dated versions
now writes the set as a datasets job would. hvtiRutilities' stage gate serves
a dataset with no dated version only to the console and to datasets or
descriptive jobs, so the test's non-interactive run was refused. Test-only.
hvtiRdatabuild 0.3.0
-
Now requires R 4.4.0 or newer, up from 4.1.0, to match the rest of the
HVTI family.hvtiR::install()installs the members together, and several
already required 4.4.0, so on an older R the install failed whatever this
package declared. Now requires hvtiRutilities 1.5.1. -
The build layer. Three pieces sit between
dw_pull()and the analysis
sets.build_dataset()joins a study's patient-level modules on its cohort
and keeps repeated-event tables beside the result rather than widening them.
derive_vars()runs the study's own R code, committed with the study, and
takes back the data with an updated cohort. Acohort_record, from
new_cohort_record(),append_cohort_stage()andcohort_summary(), carries
inclusion, exclusion and attrition history through both, in order. A missing
inclusion value is an error, not an implicit pass or exclusion. Duplicate
source identifier columns are kept for audit, patient identifiers stay stable
across joins, and theprint()andstr()methods and callback diagnostics
never show a patient value.vignette("coming-from-sas")walks the build,
study code and attrition through a synthetic example.dplyrmoves from
SuggeststoImports. -
A stale analysis set no longer stops a draft.
read_analysis_set()
reads a set whose parent or declaration has changed, and signals
hvtiRutilities_stale_analysis_setwith thewrite_analysis_set()call that
updates it. A final render (HVTI_TEMPLATE_STRICTset, as
render_job(final = TRUE)does) still stops, with the same commands. This
reverses the earlier rule that every stale read stopped; a damaged set still
stops in every render. -
An analysis set's parent is now the registered version of the built dataset
when hvtiRutilities registered it as a dated parquet. Rebuilding
built.sas7bdatwithout registering it no longer makes every set stale,
because jobs read nothing new untilhvtiRutilities::update_manifest()
registers it. Sets written before this change name the SAS file as their
parent, so each is stale once after the study's firstupdate_manifest();
write_analysis_set()refreshes it. A dataset still registered the old way
is identified by its source, as before. -
write_analysis_set()andread_analysis_set()skip the line of text that
hvtiRutilities 1.5.1 writes at the head of amanifest.yamlholding a
registered dated version. Without this they stopped with "$ operator is
invalid for atomic vectors" on every study registered from 1.5.1 on. -
write_analysis_set()refuses a set named for a registered dated version,
such asbuilt_20261008, before writing anything. Its parquet would have
replaced that version, the registered data every job reads. -
DESCRIPTIONnow declares the Quarto command line tool in
SystemRequirements. The vignettes have always needed it to build; the
field makes that visible to installers and toR CMD check. -
Articles on the pkgdown site put the table of contents on the left and use
the full width of the window, throughpkgdown/extra.css. The installed
vignettes are unchanged: the Quarto vignette engine renders them in its own
minimal format, which has no sidebar layout.
hvtiRdatabuild 0.2.4
-
Master datasets. Six new functions snapshot, lift and correct the master
datasets that study builds read.read_master_config()reads amaster.yml
declaring a master's key, alternate keys and parent.snapshot_master()
freezes its SAS builds as parquet and records which parent release each was
built from, read from the run's log first.lift_master()loads a snapshot
into the warehouse behind key and full-parity gates and creates the master's
view.backfill_corrections(),propose_correction()and
decide_correction()keep an append-only record of corrections the view
applies. The bulk functions dry-run by default. See
vignette("master-datasets").duckdbandtidyselectjoinSuggests. -
An analysis set names its own
eventcolumn. hvtiRutilities 1.4.0 made
study registration endpoint-neutral:register_data()no longer takes
eventortime, andstudy_config()no longer carries a study-wide
cohort.write_analysis_set()had countedn_eventsandn_censoredfrom
that cohort, so under 1.4.0 every write failed. A set now declaresevent:
(one of itsvars) to get both counts, and anexpecton either count
without one stops with a message naming the missing key. A set with no
eventrecordsnalone. A set that relied on the study-wide cohort needs
the one-lineevent:added, which also changes its declaration hash, so
read_analysis_set()asks for one rewrite.
hvtiRdatabuild now requires hvtiRutilities 1.4.0 or later, since its tests
and registration fixture use the endpoint-neutralregister_data(). -
snapshot_oracle()can snapshot a dataset too large for memory. The new
chunk_rowsargument reads the SAS dataset a chunk at a time and writes the
chunks as row groups of one parquet file. It also records a SHA-256 of the
SAS source beside the parquet checksum, and writes a.meta.jsonsidecar
holding each column's label, SAS format and type, so that metadata survives
into systems that cannot read R attributes.jsonlitejoinsSuggests. The
source checksum now brackets the read, taken before and after; a mismatch
removes the parquet and stops rather than write a snapshot of a file that
changed underneath it.
hvtiRdatabuild 0.2.3
-
New
publish_dataset(). A programmer can publish a mutable clinical-data
draft as an immutable dated release. Publication preserves the draft bytes,
verifies that the staged copy is readable, records its checksum and shape in
dataset-catalog.yml, and serializes concurrent publishers with a catalog
lock. Same-day corrections receive revision filenames; retrying identical
bytes is idempotent, including recovery of a file moved before a failed
catalog write. -
New
withdraw_dataset_release(). Withdrawal records a reason and an
optional replacement in the catalog without changing or deleting the
published file. The package now requires hvtiRutilities 1.3.1, whose study
workflow discovers, reviews, and explicitly adopts these releases.
hvtiRdatabuild 0.2.2
-
Documentation no longer points
dw_pull()users at a
building-a-study-datasetvignette that does not exist, and the SAS migration
guide no longer lists the shippeddw_connect()anddw_pull()as future
work. Prose across the README, vignette and reference pages was tightened. -
The SAS migration guide now treats
bd.SAStoR.sasoutputs as general
downstream data contracts rather than exports owned by their first consumer.
It shows how the final selection becomes a shared analysis set while variable
derivation remains in the build layer. -
Analysis-set declarations now reject duplicate variables and require every
expected row/event count to be one non-negative whole number. This prevents
duplicate parquet columns from being silently renamed and fractional counts
from matching after integer truncation. -
Analysis sets resolve their outputs through the study's logical datasets
directory, preserving numbered new studies and adopted legacy layouts.