Repository navigation
hvtiRdatabuild 0.2.4
-
Master datasets. Six new functions snapshot, lift and correct the master
datasets that study builds read.read_master_config()reads amaster.yml
declaring a master's key, alternate keys and parent.snapshot_master()
freezes its SAS builds as parquet and records which parent release each was
built from, read from the run's log first.lift_master()loads a snapshot
into the warehouse behind key and full-parity gates and creates the master's
view.backfill_corrections(),propose_correction()and
decide_correction()keep an append-only record of corrections the view
applies. The bulk functions dry-run by default. See
vignette("master-datasets").duckdbandtidyselectjoinSuggests. -
An analysis set names its own
eventcolumn. hvtiRutilities 1.4.0 made
study registration endpoint-neutral:register_data()no longer takes
eventortime, andstudy_config()no longer carries a study-wide
cohort.write_analysis_set()had countedn_eventsandn_censoredfrom
that cohort, so under 1.4.0 every write failed. A set now declaresevent:
(one of itsvars) to get both counts, and anexpecton either count
without one stops with a message naming the missing key. A set with no
eventrecordsnalone. A set that relied on the study-wide cohort needs
the one-lineevent:added, which also changes its declaration hash, so
read_analysis_set()asks for one rewrite.
hvtiRdatabuild now requires hvtiRutilities 1.4.0 or later, since its tests
and registration fixture use the endpoint-neutralregister_data(). -
snapshot_oracle()can snapshot a dataset too large for memory. The new
chunk_rowsargument reads the SAS dataset a chunk at a time and writes the
chunks as row groups of one parquet file. It also records a SHA-256 of the
SAS source beside the parquet checksum, and writes a.meta.jsonsidecar
holding each column's label, SAS format and type, so that metadata survives
into systems that cannot read R attributes.jsonlitejoinsSuggests. The
source checksum now brackets the read, taken before and after; a mismatch
removes the parquet and stops rather than write a snapshot of a file that
changed underneath it.