Skip to content

Releases: ehrlinger/hvtiRdatabuild

hvtiRdatabuild 0.3.3

Choose a tag to compare

@ehrlinger ehrlinger released this 25 Oct 12:21
3a836b4
  • write_analysis_set() builds a set from registered data only, because an analysis set is
    reused by other jobs. It refuses a working read of the study dataset, stops and writes
    nothing.

    • A dataset with no dated version, including one registered before dated versions (a
      manifest entry with no parquet), is refused before it is read and the message names
      hvtiRutilities::update_manifest(). Run it once to convert a legacy registration; sets
      already written read as before.
    • A registered dataset read as working because HVTI_DATA=working is set is refused with a
      message that says to unset HVTI_DATA.
    • In a final render or an Rscript run, hvtiRutilities' own gate may refuse the read first,
      with its own message.
    • Requires hvtiRutilities 1.5.3, the first release that signals a working read.
  • The README points analysts to vignette("registered-datasets", package = "hvtiRutilities"),
    which explains dataset registration, version names and which version a job reads. It ships
    with hvtiRutilities 1.5.12 and later.

hvtiRdatabuild 0.3.1

Choose a tag to compare

@ehrlinger ehrlinger released this 14 Oct 21:25
b0ef4b7
  • The test of an analysis set over a dataset registered before dated versions
    now writes the set as a datasets job would. hvtiRutilities' stage gate serves
    a dataset with no dated version only to the console and to datasets or
    descriptive jobs, so the test's non-interactive run was refused. Test-only.

hvtiRdatabuild 0.3.0

Choose a tag to compare

@ehrlinger ehrlinger released this 12 Oct 18:58
c2d2b8b
  • Now requires R 4.4.0 or newer, up from 4.1.0, to match the rest of the
    HVTI family. hvtiR::install() installs the members together, and several
    already required 4.4.0, so on an older R the install failed whatever this
    package declared. Now requires hvtiRutilities 1.5.1.

  • The build layer. Three pieces sit between dw_pull() and the analysis
    sets. build_dataset() joins a study's patient-level modules on its cohort
    and keeps repeated-event tables beside the result rather than widening them.
    derive_vars() runs the study's own R code, committed with the study, and
    takes back the data with an updated cohort. A cohort_record, from
    new_cohort_record(), append_cohort_stage() and cohort_summary(), carries
    inclusion, exclusion and attrition history through both, in order. A missing
    inclusion value is an error, not an implicit pass or exclusion. Duplicate
    source identifier columns are kept for audit, patient identifiers stay stable
    across joins, and the print() and str() methods and callback diagnostics
    never show a patient value. vignette("coming-from-sas") walks the build,
    study code and attrition through a synthetic example. dplyr moves from
    Suggests to Imports.

  • A stale analysis set no longer stops a draft. read_analysis_set()
    reads a set whose parent or declaration has changed, and signals
    hvtiRutilities_stale_analysis_set with the write_analysis_set() call that
    updates it. A final render (HVTI_TEMPLATE_STRICT set, as
    render_job(final = TRUE) does) still stops, with the same commands. This
    reverses the earlier rule that every stale read stopped; a damaged set still
    stops in every render.

  • An analysis set's parent is now the registered version of the built dataset
    when hvtiRutilities registered it as a dated parquet. Rebuilding
    built.sas7bdat without registering it no longer makes every set stale,
    because jobs read nothing new until hvtiRutilities::update_manifest()
    registers it. Sets written before this change name the SAS file as their
    parent, so each is stale once after the study's first update_manifest();
    write_analysis_set() refreshes it. A dataset still registered the old way
    is identified by its source, as before.

  • write_analysis_set() and read_analysis_set() skip the line of text that
    hvtiRutilities 1.5.1 writes at the head of a manifest.yaml holding a
    registered dated version. Without this they stopped with "$ operator is
    invalid for atomic vectors" on every study registered from 1.5.1 on.

  • write_analysis_set() refuses a set named for a registered dated version,
    such as built_20261008, before writing anything. Its parquet would have
    replaced that version, the registered data every job reads.

  • DESCRIPTION now declares the Quarto command line tool in
    SystemRequirements. The vignettes have always needed it to build; the
    field makes that visible to installers and to R CMD check.

  • Articles on the pkgdown site put the table of contents on the left and use
    the full width of the window, through pkgdown/extra.css. The installed
    vignettes are unchanged: the Quarto vignette engine renders them in its own
    minimal format, which has no sidebar layout.

hvtiRdatabuild 0.2.4

Choose a tag to compare

@ehrlinger ehrlinger released this 29 Sep 22:52
02af8bb
  • Master datasets. Six new functions snapshot, lift and correct the master
    datasets that study builds read. read_master_config() reads a master.yml
    declaring a master's key, alternate keys and parent. snapshot_master()
    freezes its SAS builds as parquet and records which parent release each was
    built from, read from the run's log first. lift_master() loads a snapshot
    into the warehouse behind key and full-parity gates and creates the master's
    view. backfill_corrections(), propose_correction() and
    decide_correction() keep an append-only record of corrections the view
    applies. The bulk functions dry-run by default. See
    vignette("master-datasets"). duckdb and tidyselect join Suggests.

  • An analysis set names its own event column. hvtiRutilities 1.4.0 made
    study registration endpoint-neutral: register_data() no longer takes
    event or time, and study_config() no longer carries a study-wide
    cohort. write_analysis_set() had counted n_events and n_censored from
    that cohort, so under 1.4.0 every write failed. A set now declares event:
    (one of its vars) to get both counts, and an expect on either count
    without one stops with a message naming the missing key. A set with no
    event records n alone. A set that relied on the study-wide cohort needs
    the one-line event: added, which also changes its declaration hash, so
    read_analysis_set() asks for one rewrite.
    hvtiRdatabuild now requires hvtiRutilities 1.4.0 or later, since its tests
    and registration fixture use the endpoint-neutral register_data().

  • snapshot_oracle() can snapshot a dataset too large for memory. The new
    chunk_rows argument reads the SAS dataset a chunk at a time and writes the
    chunks as row groups of one parquet file. It also records a SHA-256 of the
    SAS source beside the parquet checksum, and writes a .meta.json sidecar
    holding each column's label, SAS format and type, so that metadata survives
    into systems that cannot read R attributes. jsonlite joins Suggests. The
    source checksum now brackets the read, taken before and after; a mismatch
    removes the parquet and stops rather than write a snapshot of a file that
    changed underneath it.

hvtiRdatabuild 0.2.3

Choose a tag to compare

@ehrlinger ehrlinger released this 29 Sep 22:52
c00f895
  • New publish_dataset(). A programmer can publish a mutable clinical-data
    draft as an immutable dated release. Publication preserves the draft bytes,
    verifies that the staged copy is readable, records its checksum and shape in
    dataset-catalog.yml, and serializes concurrent publishers with a catalog
    lock. Same-day corrections receive revision filenames; retrying identical
    bytes is idempotent, including recovery of a file moved before a failed
    catalog write.

  • New withdraw_dataset_release(). Withdrawal records a reason and an
    optional replacement in the catalog without changing or deleting the
    published file. The package now requires hvtiRutilities 1.3.1, whose study
    workflow discovers, reviews, and explicitly adopts these releases.

hvtiRdatabuild 0.2.2

Choose a tag to compare

@ehrlinger ehrlinger released this 29 Sep 22:52
c260e30
  • Documentation no longer points dw_pull() users at a
    building-a-study-dataset vignette that does not exist, and the SAS migration
    guide no longer lists the shipped dw_connect() and dw_pull() as future
    work. Prose across the README, vignette and reference pages was tightened.

  • The SAS migration guide now treats bd.SAStoR.sas outputs as general
    downstream data contracts rather than exports owned by their first consumer.
    It shows how the final selection becomes a shared analysis set while variable
    derivation remains in the build layer.

  • Analysis-set declarations now reject duplicate variables and require every
    expected row/event count to be one non-negative whole number. This prevents
    duplicate parquet columns from being silently renamed and fractional counts
    from matching after integer truncation.

  • Analysis sets resolve their outputs through the study's logical datasets
    directory, preserving numbered new studies and adopted legacy layouts.