Skip to content

Revise epix_merge to always fit in last-version-carried-forward framework, merge additional metadata #136

@brookslogan

Description

@brookslogan

as_of and other planned epi_archive operations assume we formed an archive using last-version-carry-forwardable data. Using epix_merge without all=TRUE and locf=TRUE breaks this.

locf=TRUE can produce a result that gives inconsistent as_of results with the input archives if the user has NAs in the input archive non-key columns. We want to locf only for rows that didn't exist before, at least when interpolating versions, and always want to locf when interpolating versions. However, there are other cases, such as when one archive has more up-to-date version data than the other, where we wonder about what, if any, extrapolation we should be doing; locf should be replaced with an observed_versions_end_conflict arg that allows various options there (e.g., to raise an error if one input isn't as fresh, to insert an all-NA version immediately after the last version in the less fresh input, to extrapolate with last-observation-carried-forward, or to truncate versions from the fresher input).

Additionally, epix_merge should appropriately check input metadata and set result metadata: the DT key, geo and time types, clobberable and observed versions, etc.

Metadata

Metadata

Assignees

Labels

P1medium priorityop-semanticsOperational semantics; many potentially breaking changes here

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions