Skip to content

Releases: Gilead-Public/workr

Release list

v1.1.0

v1.1.0 Latest
Latest

Choose a tag to compare

View all tags
@lauramaxwell lauramaxwell released this 17 Jul 19:18
v1.1.0
d19925e
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

workr 1.1.0

Highlights

  • Improved project orchestration with phase-level configuration files,
    project/phase load-save hooks, optional continue-on-error behavior, and
    clearer diagnostics for missing paths, empty phases, and workflow failures.

  • Added workflow discovery helpers and made workflow loading more predictable,
    including active/inactive filtering, priority ordering, exact-name matching,
    and support for the singular inst/workflow directory.

  • Added reusable load/save hook registration plus built-in GitHub Actions
    artifact providers for saving and restoring workflow data between runs.

  • Hardened GitHub, manifest, snapshot, and CI automation, including expanded
    mocked coverage for GitHub-backed workflows.

  • Updated examples, documentation, pkgdown configuration, and tests for the new
    workflow loading, hook, artifact, and project orchestration behavior.

  • Fixed schema-enforced timestamp parsing in RunQuery() when inputs mix Date
    and POSIXct-like values.

What's Changed

New Contributors

Full Changelog: v1.0.0...v1.1.0

{qcthat} Reports

Milestone

✅ A qcthat issue test matrix with 1 milestone, 17 issues, and 96 tests 
└─█─Milestone: v1.1.0 (17 issues, 96 tests)
  ├─☑️─Issue 72: Add FilterWorkflows() generic with ... filtering interface
  │ ├─✅─FilterWorkflows.character filters single-value meta field (#72)
  │ ├─✅─FilterWorkflows.character filters list-valued meta field (#72)
  │ ├─✅─FilterWorkflows.character supports multiple allowed values (#72)
  │ ├─✅─FilterWorkflows.character is case-insensitive (#72)
  │ ├─✅─FilterWorkflows.character returns empty list for no match (#72)
  │ ├─✅─FilterWorkflows.character excludes workflows missing the field (#72)
  │ ├─✅─FilterWorkflows.character excludes workflows with NULL field value (#72)
  │ ├─✅─FilterWorkflows.logical filters by TRUE (#72)
  │ ├─✅─FilterWorkflows.logical filters by FALSE (#72)
  │ ├─✅─FilterWorkflows.logical field lookup is case-insensitive (#72)
  │ ├─✅─FilterWorkflows.default errors on unsupported value type (#72)
  │ ├─✅─FilterWorkflows.numeric filters by single value (#72)
  │ ├─✅─FilterWorkflows.numeric filters by multiple values (#72)
  │ ├─✅─FilterWorkflows errors on unnamed ... argument (#72)
  │ ├─✅─FilterWorkflows.logical excludes workflows with non-coercible Active value (#72)
  │ ├─✅─FilterWorkflows.integer dispatches to numeric method (#72)
  │ ├─✅─.keep_workflows warns and uses first key when meta has duplicate case-variant keys (#72)
  │ ├─✅─MakeWorkflowList filters via ... character value (#72)
  │ ├─✅─MakeWorkflowList filters via ... logical value (#72)
  │ ├─✅─MakeWorkflowList applies multiple ... filters with AND logic (#72)
  │ └─✅─MakeWorkflowList warns when ... filters leave empty list (#72)
  ├─☑️─Issue 59: Implement `gsm.datasim` load provider
  │ ├─✅─gsm.datasim provider adapts standard single-snapshot data and aliases (#59)
  │ ├─✅─gsm.datasim provider tolerates standard config API drift (#59)
  │ ├─✅─gsm.datasim provider uses latest longitudinal snapshot by default (#59)
  │ ├─✅─gsm.datasim provider validates profile and study_type early (#59)
  │ ├─✅─gsm.datasim provider handles edge-case scalar config values (#59)
  │ ├─✅─gsm.datasim provider requires custom config for custom profile (#59)
  │ └─✅─built-in providers are registered together (#59, #60, #61)
  ├─☑️─Issue 58: Formalize `LoadData` / `SaveData` hook contract
  │ ├─✅─RunWorkflow invokes environment-backed lConfig$LoadData (#58)
  │ ├─✅─RunWorkflow accepts lConfig without data hooks (#58)
  │ ├─✅─RunWorkflow resolves registered load providers (#58)
  │ ├─✅─RunWorkflow resolves registered save providers (#58)
  │ ├─✅─RunWorkflow errors on unknown registered load provider (#58)
  │ ├─✅─RunWorkflow errors on invalid LoadData formals (#58)
  │ ├─✅─RunWorkflow errors on invalid SaveData formals (#58)
  │ └─✅─register_load_provider validates provider signatures (#58)
  ├─☑️─Feature 53: Filter MakeWorkflowList() to active-only
  │ ├─✅─.workflow_path resolves package path (#53)
  │ ├─✅─.workflow_path errors on missing directory (#53)
  │ ├─✅─.discover_yaml_files returns named character vector (#53)
  │ ├─✅─ListWorkflows returns named character vector of yaml paths (#53)
  │ ├─✅─ListWorkflows resolves strPackage (#53)
  │ ├─✅─ListWorkflows errors on invalid strPath (#53)
  │ ├─✅─ListWorkflows bRecursive = FALSE only returns top-level files (#53)
  │ ├─✅─ListWorkflowNames returns AsIs character vector without .yaml (#53)
  │ ├─✅─ListWorkflowNames includes all workflows regardless of Active (#53)
  │ ├─✅─ListWorkflowNames resolves strPackage (#53)
  │ ├─✅─MakeWorkflowList bActiveOnly = TRUE (default) excludes inactive workflows (#53)
  │ ├─✅─MakeWorkflowList bActiveOnly = FALSE includes all workflows (#53)
  │ ├─✅─MakeWorkflowList strNames = NULL with bActiveOnly = FALSE includes all workflows (#53)
  │ ├─✅─.filter_yaml_files NULL strNames returns all (#53)
  │ ├─✅─.filter_yaml_files exact match works (#53)
  │ └─✅─.filter_yaml_files partial match works (#53)
  ├─☑️─Issue 46: [workr] Workflow filename collisions silently overwrite
  │ ├─✅─pull_workflows does not register missing files as collisions (#46)
  │ ├─✅─pull_workflows warns on destination collisions by default (#46)
  │ ├─✅─pull_workflows warns on nested destination collisions in directories (#46)
  │ └─✅─pull_workflows warns and skips file-vs-directory structural collisions (#46)
  ├─☑️─Issue 45: [workr] Snapshot silently skips packages that use inst/workflows (plural)
  │ ├─✅─pull_workflows skips packages with no workflow directory (#45)
  │ └─✅─pull_workflows warns when only inst/workflows exists (#45)
  ├─☑️─Issue 43: [workr] Add testthat coverage for pull_workflows and resolve_package
  │ ├─✅─resolve_package resolves explicit refs via mocked gh api (#43)
  │ ├─✅─resolve_package uses date-based fallback when ref is not provided (#43)
  │ ├─✅─resolve_package errors on malformed gh commit lookup response (#43)
  │ ├─✅─resolve_ref_by_date falls back to default branch when latest release is missing (#43)
  │ ├─✅─gh_list_contents parses directory entries from mocked gh api (#43)
  │ ├─✅─pkgManifest writes manifest outputs with mocked GitHub resolution (#43)
  │ ├─✅─pull_workflows dispatches file and directory entries (#43)
  │ ├─✅─pull_workflow_file skips write when content lookup is Not Found (#43)
  │ └─✅─pull_workflow_file writes decoded file when content is available (#43)
  ├─📥─Issue 23: Improve RunProject data handoff and per-folder configuration
  │ └─(no tests)
  ├─📥─Issue 13: RunStep: dplyr::filter fails because do.call passes named ... arguments
  │ └─(no tests)
  ├─☑️─Issue 67: `RunProject` regression tests + docs
  │ ├─✅─github_artifact load provider ignores GITHUB_RUN_ID and honors latest_success in CI (#67)
  │ ├─✅─github_artifact save provider stamps manifest run_id from GITHUB_RUN_ID (#67)
  │ ├─✅─RunProject without phase configs preserves flat carry-forward behavior (#67)
  │ ├─✅─RunProject supports a four-phase configured project scenario (#67)
  │ ├─✅─RunProject retains fail-fast behavior by default (#67)
  │ ├─✅─RunWorkflow allows LoadData-only and SaveData-only configs (#67)
  │ ├─✅─RunWorkflow fails when spec-declared input is absent (#67)
  │ ├─✅─RunWorkflows retains fail-fast behavior by default (#67)
  │ └─✅─RunWorkflows conti...
Read more

Contributors

jwildfire, lauramaxwell, and 2 other contributors
Assets 4
Loading

v1.0.0

v1.0.0

Choose a tag to compare

View all tags
@jwildfire jwildfire released this 09 Apr 19:11
v1.0.0
9f8cbd6
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

workr 1.0.0

Initial Release

{workr} provides a minimal framework for describing and executing step-by-step R data pipelines using YAML-defined workflows.

Core Features

  • RunStep() — execute a single workflow step
  • RunWorkflow() — execute a full workflow specification (YAML)
  • RunWorkflows() — run multiple workflows in sequence
  • RunProject() — orchestrate multi-phase workflow pipelines (experimental)
  • MakeWorkflowList() — read a folder of YAML workflows into a list
  • RunQuery() — execute database queries via DBI/duckdb
  • loadExample() — load bundled example workflows for learning and testing

Shiny Demo App

Package Manifests

  • pkgManifest() — resolve GitHub packages to pinned versions with manifest.csv and rproject.toml
  • Snapshot branches (ss-*) for reproducible package environments
  • GitHub Actions for nightly manifest updates

GitHub Actions

  • manifest.yaml — reusable workflow for generating manifest artifacts
  • nightly-manifest.yaml — scheduled manifest updates
  • R-CMD-check.yaml — R CMD check CI
  • R-CMD-check-dev.yaml — development R CMD check CI
  • pkgdown-with-examples.yaml — pkgdown site deployment
  • pkgdown-cleanup.yaml — pkgdown cleanup

Documentation

{qcthat} Reports

Milestone

✅ A qcthat issue test matrix with 1 milestone, 3 issues, and 71 tests 
└─█─Milestone: v1.0.0 (3 issues, 71 tests)
  ├─☑️─Issue 26: Improve unit test coverage for v1.0 release
  │ ├─✅─MakeWorkflowList loads YAML and RunStep executes workflow correctly #26
  │ ├─✅─Complex workflow: multi-step loaded from YAML and executed end-to-end #26
  │ ├─✅─Workflow from YAML loads and executes with meta reference #26
  │ ├─✅─loadExample returns input data when initData.R is missing (#9, #26)
  │ ├─✅─loadExample loads initData.R and merges with lData (#9, #26)
  │ ├─✅─loadExample validates initData return type (#9, #26)
  │ ├─✅─loadExample does not create duplicate names on repeated calls (#9, #26)
  │ ├─✅─MakeWorkflowList output is a named list with expected structure #26
  │ ├─✅─MakeWorkflowList returns metadata as expected #26
  │ ├─✅─MakeWorkflowList errors on invalid strPath #26
  │ ├─✅─MakeWorkflowList errors on invalid strPackage #26
  │ ├─✅─MakeWorkflowList filters by strNames #26
  │ ├─✅─MakeWorkflowList bExact filters exactly #26
  │ ├─✅─MakeWorkflowList names list elements by meta$ID #26
  │ ├─✅─MakeWorkflowList sorts by Priority #26
  │ ├─✅─MakeWorkflowList sets default Priority to 0 #26
  │ ├─✅─MakeWorkflowList loads from package #26
  │ ├─✅─MakeWorkflowList bRecursive finds nested workflows #26
  │ ├─✅─RunProject runs all phases in sorted order #26
  │ ├─✅─RunProject passes data between phases #26
  │ ├─✅─RunProject respects strPhases subset and order #26
  │ ├─✅─RunProject errors on missing strPath #26
  │ ├─✅─RunProject errors on missing requested phases #26
  │ ├─✅─RunProject errors on empty project directory #26
  │ ├─✅─RunProject bReturnResult and bKeepInputData pass through #26
  │ ├─✅─RunQuery returns correct result (#26)
  │ ├─✅─RunQuery handles empty df #26
  │ ├─✅─RunQuery handles invalid input #26
  │ ├─✅─RunQuery checks if strQuery contains 'FROM df' #26
  │ ├─✅─RunQuery applies schema appropriately #26
  │ ├─✅─RunQuery applies incomplete schema appropriately #26
  │ ├─✅─RunQuery parses invalid date/times correctly #26
  │ ├─✅─RunQuery requires lColumnMapping when bUseSchema is TRUE #26
  │ ├─✅─RunStep handles lMeta and lData parameters correctly #26
  │ ├─✅─RunStep handles parameters referencing elements within lMeta and lData #26
  │ ├─✅─RunStep passes direct value parameters correctly #26
  │ ├─✅─RunStep passes direct value vector parameters correctly #26
  │ ├─✅─RunStep handles multiple parameters and function invocation correctly #26
  │ ├─✅─RunStep will run a function from a namespace #26
  │ ├─✅─RunStep will run a function without a namespace #26
  │ ├─✅─RunStep will run a function with no parameters #26
  │ ├─✅─RunStep recognizes rlang::expr() string parameters #26
  │ ├─✅─RunStep recognizes expr() shorthand parameters #26
  │ ├─✅─RunStep recognizes rlang::exprs() parameters #26
  │ ├─✅─RunStep recognizes exprs() shorthand parameters #26
  │ ├─✅─RunStep passes lSpec parameter correctly #26
  │ ├─✅─RunWorkflow returns final step result by default #26
  │ ├─✅─RunWorkflow returns full workflow object when bReturnResult = FALSE #26
  │ ├─✅─RunWorkflow bKeepInputData = FALSE removes input data from lData #26
  │ ├─✅─RunWorkflow bKeepInputData = TRUE preserves input data in lData #26
  │ ├─✅─RunWorkflow multi-step preserves intermediate results in lData #26
  │ ├─✅─RunWorkflow invokes lConfig$LoadData #26
  │ ├─✅─RunWorkflow invokes lConfig$SaveData on bReturnResult = TRUE #26
  │ ├─✅─RunWorkflow validates spec when present #26
  │ ├─✅─RunWorkflow runs without spec #26
  │ ├─✅─RunWorkflow runs with YAML files #26
  │ ├─✅─RunWorkflows returns named list #26
  │ ├─✅─RunWorkflows passes prior workflow results as data to later workflows #26
  │ ├─✅─RunWorkflows custom strResultNames #26
  │ ├─✅─RunWorkflows passes lConfig through to RunWorkflow #26
  │ └─✅─RunWorkflows handles empty workflow list #26
  ├─☑️─Issue 25: Differentiate package manifests from execution snapshots
  │ ├─✅─pkgManifest function exists #25
  │ ├─✅─pkgManifest has correct parameters #25
  │ ├─✅─pkgSnapshot function name is deprecated/removed #25
  │ └─✅─pkgListFromManifest references correct function #25
  └─☑️─Issue 19: Remove Legacy /site Explorer From workr
    ├─✅─site/ directory does not exist #19
    ├─✅─site-build.yaml workflow does not exist #19
    ├─✅─site-deploy.yaml workflow does not exist #19
    ├─✅─pkgdown-with-examples.yaml has no Node/explorer steps #19
    ├─✅─_pkgdown.yml has no explorer navbar entry #19
    └─✅─.gitignore has no site/node_modules/ entry #19
# Issue state: 📥 = open, ☑️ = closed (completed), ⛔ = closed (won't fix)
# Test disposition: ✅ = passed, ❌ = failed, 🚫 = skipped
✅ All tests passed

🟢 All issues have at least one test

🙈 5 issues with label "qcthat-nocov" were ignored

Completed Issues

🚫 A qcthat issue test matrix with 1 milestone, 5 issues, and 78 tests 
├─█─Milestone: v1.0.0 (3 issues, 71 tests)
│ ├─☑️─Issue 26: Improve unit test coverage for v1.0 release
│ │ ├─✅─MakeWorkflowList loads YAML and RunStep executes workflow correctly #26
│ │ ├─✅─Complex workflow: multi-step loaded from YAML and executed end-to-end #26
│ │ ├─✅─Workflow from YAML loads and executes with meta reference #26
│ │ ├─✅─loadExample returns input data when initData.R is missing (#9, #26)
│ │ ├─✅─loadExample loads initData.R and merges with lData (#9, #26)
│ │ ├─✅─loadExample validates initData return type (#9, #26)
│ │ ├─✅─loadExample does not create duplicate names on repeated calls (#9, #26)
│ │ ├─✅─MakeWorkflowList output is a named list with expected structure #26
│ │ ├─✅─MakeWorkflowList returns metadata as expected #26
│ │ ├─✅─MakeWorkflowList errors on invalid strPath #26
│ │ ├─✅─MakeWorkflowList errors on invalid strPackage #26
│ │ ├─✅─MakeWorkflowList filters by strNames #26
│ │ ├─✅─MakeWorkflowList bExact filters exactly #26
│ │ ├─✅─MakeWorkflowList names list elements by meta$ID #26
│ │ ├─✅─MakeWorkflowList sorts by Priority #26
│ │ ├─✅─MakeWorkflowList sets default Priority to 0 #26
│ │ ├─✅─MakeWorkflowList loads from package #26
│ │ ├─✅─MakeWorkflowList bRecursive finds nested workflows #26
│ │ ├─✅─RunProject runs all phases in sorted order #26
│ │ ├─✅─RunProject passes data between phases #26
│ │ ├─✅─RunProject respects strPhases subset and order #26
│ │ ├─✅─RunProject errors on missing strPath #26
│ │ ├─✅─RunProject errors on missing requested phases #26
│ │ ├─✅─RunProject errors on empty project directory #26
│ │ ├─✅─RunProject bReturnResult and bKeepInputData pass through #26
│ │ ├─✅─RunQuery returns correct result (#26)
│ │ ├─✅─RunQuery handles empty df #26
│ │ ├─✅─RunQuery handles invalid input #26
│ │ ├─✅─RunQuery checks if strQuery contains 'FROM df' #26
│ │ ├─✅─RunQuery applies schema appropriately #26
│ │ ├─✅─RunQuery applies incomplete schema appropriately #26
│ │ ├─✅─RunQuery parses invalid date/times correctly #26
│ │ ├─✅─RunQuery requires lColumnMapping when bUseSchema is TRUE #26
│ │ ├─✅─RunStep handles lMeta and lData parameters correctly #26
│ │ ├─✅─RunStep handles parameters referencing elements within lMeta and lData #26
│ │ ├─✅─RunStep passes direct value parameters correctly #26
│ │ ├─✅─RunStep passes direct value vector parameters correctly #26
│ │ ├─✅─RunStep handles multiple parameters and function invocation correctly #26
│ │ ├─✅─RunStep will run a function from a namespace #26
│ │ ├─✅─RunStep will run a function without a namespace #26
│ │ ├─✅─RunStep will run a function with no parameters #26
│ │ ├─✅─RunStep recognizes rlang::expr() string parameters #26
│ │ ├─✅─RunStep recognizes expr() shorthand parameters #26
│ │ ├─✅─RunStep recognizes rlang::exprs() parameters #26
│ │ ├─✅─RunStep recognizes exprs() shorthand parameters #26
│ │ ├─✅─RunStep passes lSpec parameter correctly #26
│ │ ├─✅─RunWorkflow returns final step result by default #26
│ │ ├─✅─RunWorkflow returns full workflow object when bReturnResult = FALSE #26
│ │ ├─✅─RunWorkflow bKeepInputData = FALSE removes input data from lData #26
│ │ ├─✅─RunWorkflow bKeepInputData = TRUE preserves input data in lData #26
│ │ ├─✅─RunWorkflow multi-step preserves intermediate results in lData #26
│ │ ├─✅─RunWorkflo...
Read more
Loading