[ENH] Add schema version to Zenodo record keywords (and pipeline config files) #602
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
michellewang
changed the title
Codecov ReportAll modified and coverable lines are covered by tests ✅
... and 1 file with indirect coverage changes 🚀 New features to boost your workflow:
|
michellewang
added a commit
that referenced
this pull request
Apr 29, 2025
* Merge `main` -> `dev-catalog` (#535) * [MAINT] Add dependabot for GitHub actions (#511) * Create dependabot.yml * add "bot" label * fix indentation * change schedule interval to monthly * [MAINT] Fix/update issue labeller (#509) * add new workflow and remove old ones * add required `enable-versioned-regex` field * try to fix labeler.yml * add write permissions for PRs * fix typo? * use pull_request_target insteade of pull_request event trigger * [MAINT] Use Nipoppy bot instead of PAT for projects automation workflow (#508) use bot instead of PAT * [MAINT] Add GitHub Actions workflow to automatically update `CITATION.cff` version after release (#512) * add DOI and update contact/authors in CITATION.cff * rename validation workflow * add initial version of update workflow * update workflow * update names * exclude `nipoppy-bot` PRs from changelog * Bump actions/stale from 9.0.0 to 9.1.0 (#515) Bumps [actions/stale](https://github.com/actions/stale) from 9.0.0 to 9.1.0. - [Release notes](https://github.com/actions/stale/releases) - [Changelog](https://github.com/actions/stale/blob/main/CHANGELOG.md) - [Commits](actions/stale@v9.0.0...v9.1.0) --- updated-dependencies: - dependency-name: actions/stale dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump actions/checkout from 3 to 4 (#514) Bumps [actions/checkout](https://github.com/actions/checkout) from 3 to 4. - [Release notes](https://github.com/actions/checkout/releases) - [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md) - [Commits](actions/checkout@v3...v4) --- updated-dependencies: - dependency-name: actions/checkout dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump actions/setup-python from 4 to 5 (#516) Bumps [actions/setup-python](https://github.com/actions/setup-python) from 4 to 5. - [Release notes](https://github.com/actions/setup-python/releases) - [Commits](actions/setup-python@v4...v5) --- updated-dependencies: - dependency-name: actions/setup-python dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * [MAINT] Bump codecov/codecov-action from 4 to 5 (#513) * Bump codecov/codecov-action from 4 to 5 Bumps [codecov/codecov-action](https://github.com/codecov/codecov-action) from 4 to 5. - [Release notes](https://github.com/codecov/codecov-action/releases) - [Changelog](https://github.com/codecov/codecov-action/blob/main/CHANGELOG.md) - [Commits](codecov/codecov-action@v4...v5) --- updated-dependencies: - dependency-name: codecov/codecov-action dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> * use `files` instead of deprecated `file` * add prefix to PR titles --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Michelle Wang <tomichellewang@gmail.com> * [MAINT] Clean up `.gitignore` (#521) * clean up .gitignore * add entries for Python venvs * [DOC] Add "how-to" guide for dataset initialization (#489) * add "how-to" guide for dataset init * address PR comments * split init doc in multiple pages for non-bids vs. bids start * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix error heading * fix heading and paths * fix header bids.md * update index and beginning of each page * update titles * minor changes --------- Co-authored-by: julia-pfarr <juliakatharina.pfarr@gmx.de> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Julia-Katharina Pfarr <111446107+julia-pfarr@users.noreply.github.com> * [pre-commit.ci] pre-commit autoupdate (#527) updates: - [github.com/pycqa/isort: 6.0.0 → 6.0.1](PyCQA/isort@6.0.0...6.0.1) Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * [DOC] Add a CLI reference page for `nipoppy status` (#529) * add "nipoppy status" page to docs * import StatusWorkflow in workflows module init * [DOC] Add tooltips for glossary terms (#526) try adding tooltip for glossary terms * [DOC] Do not include extractor scripts in autodocs (#525) do not include extractor scripts in autodocs (fix issue #502) * [DOC] Add external link checking for docs build + fix broken links (#524) * add linkcheck in Makefile * add linkcheck to readthedocs config * try to fix readthedocs config * fix broken links * add comment * address PR review comment * increase timeout limit --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: julia-pfarr <juliakatharina.pfarr@gmx.de> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Julia-Katharina Pfarr <111446107+julia-pfarr@users.noreply.github.com> * [ENH] Remove pipeline configs from global config (#503) * add path to pipeline config in layout * remove pipeline configs from global config and put them in pipeline bundle directories instead * update PipelineWorkflow (and subclasses) to handle new config file location * refactor get_pipeline_version to get dpath input instead of list * fix PipelineWorkflow tests * fix PipelineRunner tests * fix bug in _get_pipeline_config() * fix InitWorkflow tests * fix tests for BidsConversionRunner * fix tests for ExtractionRunner * fix TrackerWorkflow test * fix docs (?) * add/update tests * [MAINT] Add dependabot for GitHub actions (#511) * Create dependabot.yml * add "bot" label * fix indentation * change schedule interval to monthly * [MAINT] Fix/update issue labeller (#509) * add new workflow and remove old ones * add required `enable-versioned-regex` field * try to fix labeler.yml * add write permissions for PRs * fix typo? * use pull_request_target insteade of pull_request event trigger * [MAINT] Use Nipoppy bot instead of PAT for projects automation workflow (#508) use bot instead of PAT * [MAINT] Add GitHub Actions workflow to automatically update `CITATION.cff` version after release (#512) * add DOI and update contact/authors in CITATION.cff * rename validation workflow * add initial version of update workflow * update workflow * update names * exclude `nipoppy-bot` PRs from changelog * Bump actions/stale from 9.0.0 to 9.1.0 (#515) Bumps [actions/stale](https://github.com/actions/stale) from 9.0.0 to 9.1.0. - [Release notes](https://github.com/actions/stale/releases) - [Changelog](https://github.com/actions/stale/blob/main/CHANGELOG.md) - [Commits](actions/stale@v9.0.0...v9.1.0) --- updated-dependencies: - dependency-name: actions/stale dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump actions/checkout from 3 to 4 (#514) Bumps [actions/checkout](https://github.com/actions/checkout) from 3 to 4. - [Release notes](https://github.com/actions/checkout/releases) - [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md) - [Commits](actions/checkout@v3...v4) --- updated-dependencies: - dependency-name: actions/checkout dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump actions/setup-python from 4 to 5 (#516) Bumps [actions/setup-python](https://github.com/actions/setup-python) from 4 to 5. - [Release notes](https://github.com/actions/setup-python/releases) - [Commits](actions/setup-python@v4...v5) --- updated-dependencies: - dependency-name: actions/setup-python dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * [MAINT] Bump codecov/codecov-action from 4 to 5 (#513) * Bump codecov/codecov-action from 4 to 5 Bumps [codecov/codecov-action](https://github.com/codecov/codecov-action) from 4 to 5. - [Release notes](https://github.com/codecov/codecov-action/releases) - [Changelog](https://github.com/codecov/codecov-action/blob/main/CHANGELOG.md) - [Commits](codecov/codecov-action@v4...v5) --- updated-dependencies: - dependency-name: codecov/codecov-action dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> * use `files` instead of deprecated `file` * add prefix to PR titles --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Michelle Wang <tomichellewang@gmail.com> * [MAINT] Clean up `.gitignore` (#521) * clean up .gitignore * add entries for Python venvs * [DOC] Add "how-to" guide for dataset initialization (#489) * add "how-to" guide for dataset init * address PR comments * split init doc in multiple pages for non-bids vs. bids start * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix error heading * fix heading and paths * fix header bids.md * update index and beginning of each page * update titles * minor changes --------- Co-authored-by: julia-pfarr <juliakatharina.pfarr@gmx.de> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Julia-Katharina Pfarr <111446107+julia-pfarr@users.noreply.github.com> * [pre-commit.ci] pre-commit autoupdate (#527) updates: - [github.com/pycqa/isort: 6.0.0 → 6.0.1](PyCQA/isort@6.0.0...6.0.1) Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * [DOC] Add a CLI reference page for `nipoppy status` (#529) * add "nipoppy status" page to docs * import StatusWorkflow in workflows module init * [DOC] Add tooltips for glossary terms (#526) try adding tooltip for glossary terms * [DOC] Do not include extractor scripts in autodocs (#525) do not include extractor scripts in autodocs (fix issue #502) * [DOC] Add external link checking for docs build + fix broken links (#524) * add linkcheck in Makefile * add linkcheck to readthedocs config * try to fix readthedocs config * fix broken links * add comment * address PR review comment * increase timeout limit * fix typo * fix paths --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: julia-pfarr <juliakatharina.pfarr@gmx.de> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Julia-Katharina Pfarr <111446107+julia-pfarr@users.noreply.github.com> * [ENH] Add `PIPELINE_VARIABLES` to global config (#548) * add `PIPELINE_VARIABLES` field to global config * add PIPELINE_TYPE: PipelineTypeEnum in pipeline configs * use pipeline variables instead of substitutions for replacement in pipeline files * increase test coverage * address PR review comments * [ENH] Deprecate `DATASET_NAME`, `VISIT_IDS` and `SESSION_IDS` fields from `Config` (#553) * deprecate field from Config class * update StatusWorkflow and fix tests * rename test file * clean up BaseWorkflow tests * clean up DicomReorgWorkflow tests * clean up DoughnutWorkflow tests * fix autodoc warning * increase linkcheck timeout limit * [ENH] Rename pipeline store subdirectories (#561) * rename directories * fix tests * fix docs * [ENH] Force pipeline step config files to only have relative paths (#560) * Remove TRACKER_CONFIG_FILE from ProcPipelineConfig (should only be in ProcPipelineStepConfig) * update validation for BasePipelineStepConfig * update test for invocation/descriptor file fields * add tests for absolute path check and fix bug * update sample config files * refactor pipeline config loading + update relevant tests * handle relative paths in BasePipelineWorkflow * revert pipeline store directory names * fix some tests * fix remaining tests * fix doc warnings * increase test coverage + add test case * fix test * add helper function for getting pipeline bundle path * rename "proc" -> "processing" in test * [ENH] Split `BaseWorkflow` into `BaseWorkflow` and `BaseDatasetWorkflow` (#556) * rename `BaseWorkflow` -> `BaseDatasetWorkflow` * Extract `BaseWorkflow` from `BaseDatasetWorkflow` * move/fix tests * make validate_layout an init argument instead of class attribute * rename _skip_logging -> _skip_logfile * [ENH] Add `nipoppy pipeline validate` command (#558) * [ENH] Add `nipoppy pipeline install` command (#566) * add core validation functions + tests * Remove TRACKER_CONFIG_FILE from ProcPipelineConfig (should only be in ProcPipelineStepConfig) * add CLI command and substitutions * add substitutions test * add CLI test * rename test file * fix docs warnings * add test for PipelineValidateWorkflow * update validation for BasePipelineStepConfig * update test for invocation/descriptor file fields * add tests for absolute path check and fix bug * update sample config files * refactor pipeline config loading + update relevant tests * handle relative paths in BasePipelineWorkflow * revert pipeline store directory names * fix some tests * fix remaining tests * fix doc warnings * increase test coverage + add test case * fix test * adapt for pending relative path change * update log messages * fix test broken by merge * add helper function for getting pipeline bundle path * add base "pipeline install" functionality * Squashed commit of the following: commit e922f12 Merge: 788d3f1 5444535 Author: Michelle Wang <tomichellewang@gmail.com> Date: Mon Mar 24 14:01:07 2025 -0400 Merge remote-tracking branch 'upstream/dev-catalog' into pipeline_step_paths commit 5444535 Author: Michelle Wang <tomichellewang@gmail.com> Date: Sat Mar 22 13:04:20 2025 -0400 [ENH] Rename pipeline store subdirectories (#561) * rename directories * fix tests * fix docs * Squashed commit of the following: commit 853be29 Author: Michelle Wang <tomichellewang@gmail.com> Date: Mon Mar 24 13:53:19 2025 -0400 minor update to CLI * add CLI command and update logging * add variables to global config * add check that pipeline vars are not null * make sure existing fields in global config are not changed * add VARIABLES to default pipeline configs and add test * Squashed commit of the following: commit 34a5ddd Author: Michelle Wang <tomichellewang@gmail.com> Date: Fri Mar 28 11:15:59 2025 -0400 fix CLI test commit 8d17a8b Author: Michelle Wang <tomichellewang@gmail.com> Date: Thu Mar 27 16:51:37 2025 -0400 fix typo commit 7975bb4 Author: Michelle Wang <tomichellewang@gmail.com> Date: Thu Mar 27 16:33:59 2025 -0400 try to fix Pydantic deprecation errors commit c51fe30 Author: Michelle Wang <tomichellewang@gmail.com> Date: Thu Mar 27 16:25:09 2025 -0400 remove things related to dataset layout commit 3229e46 Merge: 9e75942 f437b59 Author: Michelle Wang <tomichellewang@gmail.com> Date: Thu Mar 27 15:59:53 2025 -0400 Merge remote-tracking branch 'upstream/dev-catalog' into 557/pipeline_validate commit 9e75942 Author: Michelle Wang <tomichellewang@gmail.com> Date: Thu Mar 27 11:17:46 2025 -0400 address most PR review comments commit 2e87f8d Merge: 9391ded fac5d2b Author: Michelle Wang <tomichellewang@gmail.com> Date: Thu Mar 27 11:06:40 2025 -0400 Merge remote-tracking branch 'upstream/dev-catalog' into 557/pipeline_validate commit 9391ded Author: Michelle Wang <tomichellewang@gmail.com> Date: Tue Mar 25 14:52:15 2025 -0400 make sure pipeline bundle content is flat list of file (no subdirectories) commit 42e904a Author: Michelle Wang <tomichellewang@gmail.com> Date: Tue Mar 25 12:40:06 2025 -0400 handle edge case with relative parent directories commit 853be29 Author: Michelle Wang <tomichellewang@gmail.com> Date: Mon Mar 24 13:53:19 2025 -0400 minor update to CLI * fix merge * fix PipelineInstallWorkflow * address PR review comments * fix CLI argument order * fix test * fix test again * do not catch exceptions * [ENH] api to download and upload pipelines from zenodo. (#506) * [ENH] api to download files from zenodo. * [FIX] add httpx dependency * [MAINT] remove requests dependency * [FIX] remove tqdm for now * [MAINT] Add dependabot for GitHub actions (#511) * Create dependabot.yml * add "bot" label * fix indentation * change schedule interval to monthly * [MAINT] Fix/update issue labeller (#509) * add new workflow and remove old ones * add required `enable-versioned-regex` field * try to fix labeler.yml * add write permissions for PRs * fix typo? * use pull_request_target insteade of pull_request event trigger * [MAINT] Use Nipoppy bot instead of PAT for projects automation workflow (#508) use bot instead of PAT * [MAINT] Add GitHub Actions workflow to automatically update `CITATION.cff` version after release (#512) * add DOI and update contact/authors in CITATION.cff * rename validation workflow * add initial version of update workflow * update workflow * update names * exclude `nipoppy-bot` PRs from changelog * Bump actions/stale from 9.0.0 to 9.1.0 (#515) Bumps [actions/stale](https://github.com/actions/stale) from 9.0.0 to 9.1.0. - [Release notes](https://github.com/actions/stale/releases) - [Changelog](https://github.com/actions/stale/blob/main/CHANGELOG.md) - [Commits](actions/stale@v9.0.0...v9.1.0) --- updated-dependencies: - dependency-name: actions/stale dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump actions/checkout from 3 to 4 (#514) Bumps [actions/checkout](https://github.com/actions/checkout) from 3 to 4. - [Release notes](https://github.com/actions/checkout/releases) - [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md) - [Commits](actions/checkout@v3...v4) --- updated-dependencies: - dependency-name: actions/checkout dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump actions/setup-python from 4 to 5 (#516) Bumps [actions/setup-python](https://github.com/actions/setup-python) from 4 to 5. - [Release notes](https://github.com/actions/setup-python/releases) - [Commits](actions/setup-python@v4...v5) --- updated-dependencies: - dependency-name: actions/setup-python dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * [MAINT] Bump codecov/codecov-action from 4 to 5 (#513) * Bump codecov/codecov-action from 4 to 5 Bumps [codecov/codecov-action](https://github.com/codecov/codecov-action) from 4 to 5. - [Release notes](https://github.com/codecov/codecov-action/releases) - [Changelog](https://github.com/codecov/codecov-action/blob/main/CHANGELOG.md) - [Commits](codecov/codecov-action@v4...v5) --- updated-dependencies: - dependency-name: codecov/codecov-action dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> * use `files` instead of deprecated `file` * add prefix to PR titles --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Michelle Wang <tomichellewang@gmail.com> * [MAINT] Clean up `.gitignore` (#521) * clean up .gitignore * add entries for Python venvs * [DOC] Add "how-to" guide for dataset initialization (#489) * add "how-to" guide for dataset init * address PR comments * split init doc in multiple pages for non-bids vs. bids start * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix error heading * fix heading and paths * fix header bids.md * update index and beginning of each page * update titles * minor changes --------- Co-authored-by: julia-pfarr <juliakatharina.pfarr@gmx.de> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Julia-Katharina Pfarr <111446107+julia-pfarr@users.noreply.github.com> * [pre-commit.ci] pre-commit autoupdate (#527) updates: - [github.com/pycqa/isort: 6.0.0 → 6.0.1](PyCQA/isort@6.0.0...6.0.1) Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * [DOC] Add a CLI reference page for `nipoppy status` (#529) * add "nipoppy status" page to docs * import StatusWorkflow in workflows module init * [DOC] Add tooltips for glossary terms (#526) try adding tooltip for glossary terms * [DOC] Do not include extractor scripts in autodocs (#525) do not include extractor scripts in autodocs (fix issue #502) * [DOC] Add external link checking for docs build + fix broken links (#524) * add linkcheck in Makefile * add linkcheck to readthedocs config * try to fix readthedocs config * fix broken links * add comment * address PR review comment * increase timeout limit * Update tests/test_zenodo.py Co-authored-by: Michelle Wang <tomichellewang@gmail.com> * Update nipoppy/zenodo.py Co-authored-by: Michelle Wang <tomichellewang@gmail.com> * Improve error message, no longer required access tokekn for download, improve test * add cli for downloading pipelines from zenodo * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update nipoppy/zenodo.py Co-authored-by: Michelle Wang <tomichellewang@gmail.com> * [ENH] Prototype for Zenodo upload * [TMP] started working on tests for upload. * Rename nipoppy pipeline add subcommand to install * add tests for authentication * Test for upload pipeline * Integrate new change for PR - Use new pipeline config for test - rename nipoppy pipeline add to download - Enforce nipoppy keywords on zenodo * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * pre-commit-ci modified the zenodo.json file for the example pipeline. * [no ci] cleanup gitignore conflict * Triger CI after adding secret to GH * Update nipoppy/cli.py Co-authored-by: Michelle Wang <tomichellewang@gmail.com> * Update nipoppy/zenodo.py Co-authored-by: Michelle Wang <tomichellewang@gmail.com> * fix pr comments * Seperated ZenodoAPI from workflow + tests for download workflow. * test get zenodo metadata from local pipeline. * test for zenodo upload workflow * rename zenodo_id to record_id * Add ZENODO as github variable * Remove and Prevent log file in history * Update nipoppy/workflows/zenodo.py Co-authored-by: Michelle Wang <tomichellewang@gmail.com> * pr comments * Fix PR comments * Fix env var name for zenodo ID * Handle empty ZENODO_TOKEN GitHub secret seems to be empty on PR * Update nipoppy/zenodo_api.py Co-authored-by: Michelle Wang <tomichellewang@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix after merge * try to fix failing test (Zenodo record not matching sample pipeline anymore) * fix new failing test * pre-commit fix * update test data directory name * fix again --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: Michelle Wang <tomichellewang@gmail.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: julia-pfarr <juliakatharina.pfarr@gmx.de> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Julia-Katharina Pfarr <111446107+julia-pfarr@users.noreply.github.com> * fix tests * [FIX] Fix Zenodo keywords not being set correctly (#598) * fix tests * remove docs file * use "subjects" metadata field instead of "keywords" * use "Nipoppy user" as default creator name * fix test * fix other tests * add pipeline name and version as keywords * [ENH] Add schema version to Zenodo record keywords (and pipeline config files) (#602) * fix tests * remove docs file * use "subjects" metadata field instead of "keywords" * use "Nipoppy user" as default creator name * fix test * fix other tests * add pipeline name and version as keywords * add mandatory schema version field and fix tests * [ENH] Implement `nipoppy pipeline search` (#599) * add search function and tests * add debug message * add search workflow and tests * add CLI command and test * minor changes * minor improvements * increase test coverage * address PR review comments * [ENH] Merge `nipoppy pipeline download` into `nipoppy pipeline install` (#594) * Merge `main` -> `dev-catalog` (#563) * [MAINT] Add dependabot for GitHub actions (#511) * Create dependabot.yml * add "bot" label * fix indentation * change schedule interval to monthly * [MAINT] Fix/update issue labeller (#509) * add new workflow and remove old ones * add required `enable-versioned-regex` field * try to fix labeler.yml * add write permissions for PRs * fix typo? * use pull_request_target insteade of pull_request event trigger * [MAINT] Use Nipoppy bot instead of PAT for projects automation workflow (#508) use bot instead of PAT * [MAINT] Add GitHub Actions workflow to automatically update `CITATION.cff` version after release (#512) * add DOI and update contact/authors in CITATION.cff * rename validation workflow * add initial version of update workflow * update workflow * update names * exclude `nipoppy-bot` PRs from changelog * Bump actions/stale from 9.0.0 to 9.1.0 (#515) Bumps [actions/stale](https://github.com/actions/stale) from 9.0.0 to 9.1.0. - [Release notes](https://github.com/actions/stale/releases) - [Changelog](https://github.com/actions/stale/blob/main/CHANGELOG.md) - [Commits](actions/stale@v9.0.0...v9.1.0) --- updated-dependencies: - dependency-name: actions/stale dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump actions/checkout from 3 to 4 (#514) Bumps [actions/checkout](https://github.com/actions/checkout) from 3 to 4. - [Release notes](https://github.com/actions/checkout/releases) - [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md) - [Commits](actions/checkout@v3...v4) --- updated-dependencies: - dependency-name: actions/checkout dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump actions/setup-python from 4 to 5 (#516) Bumps [actions/setup-python](https://github.com/actions/setup-python) from 4 to 5. - [Release notes](https://github.com/actions/setup-python/releases) - [Commits](actions/setup-python@v4...v5) --- updated-dependencies: - dependency-name: actions/setup-python dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * [MAINT] Bump codecov/codecov-action from 4 to 5 (#513) * Bump codecov/codecov-action from 4 to 5 Bumps [codecov/codecov-action](https://github.com/codecov/codecov-action) from 4 to 5. - [Release notes](https://github.com/codecov/codecov-action/releases) - [Changelog](https://github.com/codecov/codecov-action/blob/main/CHANGELOG.md) - [Commits](codecov/codecov-action@v4...v5) --- updated-dependencies: - dependency-name: codecov/codecov-action dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> * use `files` instead of deprecated `file` * add prefix to PR titles --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Michelle Wang <tomichellewang@gmail.com> * [MAINT] Clean up `.gitignore` (#521) * clean up .gitignore * add entries for Python venvs * [DOC] Add "how-to" guide for dataset initialization (#489) * add "how-to" guide for dataset init * address PR comments * split init doc in multiple pages for non-bids vs. bids start * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix error heading * fix heading and paths * fix header bids.md * update index and beginning of each page * update titles * minor changes --------- Co-authored-by: julia-pfarr <juliakatharina.pfarr@gmx.de> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Julia-Katharina Pfarr <111446107+julia-pfarr@users.noreply.github.com> * [pre-commit.ci] pre-commit autoupdate (#527) updates: - [github.com/pycqa/isort: 6.0.0 → 6.0.1](PyCQA/isort@6.0.0...6.0.1) Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * [DOC] Add a CLI reference page for `nipoppy status` (#529) * add "nipoppy status" page to docs * import StatusWorkflow in workflows module init * [DOC] Add tooltips for glossary terms (#526) try adding tooltip for glossary terms * [DOC] Do not include extractor scripts in autodocs (#525) do not include extractor scripts in autodocs (fix issue #502) * [DOC] Add external link checking for docs build + fix broken links (#524) * add linkcheck in Makefile * add linkcheck to readthedocs config * try to fix readthedocs config * fix broken links * add comment * address PR review comment * increase timeout limit * [MAINT] Add CI workflow for Read the docs preview link (#536) add Read the docs preview workflow * [ENH] Add `bids-validator` `2.0.3` (#534) * add `bids-validator` 2.0.3 * simplify container config * expose schema version field (use latest by default) in invocation * [ENH] group options in cli help. (#491) * [ENH] group options in cli help. * remove duplicate entry for --force flag * Update nipoppy/cli.py Co-authored-by: Michelle Wang <tomichellewang@gmail.com> * Remove redundant "options" suffix --------- Co-authored-by: Michelle Wang <tomichellewang@gmail.com> * [ENH] Remove Boutiques descriptor/invocation file content dumps (#542) * add `quiet` arg to `run_command` * silence bosh command * also overwrite subprocess error message * update error messages * fix regex in test * [ENH] Print container-pulling command and change global config container substitution (#540) * add test * add "apptainer pull" command to error message if URI is available * update default container store path * update container pull message * fix exception message in test * [BUG] Add validation for non-alphanumeric characters in `participant_id` and `session_id` in manifest (#546) * add failing tests * add validation * add test case with non-alphanumeric character in `visit_id` * clarify docstrings * update container path substitution pattern * [ENH] Improve `fs_stats` extractor pipeline (#550) * fix escape sequence warnings * make script crash if no subjects found + add troubleshooting suggestions for when FreeSurfer stats commands fail * make the default invocation have `"verbose": true` * rename script.py to fs_stats.py * update script * bump version to 0.2.1 * [MAINT] Fix Pydantic `model_fields` deprecations (#576) fix Pydantic `model_fields` deprecations * fix fs_stats version change * fix tests * fix test --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: julia-pfarr <juliakatharina.pfarr@gmx.de> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Julia-Katharina Pfarr <111446107+julia-pfarr@users.noreply.github.com> Co-authored-by: Mathieu Dugré <16450132+mathdugre@users.noreply.github.com> * more Zenodo workflow file into pipeline_store subdirectory * merge pipeline download with pipeline install * move tests * fix CLI tests * fix tests failing in 3.9 * increase test coverage * improve error message when Zenodo record doesn't have expected config file * address PR review comments * fix Zenodo ID for tests --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: julia-pfarr <juliakatharina.pfarr@gmx.de> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Julia-Katharina Pfarr <111446107+julia-pfarr@users.noreply.github.com> Co-authored-by: Mathieu Dugré <16450132+mathdugre@users.noreply.github.com> * [ENH] Automatically impute Zenodo record creator information (#601) * allow record_id in upload_pipeline() to start with "zenodo." * dynamically get creator info if user does not provide it * fix test * [ENH] Add `nipoppy pipeline list` command (#570) * Remove TRACKER_CONFIG_FILE from ProcPipelineConfig (should only be in ProcPipelineStepConfig) * update validation for BasePipelineStepConfig * update test for invocation/descriptor file fields * add tests for absolute path check and fix bug * update sample config files * refactor pipeline config loading + update relevant tests * handle relative paths in BasePipelineWorkflow * revert pipeline store directory names * fix some tests * fix remaining tests * fix doc warnings * increase test coverage + add test case * fix test * add helper function for getting pipeline bundle path * refactor test function args * add workflow and tests * fix failing tests * add CLI and test * switch to BaseDatasetWorkflow * update type annotation and refactor variable name * fix tests * fix tests * use defaultdict * reorder CLI subcommands * [BUG] delete --regenerate flag from rich_click.OPTION_GROUPS (#609) * delete --regenerate flag from rich_click.OPTION_GROUPS * move option groups on top of file * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Fixing merge conflict to merge dev-catalog into main (#612) * Implement RABIES (#264) * Add and edits to global_configs file * delete unused global configs file * created rabies boutiques descriptor * add bid dir and output to boutique descriptor * auto generate boutique for RABIES * adding optional arg descriptors for RABIES * all optional args added to rabies boutiques json * create boutiques descriptor for each Rabies command * add output and input for RAbies analysis boutique descriptor * finish primary section of preprocess boutique descriptors * finish boutique desc for preprocess * fix boutiques descriptors * correct value key quotations * fix value key * add optional desc to confound correct boutique file * adding boutique desc for rabies analysis * added input and output to confound correction and analysis * generate invocation schema for rabies preprocess * generate innvocation schemas for analysis and confound-correction * remove extra data_diagnosis tag * correcting types in descriptors * renameupdate descriptors est Rabies version * fix rabies preprocess descriptor * remove default values from confound_correction * describe certain tags as type FLAG * correct confound-correction with Flag type * fix command line order in analysis descriptor * remove analysis default values * set confound input default a bids dir * fix descriptor typo * reorder confound_correciton descriptor and remove data_diagnosis tag * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * conf_list now list object in descriptor * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * correct spelling to make codespell happy :) * changed confound-correction default input value to derivatives * added sample innvocation files for RABIES steps * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add RABIES to sample global config files * edits to RABIES in sample configs * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * addition or RABIES to sample global config files * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * remove scan_list arg from RABIES analysis * fix verbose descriptions * fix uri for rabies sample config * empty tracker config for rabies * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * rename tracker file * set preprocess invocation file to defaults * added default local threads * replace variables with default values * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add descriptor files again * add rabies files to their own sample_pipelines folders * add descriptor and invocation to rabies sample pipeline files * add rabies config to global config files * remove out of data 0.4.8 rabies files * add rabies parameters to test config file * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add rabies tracker config file * make edits to descriptor via boutiques suggestions * fix pathing for rabies steps config * set proper naming convention for rabies pipeline files * full refactor to most up to date rabies descriptors * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix input/output dir paths in rabies invocation files * fix codespell issues/typos * remove default input/output for descriptor, use pipeline_output in invocation * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix more codespell typos * add desciptions to all descriptor files from rabies docs * code spell fix * fix command-line-flag for verbose * add list attribute to proper analysis arguments * add value choices for detrending_order arg * adding select value choices and list attribute to args when needed * fix verbose description * remove deprecated nipoppy_cli folder * fix brackets in analysis invocation * add rabies pipelines steps to test_pipeline_runner test * input/output paths now stated in invocation files * Update nipoppy/data/examples/sample_pipelines/rabies-0.5.1/tracker_config.json Co-authored-by: Michelle Wang <tomichellewang@gmail.com> * set data_diagnosis tag as true by default * add data_diagnosis files from analysis step as tracked outputs * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update nipoppy/data/examples/sample_global_config-all_pipelines.json Co-authored-by: Michelle Wang <tomichellewang@gmail.com> * add version number to rabies container_info * Update nipoppy/data/examples/sample_global_config-all_pipelines.json Co-authored-by: Michelle Wang <tomichellewang@gmail.com> * change input_dir to bids_dir for preprocess invocation * add analysis and generate pybids status to rabies steps * remove initial `/` from track config paths * switch analysis step to participant_session level, move tracker_config_file notation into analysis step * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * moving tracker to strictly analysis step * adding separate tracking files and steps in config * create tracker config files for each step * add confound correction image path to track * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add separate tracker configs for each step * tracking anatomical image files from preprocess outputs * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * rename all tracker config files to include step name * remove extra [ from preprocess invocation files * remove extra [ from confound-correciton invocation files * remove redundant tracker descriptor/innvocation files * remove descriptor and invocation lines from tracker steps in global config * remove unused tracker config files * add * to filepaths to improve flexibility * set "acq-rare" to * in preprocess tracker config paths * Add session number variable to session section of paths * remove non-image files from tracking * fix ses- and sub- labeling in tracker config file * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * remove ses- and sub- tags from preprocess tracking file paths --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Michelle Wang <tomichellewang@gmail.com> * Add back rabies config. + Move to sample_pipelines/preprocessing/rabies-0.5.1 * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add shcema version * Add pipeline type --------- Co-authored-by: David Roper <60201980+david-roper@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Michelle Wang <tomichellewang@gmail.com> * Ignore zenodo_api.py Low coverage on response error code. --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: Michelle Wang <tomichellewang@gmail.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: julia-pfarr <juliakatharina.pfarr@gmx.de> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Julia-Katharina Pfarr <111446107+julia-pfarr@users.noreply.github.com> Co-authored-by: David Roper <60201980+david-roper@users.noreply.github.com>
michellewang
added a commit
that referenced
this pull request
Apr 29, 2025
* Merge `main` -> `dev-catalog` (#535) * [MAINT] Add dependabot for GitHub actions (#511) * Create dependabot.yml * add "bot" label * fix indentation * change schedule interval to monthly * [MAINT] Fix/update issue labeller (#509) * add new workflow and remove old ones * add required `enable-versioned-regex` field * try to fix labeler.yml * add write permissions for PRs * fix typo? * use pull_request_target insteade of pull_request event trigger * [MAINT] Use Nipoppy bot instead of PAT for projects automation workflow (#508) use bot instead of PAT * [MAINT] Add GitHub Actions workflow to automatically update `CITATION.cff` version after release (#512) * add DOI and update contact/authors in CITATION.cff * rename validation workflow * add initial version of update workflow * update workflow * update names * exclude `nipoppy-bot` PRs from changelog * Bump actions/stale from 9.0.0 to 9.1.0 (#515) Bumps [actions/stale](https://github.com/actions/stale) from 9.0.0 to 9.1.0. - [Release notes](https://github.com/actions/stale/releases) - [Changelog](https://github.com/actions/stale/blob/main/CHANGELOG.md) - [Commits](actions/stale@v9.0.0...v9.1.0) --- updated-dependencies: - dependency-name: actions/stale dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump actions/checkout from 3 to 4 (#514) Bumps [actions/checkout](https://github.com/actions/checkout) from 3 to 4. - [Release notes](https://github.com/actions/checkout/releases) - [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md) - [Commits](actions/checkout@v3...v4) --- updated-dependencies: - dependency-name: actions/checkout dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump actions/setup-python from 4 to 5 (#516) Bumps [actions/setup-python](https://github.com/actions/setup-python) from 4 to 5. - [Release notes](https://github.com/actions/setup-python/releases) - [Commits](actions/setup-python@v4...v5) --- updated-dependencies: - dependency-name: actions/setup-python dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * [MAINT] Bump codecov/codecov-action from 4 to 5 (#513) * Bump codecov/codecov-action from 4 to 5 Bumps [codecov/codecov-action](https://github.com/codecov/codecov-action) from 4 to 5. - [Release notes](https://github.com/codecov/codecov-action/releases) - [Changelog](https://github.com/codecov/codecov-action/blob/main/CHANGELOG.md) - [Commits](codecov/codecov-action@v4...v5) --- updated-dependencies: - dependency-name: codecov/codecov-action dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> * use `files` instead of deprecated `file` * add prefix to PR titles --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Michelle Wang <tomichellewang@gmail.com> * [MAINT] Clean up `.gitignore` (#521) * clean up .gitignore * add entries for Python venvs * [DOC] Add "how-to" guide for dataset initialization (#489) * add "how-to" guide for dataset init * address PR comments * split init doc in multiple pages for non-bids vs. bids start * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix error heading * fix heading and paths * fix header bids.md * update index and beginning of each page * update titles * minor changes --------- Co-authored-by: julia-pfarr <juliakatharina.pfarr@gmx.de> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Julia-Katharina Pfarr <111446107+julia-pfarr@users.noreply.github.com> * [pre-commit.ci] pre-commit autoupdate (#527) updates: - [github.com/pycqa/isort: 6.0.0 → 6.0.1](PyCQA/isort@6.0.0...6.0.1) Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * [DOC] Add a CLI reference page for `nipoppy status` (#529) * add "nipoppy status" page to docs * import StatusWorkflow in workflows module init * [DOC] Add tooltips for glossary terms (#526) try adding tooltip for glossary terms * [DOC] Do not include extractor scripts in autodocs (#525) do not include extractor scripts in autodocs (fix issue #502) * [DOC] Add external link checking for docs build + fix broken links (#524) * add linkcheck in Makefile * add linkcheck to readthedocs config * try to fix readthedocs config * fix broken links * add comment * address PR review comment * increase timeout limit --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: julia-pfarr <juliakatharina.pfarr@gmx.de> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Julia-Katharina Pfarr <111446107+julia-pfarr@users.noreply.github.com> * [ENH] Remove pipeline configs from global config (#503) * add path to pipeline config in layout * remove pipeline configs from global config and put them in pipeline bundle directories instead * update PipelineWorkflow (and subclasses) to handle new config file location * refactor get_pipeline_version to get dpath input instead of list * fix PipelineWorkflow tests * fix PipelineRunner tests * fix bug in _get_pipeline_config() * fix InitWorkflow tests * fix tests for BidsConversionRunner * fix tests for ExtractionRunner * fix TrackerWorkflow test * fix docs (?) * add/update tests * [MAINT] Add dependabot for GitHub actions (#511) * Create dependabot.yml * add "bot" label * fix indentation * change schedule interval to monthly * [MAINT] Fix/update issue labeller (#509) * add new workflow and remove old ones * add required `enable-versioned-regex` field * try to fix labeler.yml * add write permissions for PRs * fix typo? * use pull_request_target insteade of pull_request event trigger * [MAINT] Use Nipoppy bot instead of PAT for projects automation workflow (#508) use bot instead of PAT * [MAINT] Add GitHub Actions workflow to automatically update `CITATION.cff` version after release (#512) * add DOI and update contact/authors in CITATION.cff * rename validation workflow * add initial version of update workflow * update workflow * update names * exclude `nipoppy-bot` PRs from changelog * Bump actions/stale from 9.0.0 to 9.1.0 (#515) Bumps [actions/stale](https://github.com/actions/stale) from 9.0.0 to 9.1.0. - [Release notes](https://github.com/actions/stale/releases) - [Changelog](https://github.com/actions/stale/blob/main/CHANGELOG.md) - [Commits](actions/stale@v9.0.0...v9.1.0) --- updated-dependencies: - dependency-name: actions/stale dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump actions/checkout from 3 to 4 (#514) Bumps [actions/checkout](https://github.com/actions/checkout) from 3 to 4. - [Release notes](https://github.com/actions/checkout/releases) - [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md) - [Commits](actions/checkout@v3...v4) --- updated-dependencies: - dependency-name: actions/checkout dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump actions/setup-python from 4 to 5 (#516) Bumps [actions/setup-python](https://github.com/actions/setup-python) from 4 to 5. - [Release notes](https://github.com/actions/setup-python/releases) - [Commits](actions/setup-python@v4...v5) --- updated-dependencies: - dependency-name: actions/setup-python dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * [MAINT] Bump codecov/codecov-action from 4 to 5 (#513) * Bump codecov/codecov-action from 4 to 5 Bumps [codecov/codecov-action](https://github.com/codecov/codecov-action) from 4 to 5. - [Release notes](https://github.com/codecov/codecov-action/releases) - [Changelog](https://github.com/codecov/codecov-action/blob/main/CHANGELOG.md) - [Commits](codecov/codecov-action@v4...v5) --- updated-dependencies: - dependency-name: codecov/codecov-action dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> * use `files` instead of deprecated `file` * add prefix to PR titles --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Michelle Wang <tomichellewang@gmail.com> * [MAINT] Clean up `.gitignore` (#521) * clean up .gitignore * add entries for Python venvs * [DOC] Add "how-to" guide for dataset initialization (#489) * add "how-to" guide for dataset init * address PR comments * split init doc in multiple pages for non-bids vs. bids start * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix error heading * fix heading and paths * fix header bids.md * update index and beginning of each page * update titles * minor changes --------- Co-authored-by: julia-pfarr <juliakatharina.pfarr@gmx.de> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Julia-Katharina Pfarr <111446107+julia-pfarr@users.noreply.github.com> * [pre-commit.ci] pre-commit autoupdate (#527) updates: - [github.com/pycqa/isort: 6.0.0 → 6.0.1](PyCQA/isort@6.0.0...6.0.1) Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * [DOC] Add a CLI reference page for `nipoppy status` (#529) * add "nipoppy status" page to docs * import StatusWorkflow in workflows module init * [DOC] Add tooltips for glossary terms (#526) try adding tooltip for glossary terms * [DOC] Do not include extractor scripts in autodocs (#525) do not include extractor scripts in autodocs (fix issue #502) * [DOC] Add external link checking for docs build + fix broken links (#524) * add linkcheck in Makefile * add linkcheck to readthedocs config * try to fix readthedocs config * fix broken links * add comment * address PR review comment * increase timeout limit * fix typo * fix paths --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: julia-pfarr <juliakatharina.pfarr@gmx.de> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Julia-Katharina Pfarr <111446107+julia-pfarr@users.noreply.github.com> * [ENH] Add `PIPELINE_VARIABLES` to global config (#548) * add `PIPELINE_VARIABLES` field to global config * add PIPELINE_TYPE: PipelineTypeEnum in pipeline configs * use pipeline variables instead of substitutions for replacement in pipeline files * increase test coverage * address PR review comments * [ENH] Deprecate `DATASET_NAME`, `VISIT_IDS` and `SESSION_IDS` fields from `Config` (#553) * deprecate field from Config class * update StatusWorkflow and fix tests * rename test file * clean up BaseWorkflow tests * clean up DicomReorgWorkflow tests * clean up DoughnutWorkflow tests * fix autodoc warning * increase linkcheck timeout limit * [ENH] Rename pipeline store subdirectories (#561) * rename directories * fix tests * fix docs * [ENH] Force pipeline step config files to only have relative paths (#560) * Remove TRACKER_CONFIG_FILE from ProcPipelineConfig (should only be in ProcPipelineStepConfig) * update validation for BasePipelineStepConfig * update test for invocation/descriptor file fields * add tests for absolute path check and fix bug * update sample config files * refactor pipeline config loading + update relevant tests * handle relative paths in BasePipelineWorkflow * revert pipeline store directory names * fix some tests * fix remaining tests * fix doc warnings * increase test coverage + add test case * fix test * add helper function for getting pipeline bundle path * rename "proc" -> "processing" in test * [ENH] Split `BaseWorkflow` into `BaseWorkflow` and `BaseDatasetWorkflow` (#556) * rename `BaseWorkflow` -> `BaseDatasetWorkflow` * Extract `BaseWorkflow` from `BaseDatasetWorkflow` * move/fix tests * make validate_layout an init argument instead of class attribute * rename _skip_logging -> _skip_logfile * [ENH] Add `nipoppy pipeline validate` command (#558) * [ENH] Add `nipoppy pipeline install` command (#566) * add core validation functions + tests * Remove TRACKER_CONFIG_FILE from ProcPipelineConfig (should only be in ProcPipelineStepConfig) * add CLI command and substitutions * add substitutions test * add CLI test * rename test file * fix docs warnings * add test for PipelineValidateWorkflow * update validation for BasePipelineStepConfig * update test for invocation/descriptor file fields * add tests for absolute path check and fix bug * update sample config files * refactor pipeline config loading + update relevant tests * handle relative paths in BasePipelineWorkflow * revert pipeline store directory names * fix some tests * fix remaining tests * fix doc warnings * increase test coverage + add test case * fix test * adapt for pending relative path change * update log messages * fix test broken by merge * add helper function for getting pipeline bundle path * add base "pipeline install" functionality * Squashed commit of the following: commit e922f12 Merge: 788d3f1 5444535 Author: Michelle Wang <tomichellewang@gmail.com> Date: Mon Mar 24 14:01:07 2025 -0400 Merge remote-tracking branch 'upstream/dev-catalog' into pipeline_step_paths commit 5444535 Author: Michelle Wang <tomichellewang@gmail.com> Date: Sat Mar 22 13:04:20 2025 -0400 [ENH] Rename pipeline store subdirectories (#561) * rename directories * fix tests * fix docs * Squashed commit of the following: commit 853be29 Author: Michelle Wang <tomichellewang@gmail.com> Date: Mon Mar 24 13:53:19 2025 -0400 minor update to CLI * add CLI command and update logging * add variables to global config * add check that pipeline vars are not null * make sure existing fields in global config are not changed * add VARIABLES to default pipeline configs and add test * Squashed commit of the following: commit 34a5ddd Author: Michelle Wang <tomichellewang@gmail.com> Date: Fri Mar 28 11:15:59 2025 -0400 fix CLI test commit 8d17a8b Author: Michelle Wang <tomichellewang@gmail.com> Date: Thu Mar 27 16:51:37 2025 -0400 fix typo commit 7975bb4 Author: Michelle Wang <tomichellewang@gmail.com> Date: Thu Mar 27 16:33:59 2025 -0400 try to fix Pydantic deprecation errors commit c51fe30 Author: Michelle Wang <tomichellewang@gmail.com> Date: Thu Mar 27 16:25:09 2025 -0400 remove things related to dataset layout commit 3229e46 Merge: 9e75942 f437b59 Author: Michelle Wang <tomichellewang@gmail.com> Date: Thu Mar 27 15:59:53 2025 -0400 Merge remote-tracking branch 'upstream/dev-catalog' into 557/pipeline_validate commit 9e75942 Author: Michelle Wang <tomichellewang@gmail.com> Date: Thu Mar 27 11:17:46 2025 -0400 address most PR review comments commit 2e87f8d Merge: 9391ded fac5d2b Author: Michelle Wang <tomichellewang@gmail.com> Date: Thu Mar 27 11:06:40 2025 -0400 Merge remote-tracking branch 'upstream/dev-catalog' into 557/pipeline_validate commit 9391ded Author: Michelle Wang <tomichellewang@gmail.com> Date: Tue Mar 25 14:52:15 2025 -0400 make sure pipeline bundle content is flat list of file (no subdirectories) commit 42e904a Author: Michelle Wang <tomichellewang@gmail.com> Date: Tue Mar 25 12:40:06 2025 -0400 handle edge case with relative parent directories commit 853be29 Author: Michelle Wang <tomichellewang@gmail.com> Date: Mon Mar 24 13:53:19 2025 -0400 minor update to CLI * fix merge * fix PipelineInstallWorkflow * address PR review comments * fix CLI argument order * fix test * fix test again * do not catch exceptions * [ENH] api to download and upload pipelines from zenodo. (#506) * [ENH] api to download files from zenodo. * [FIX] add httpx dependency * [MAINT] remove requests dependency * [FIX] remove tqdm for now * [MAINT] Add dependabot for GitHub actions (#511) * Create dependabot.yml * add "bot" label * fix indentation * change schedule interval to monthly * [MAINT] Fix/update issue labeller (#509) * add new workflow and remove old ones * add required `enable-versioned-regex` field * try to fix labeler.yml * add write permissions for PRs * fix typo? * use pull_request_target insteade of pull_request event trigger * [MAINT] Use Nipoppy bot instead of PAT for projects automation workflow (#508) use bot instead of PAT * [MAINT] Add GitHub Actions workflow to automatically update `CITATION.cff` version after release (#512) * add DOI and update contact/authors in CITATION.cff * rename validation workflow * add initial version of update workflow * update workflow * update names * exclude `nipoppy-bot` PRs from changelog * Bump actions/stale from 9.0.0 to 9.1.0 (#515) Bumps [actions/stale](https://github.com/actions/stale) from 9.0.0 to 9.1.0. - [Release notes](https://github.com/actions/stale/releases) - [Changelog](https://github.com/actions/stale/blob/main/CHANGELOG.md) - [Commits](actions/stale@v9.0.0...v9.1.0) --- updated-dependencies: - dependency-name: actions/stale dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump actions/checkout from 3 to 4 (#514) Bumps [actions/checkout](https://github.com/actions/checkout) from 3 to 4. - [Release notes](https://github.com/actions/checkout/releases) - [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md) - [Commits](actions/checkout@v3...v4) --- updated-dependencies: - dependency-name: actions/checkout dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump actions/setup-python from 4 to 5 (#516) Bumps [actions/setup-python](https://github.com/actions/setup-python) from 4 to 5. - [Release notes](https://github.com/actions/setup-python/releases) - [Commits](actions/setup-python@v4...v5) --- updated-dependencies: - dependency-name: actions/setup-python dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * [MAINT] Bump codecov/codecov-action from 4 to 5 (#513) * Bump codecov/codecov-action from 4 to 5 Bumps [codecov/codecov-action](https://github.com/codecov/codecov-action) from 4 to 5. - [Release notes](https://github.com/codecov/codecov-action/releases) - [Changelog](https://github.com/codecov/codecov-action/blob/main/CHANGELOG.md) - [Commits](codecov/codecov-action@v4...v5) --- updated-dependencies: - dependency-name: codecov/codecov-action dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> * use `files` instead of deprecated `file` * add prefix to PR titles --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Michelle Wang <tomichellewang@gmail.com> * [MAINT] Clean up `.gitignore` (#521) * clean up .gitignore * add entries for Python venvs * [DOC] Add "how-to" guide for dataset initialization (#489) * add "how-to" guide for dataset init * address PR comments * split init doc in multiple pages for non-bids vs. bids start * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix error heading * fix heading and paths * fix header bids.md * update index and beginning of each page * update titles * minor changes --------- Co-authored-by: julia-pfarr <juliakatharina.pfarr@gmx.de> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Julia-Katharina Pfarr <111446107+julia-pfarr@users.noreply.github.com> * [pre-commit.ci] pre-commit autoupdate (#527) updates: - [github.com/pycqa/isort: 6.0.0 → 6.0.1](PyCQA/isort@6.0.0...6.0.1) Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * [DOC] Add a CLI reference page for `nipoppy status` (#529) * add "nipoppy status" page to docs * import StatusWorkflow in workflows module init * [DOC] Add tooltips for glossary terms (#526) try adding tooltip for glossary terms * [DOC] Do not include extractor scripts in autodocs (#525) do not include extractor scripts in autodocs (fix issue #502) * [DOC] Add external link checking for docs build + fix broken links (#524) * add linkcheck in Makefile * add linkcheck to readthedocs config * try to fix readthedocs config * fix broken links * add comment * address PR review comment * increase timeout limit * Update tests/test_zenodo.py Co-authored-by: Michelle Wang <tomichellewang@gmail.com> * Update nipoppy/zenodo.py Co-authored-by: Michelle Wang <tomichellewang@gmail.com> * Improve error message, no longer required access tokekn for download, improve test * add cli for downloading pipelines from zenodo * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update nipoppy/zenodo.py Co-authored-by: Michelle Wang <tomichellewang@gmail.com> * [ENH] Prototype for Zenodo upload * [TMP] started working on tests for upload. * Rename nipoppy pipeline add subcommand to install * add tests for authentication * Test for upload pipeline * Integrate new change for PR - Use new pipeline config for test - rename nipoppy pipeline add to download - Enforce nipoppy keywords on zenodo * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * pre-commit-ci modified the zenodo.json file for the example pipeline. * [no ci] cleanup gitignore conflict * Triger CI after adding secret to GH * Update nipoppy/cli.py Co-authored-by: Michelle Wang <tomichellewang@gmail.com> * Update nipoppy/zenodo.py Co-authored-by: Michelle Wang <tomichellewang@gmail.com> * fix pr comments * Seperated ZenodoAPI from workflow + tests for download workflow. * test get zenodo metadata from local pipeline. * test for zenodo upload workflow * rename zenodo_id to record_id * Add ZENODO as github variable * Remove and Prevent log file in history * Update nipoppy/workflows/zenodo.py Co-authored-by: Michelle Wang <tomichellewang@gmail.com> * pr comments * Fix PR comments * Fix env var name for zenodo ID * Handle empty ZENODO_TOKEN GitHub secret seems to be empty on PR * Update nipoppy/zenodo_api.py Co-authored-by: Michelle Wang <tomichellewang@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix after merge * try to fix failing test (Zenodo record not matching sample pipeline anymore) * fix new failing test * pre-commit fix * update test data directory name * fix again --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: Michelle Wang <tomichellewang@gmail.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: julia-pfarr <juliakatharina.pfarr@gmx.de> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Julia-Katharina Pfarr <111446107+julia-pfarr@users.noreply.github.com> * fix tests * [FIX] Fix Zenodo keywords not being set correctly (#598) * fix tests * remove docs file * use "subjects" metadata field instead of "keywords" * use "Nipoppy user" as default creator name * fix test * fix other tests * add pipeline name and version as keywords * [ENH] Add schema version to Zenodo record keywords (and pipeline config files) (#602) * fix tests * remove docs file * use "subjects" metadata field instead of "keywords" * use "Nipoppy user" as default creator name * fix test * fix other tests * add pipeline name and version as keywords * add mandatory schema version field and fix tests * [ENH] Implement `nipoppy pipeline search` (#599) * add search function and tests * add debug message * add search workflow and tests * add CLI command and test * minor changes * minor improvements * increase test coverage * address PR review comments * [ENH] Merge `nipoppy pipeline download` into `nipoppy pipeline install` (#594) * Merge `main` -> `dev-catalog` (#563) * [MAINT] Add dependabot for GitHub actions (#511) * Create dependabot.yml * add "bot" label * fix indentation * change schedule interval to monthly * [MAINT] Fix/update issue labeller (#509) * add new workflow and remove old ones * add required `enable-versioned-regex` field * try to fix labeler.yml * add write permissions for PRs * fix typo? * use pull_request_target insteade of pull_request event trigger * [MAINT] Use Nipoppy bot instead of PAT for projects automation workflow (#508) use bot instead of PAT * [MAINT] Add GitHub Actions workflow to automatically update `CITATION.cff` version after release (#512) * add DOI and update contact/authors in CITATION.cff * rename validation workflow * add initial version of update workflow * update workflow * update names * exclude `nipoppy-bot` PRs from changelog * Bump actions/stale from 9.0.0 to 9.1.0 (#515) Bumps [actions/stale](https://github.com/actions/stale) from 9.0.0 to 9.1.0. - [Release notes](https://github.com/actions/stale/releases) - [Changelog](https://github.com/actions/stale/blob/main/CHANGELOG.md) - [Commits](actions/stale@v9.0.0...v9.1.0) --- updated-dependencies: - dependency-name: actions/stale dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump actions/checkout from 3 to 4 (#514) Bumps [actions/checkout](https://github.com/actions/checkout) from 3 to 4. - [Release notes](https://github.com/actions/checkout/releases) - [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md) - [Commits](actions/checkout@v3...v4) --- updated-dependencies: - dependency-name: actions/checkout dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump actions/setup-python from 4 to 5 (#516) Bumps [actions/setup-python](https://github.com/actions/setup-python) from 4 to 5. - [Release notes](https://github.com/actions/setup-python/releases) - [Commits](actions/setup-python@v4...v5) --- updated-dependencies: - dependency-name: actions/setup-python dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * [MAINT] Bump codecov/codecov-action from 4 to 5 (#513) * Bump codecov/codecov-action from 4 to 5 Bumps [codecov/codecov-action](https://github.com/codecov/codecov-action) from 4 to 5. - [Release notes](https://github.com/codecov/codecov-action/releases) - [Changelog](https://github.com/codecov/codecov-action/blob/main/CHANGELOG.md) - [Commits](codecov/codecov-action@v4...v5) --- updated-dependencies: - dependency-name: codecov/codecov-action dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> * use `files` instead of deprecated `file` * add prefix to PR titles --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Michelle Wang <tomichellewang@gmail.com> * [MAINT] Clean up `.gitignore` (#521) * clean up .gitignore * add entries for Python venvs * [DOC] Add "how-to" guide for dataset initialization (#489) * add "how-to" guide for dataset init * address PR comments * split init doc in multiple pages for non-bids vs. bids start * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix error heading * fix heading and paths * fix header bids.md * update index and beginning of each page * update titles * minor changes --------- Co-authored-by: julia-pfarr <juliakatharina.pfarr@gmx.de> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Julia-Katharina Pfarr <111446107+julia-pfarr@users.noreply.github.com> * [pre-commit.ci] pre-commit autoupdate (#527) updates: - [github.com/pycqa/isort: 6.0.0 → 6.0.1](PyCQA/isort@6.0.0...6.0.1) Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * [DOC] Add a CLI reference page for `nipoppy status` (#529) * add "nipoppy status" page to docs * import StatusWorkflow in workflows module init * [DOC] Add tooltips for glossary terms (#526) try adding tooltip for glossary terms * [DOC] Do not include extractor scripts in autodocs (#525) do not include extractor scripts in autodocs (fix issue #502) * [DOC] Add external link checking for docs build + fix broken links (#524) * add linkcheck in Makefile * add linkcheck to readthedocs config * try to fix readthedocs config * fix broken links * add comment * address PR review comment * increase timeout limit * [MAINT] Add CI workflow for Read the docs preview link (#536) add Read the docs preview workflow * [ENH] Add `bids-validator` `2.0.3` (#534) * add `bids-validator` 2.0.3 * simplify container config * expose schema version field (use latest by default) in invocation * [ENH] group options in cli help. (#491) * [ENH] group options in cli help. * remove duplicate entry for --force flag * Update nipoppy/cli.py Co-authored-by: Michelle Wang <tomichellewang@gmail.com> * Remove redundant "options" suffix --------- Co-authored-by: Michelle Wang <tomichellewang@gmail.com> * [ENH] Remove Boutiques descriptor/invocation file content dumps (#542) * add `quiet` arg to `run_command` * silence bosh command * also overwrite subprocess error message * update error messages * fix regex in test * [ENH] Print container-pulling command and change global config container substitution (#540) * add test * add "apptainer pull" command to error message if URI is available * update default container store path * update container pull message * fix exception message in test * [BUG] Add validation for non-alphanumeric characters in `participant_id` and `session_id` in manifest (#546) * add failing tests * add validation * add test case with non-alphanumeric character in `visit_id` * clarify docstrings * update container path substitution pattern * [ENH] Improve `fs_stats` extractor pipeline (#550) * fix escape sequence warnings * make script crash if no subjects found + add troubleshooting suggestions for when FreeSurfer stats commands fail * make the default invocation have `"verbose": true` * rename script.py to fs_stats.py * update script * bump version to 0.2.1 * [MAINT] Fix Pydantic `model_fields` deprecations (#576) fix Pydantic `model_fields` deprecations * fix fs_stats version change * fix tests * fix test --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: julia-pfarr <juliakatharina.pfarr@gmx.de> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Julia-Katharina Pfarr <111446107+julia-pfarr@users.noreply.github.com> Co-authored-by: Mathieu Dugré <16450132+mathdugre@users.noreply.github.com> * more Zenodo workflow file into pipeline_store subdirectory * merge pipeline download with pipeline install * move tests * fix CLI tests * fix tests failing in 3.9 * increase test coverage * improve error message when Zenodo record doesn't have expected config file * address PR review comments * fix Zenodo ID for tests --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: julia-pfarr <juliakatharina.pfarr@gmx.de> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Julia-Katharina Pfarr <111446107+julia-pfarr@users.noreply.github.com> Co-authored-by: Mathieu Dugré <16450132+mathdugre@users.noreply.github.com> * [ENH] Automatically impute Zenodo record creator information (#601) * allow record_id in upload_pipeline() to start with "zenodo." * dynamically get creator info if user does not provide it * fix test * [ENH] Add `nipoppy pipeline list` command (#570) * Remove TRACKER_CONFIG_FILE from ProcPipelineConfig (should only be in ProcPipelineStepConfig) * update validation for BasePipelineStepConfig * update test for invocation/descriptor file fields * add tests for absolute path check and fix bug * update sample config files * refactor pipeline config loading + update relevant tests * handle relative paths in BasePipelineWorkflow * revert pipeline store directory names * fix some tests * fix remaining tests * fix doc warnings * increase test coverage + add test case * fix test * add helper function for getting pipeline bundle path * refactor test function args * add workflow and tests * fix failing tests * add CLI and test * switch to BaseDatasetWorkflow * update type annotation and refactor variable name * fix tests * fix tests * use defaultdict * reorder CLI subcommands * restore substitutions behaviour for backward-compatibility * fix test * [BUG] delete --regenerate flag from rich_click.OPTION_GROUPS (#609) * delete --regenerate flag from rich_click.OPTION_GROUPS * move option groups on top of file * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Fixing merge conflict to merge dev-catalog into main (#612) * Implement RABIES (#264) * Add and edits to global_configs file * delete unused global configs file * created rabies boutiques descriptor * add bid dir and output to boutique descriptor * auto generate boutique for RABIES * adding optional arg descriptors for RABIES * all optional args added to rabies boutiques json * create boutiques descriptor for each Rabies command * add output and input for RAbies analysis boutique descriptor * finish primary section of preprocess boutique descriptors * finish boutique desc for preprocess * fix boutiques descriptors * correct value key quotations * fix value key * add optional desc to confound correct boutique file * adding boutique desc for rabies analysis * added input and output to confound correction and analysis * generate invocation schema for rabies preprocess * generate innvocation schemas for analysis and confound-correction * remove extra data_diagnosis tag * correcting types in descriptors * renameupdate descriptors est Rabies version * fix rabies preprocess descriptor * remove default values from confound_correction * describe certain tags as type FLAG * correct confound-correction with Flag type * fix command line order in analysis descriptor * remove analysis default values * set confound input default a bids dir * fix descriptor typo * reorder confound_correciton descriptor and remove data_diagnosis tag * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * conf_list now list object in descriptor * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * correct spelling to make codespell happy :) * changed confound-correction default input value to derivatives * added sample innvocation files for RABIES steps * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add RABIES to sample global config files * edits to RABIES in sample configs * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * addition or RABIES to sample global config files * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * remove scan_list arg from RABIES analysis * fix verbose descriptions * fix uri for rabies sample config * empty tracker config for rabies * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * rename tracker file * set preprocess invocation file to defaults * added default local threads * replace variables with default values * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add descriptor files again * add rabies files to their own sample_pipelines folders * add descriptor and invocation to rabies sample pipeline files * add rabies config to global config files * remove out of data 0.4.8 rabies files * add rabies parameters to test config file * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add rabies tracker config file * make edits to descriptor via boutiques suggestions * fix pathing for rabies steps config * set proper naming convention for rabies pipeline files * full refactor to most up to date rabies descriptors * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix input/output dir paths in rabies invocation files * fix codespell issues/typos * remove default input/output for descriptor, use pipeline_output in invocation * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix more codespell typos * add desciptions to all descriptor files from rabies docs * code spell fix * fix command-line-flag for verbose * add list attribute to proper analysis arguments * add value choices for detrending_order arg * adding select value choices and list attribute to args when needed * fix verbose description * remove deprecated nipoppy_cli folder * fix brackets in analysis invocation * add rabies pipelines steps to test_pipeline_runner test * input/output paths now stated in invocation files * Update nipoppy/data/examples/sample_pipelines/rabies-0.5.1/tracker_config.json Co-authored-by: Michelle Wang <tomichellewang@gmail.com> * set data_diagnosis tag as true by default * add data_diagnosis files from analysis step as tracked outputs * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update nipoppy/data/examples/sample_global_config-all_pipelines.json Co-authored-by: Michelle Wang <tomichellewang@gmail.com> * add version number to rabies container_info * Update nipoppy/data/examples/sample_global_config-all_pipelines.json Co-authored-by: Michelle Wang <tomichellewang@gmail.com> * change input_dir to bids_dir for preprocess invocation * add analysis and generate pybids status to rabies steps * remove initial `/` from track config paths * switch analysis step to participant_session level, move tracker_config_file notation into analysis step * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * moving tracker to strictly analysis step * adding separate tracking files and steps in config * create tracker config files for each step * add confound correction image path to track * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add separate tracker configs for each step * tracking anatomical image files from preprocess outputs * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * rename all tracker config files to include step name * remove extra [ from preprocess invocation files * remove extra [ from confound-correciton invocation files * remove redundant tracker descriptor/innvocation files * remove descriptor and invocation lines from tracker steps in global config * remove unused tracker config files * add * to filepaths to improve flexibility * set "acq-rare" to * in preprocess tracker config paths * Add session number variable to session section of paths * remove non-image files from tracking * fix ses- and sub- labeling in tracker config file * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * remove ses- and sub- tags from preprocess tracking file paths --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Michelle Wang <tomichellewang@gmail.com> * Add back rabies config. + Move to sample_pipelines/preprocessing/rabies-0.5.1 * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add shcema version * Add pipeline type --------- Co-authored-by: David Roper <60201980+david-roper@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Michelle Wang <tomichellewang@gmail.com> * Ignore zenodo_api.py Low coverage on response error code. * fix broken link in docs --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: julia-pfarr <juliakatharina.pfarr@gmx.de> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Julia-Katharina Pfarr <111446107+julia-pfarr@users.noreply.github.com> Co-authored-by: Mathieu Dugré <16450132+mathdugre@users.noreply.github.com> Co-authored-by: David Roper <60201980+david-roper@users.noreply.github.com>
mathdugre
added a commit
that referenced
this pull request
May 1, 2025
* Merge `main` -> `dev-catalog` (#535) * [MAINT] Add dependabot for GitHub actions (#511) * Create dependabot.yml * add "bot" label * fix indentation * change schedule interval to monthly * [MAINT] Fix/update issue labeller (#509) * add new workflow and remove old ones * add required `enable-versioned-regex` field * try to fix labeler.yml * add write permissions for PRs * fix typo? * use pull_request_target insteade of pull_request event trigger * [MAINT] Use Nipoppy bot instead of PAT for projects automation workflow (#508) use bot instead of PAT * [MAINT] Add GitHub Actions workflow to automatically update `CITATION.cff` version after release (#512) * add DOI and update contact/authors in CITATION.cff * rename validation workflow * add initial version of update workflow * update workflow * update names * exclude `nipoppy-bot` PRs from changelog * Bump actions/stale from 9.0.0 to 9.1.0 (#515) Bumps [actions/stale](https://github.com/actions/stale) from 9.0.0 to 9.1.0. - [Release notes](https://github.com/actions/stale/releases) - [Changelog](https://github.com/actions/stale/blob/main/CHANGELOG.md) - [Commits](actions/stale@v9.0.0...v9.1.0) --- updated-dependencies: - dependency-name: actions/stale dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump actions/checkout from 3 to 4 (#514) Bumps [actions/checkout](https://github.com/actions/checkout) from 3 to 4. - [Release notes](https://github.com/actions/checkout/releases) - [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md) - [Commits](actions/checkout@v3...v4) --- updated-dependencies: - dependency-name: actions/checkout dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump actions/setup-python from 4 to 5 (#516) Bumps [actions/setup-python](https://github.com/actions/setup-python) from 4 to 5. - [Release notes](https://github.com/actions/setup-python/releases) - [Commits](actions/setup-python@v4...v5) --- updated-dependencies: - dependency-name: actions/setup-python dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * [MAINT] Bump codecov/codecov-action from 4 to 5 (#513) * Bump codecov/codecov-action from 4 to 5 Bumps [codecov/codecov-action](https://github.com/codecov/codecov-action) from 4 to 5. - [Release notes](https://github.com/codecov/codecov-action/releases) - [Changelog](https://github.com/codecov/codecov-action/blob/main/CHANGELOG.md) - [Commits](codecov/codecov-action@v4...v5) --- updated-dependencies: - dependency-name: codecov/codecov-action dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> * use `files` instead of deprecated `file` * add prefix to PR titles --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Michelle Wang <tomichellewang@gmail.com> * [MAINT] Clean up `.gitignore` (#521) * clean up .gitignore * add entries for Python venvs * [DOC] Add "how-to" guide for dataset initialization (#489) * add "how-to" guide for dataset init * address PR comments * split init doc in multiple pages for non-bids vs. bids start * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix error heading * fix heading and paths * fix header bids.md * update index and beginning of each page * update titles * minor changes --------- Co-authored-by: julia-pfarr <juliakatharina.pfarr@gmx.de> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Julia-Katharina Pfarr <111446107+julia-pfarr@users.noreply.github.com> * [pre-commit.ci] pre-commit autoupdate (#527) updates: - [github.com/pycqa/isort: 6.0.0 → 6.0.1](PyCQA/isort@6.0.0...6.0.1) Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * [DOC] Add a CLI reference page for `nipoppy status` (#529) * add "nipoppy status" page to docs * import StatusWorkflow in workflows module init * [DOC] Add tooltips for glossary terms (#526) try adding tooltip for glossary terms * [DOC] Do not include extractor scripts in autodocs (#525) do not include extractor scripts in autodocs (fix issue #502) * [DOC] Add external link checking for docs build + fix broken links (#524) * add linkcheck in Makefile * add linkcheck to readthedocs config * try to fix readthedocs config * fix broken links * add comment * address PR review comment * increase timeout limit --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: julia-pfarr <juliakatharina.pfarr@gmx.de> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Julia-Katharina Pfarr <111446107+julia-pfarr@users.noreply.github.com> * [ENH] Remove pipeline configs from global config (#503) * add path to pipeline config in layout * remove pipeline configs from global config and put them in pipeline bundle directories instead * update PipelineWorkflow (and subclasses) to handle new config file location * refactor get_pipeline_version to get dpath input instead of list * fix PipelineWorkflow tests * fix PipelineRunner tests * fix bug in _get_pipeline_config() * fix InitWorkflow tests * fix tests for BidsConversionRunner * fix tests for ExtractionRunner * fix TrackerWorkflow test * fix docs (?) * add/update tests * [MAINT] Add dependabot for GitHub actions (#511) * Create dependabot.yml * add "bot" label * fix indentation * change schedule interval to monthly * [MAINT] Fix/update issue labeller (#509) * add new workflow and remove old ones * add required `enable-versioned-regex` field * try to fix labeler.yml * add write permissions for PRs * fix typo? * use pull_request_target insteade of pull_request event trigger * [MAINT] Use Nipoppy bot instead of PAT for projects automation workflow (#508) use bot instead of PAT * [MAINT] Add GitHub Actions workflow to automatically update `CITATION.cff` version after release (#512) * add DOI and update contact/authors in CITATION.cff * rename validation workflow * add initial version of update workflow * update workflow * update names * exclude `nipoppy-bot` PRs from changelog * Bump actions/stale from 9.0.0 to 9.1.0 (#515) Bumps [actions/stale](https://github.com/actions/stale) from 9.0.0 to 9.1.0. - [Release notes](https://github.com/actions/stale/releases) - [Changelog](https://github.com/actions/stale/blob/main/CHANGELOG.md) - [Commits](actions/stale@v9.0.0...v9.1.0) --- updated-dependencies: - dependency-name: actions/stale dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump actions/checkout from 3 to 4 (#514) Bumps [actions/checkout](https://github.com/actions/checkout) from 3 to 4. - [Release notes](https://github.com/actions/checkout/releases) - [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md) - [Commits](actions/checkout@v3...v4) --- updated-dependencies: - dependency-name: actions/checkout dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump actions/setup-python from 4 to 5 (#516) Bumps [actions/setup-python](https://github.com/actions/setup-python) from 4 to 5. - [Release notes](https://github.com/actions/setup-python/releases) - [Commits](actions/setup-python@v4...v5) --- updated-dependencies: - dependency-name: actions/setup-python dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * [MAINT] Bump codecov/codecov-action from 4 to 5 (#513) * Bump codecov/codecov-action from 4 to 5 Bumps [codecov/codecov-action](https://github.com/codecov/codecov-action) from 4 to 5. - [Release notes](https://github.com/codecov/codecov-action/releases) - [Changelog](https://github.com/codecov/codecov-action/blob/main/CHANGELOG.md) - [Commits](codecov/codecov-action@v4...v5) --- updated-dependencies: - dependency-name: codecov/codecov-action dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> * use `files` instead of deprecated `file` * add prefix to PR titles --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Michelle Wang <tomichellewang@gmail.com> * [MAINT] Clean up `.gitignore` (#521) * clean up .gitignore * add entries for Python venvs * [DOC] Add "how-to" guide for dataset initialization (#489) * add "how-to" guide for dataset init * address PR comments * split init doc in multiple pages for non-bids vs. bids start * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix error heading * fix heading and paths * fix header bids.md * update index and beginning of each page * update titles * minor changes --------- Co-authored-by: julia-pfarr <juliakatharina.pfarr@gmx.de> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Julia-Katharina Pfarr <111446107+julia-pfarr@users.noreply.github.com> * [pre-commit.ci] pre-commit autoupdate (#527) updates: - [github.com/pycqa/isort: 6.0.0 → 6.0.1](PyCQA/isort@6.0.0...6.0.1) Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * [DOC] Add a CLI reference page for `nipoppy status` (#529) * add "nipoppy status" page to docs * import StatusWorkflow in workflows module init * [DOC] Add tooltips for glossary terms (#526) try adding tooltip for glossary terms * [DOC] Do not include extractor scripts in autodocs (#525) do not include extractor scripts in autodocs (fix issue #502) * [DOC] Add external link checking for docs build + fix broken links (#524) * add linkcheck in Makefile * add linkcheck to readthedocs config * try to fix readthedocs config * fix broken links * add comment * address PR review comment * increase timeout limit * fix typo * fix paths --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: julia-pfarr <juliakatharina.pfarr@gmx.de> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Julia-Katharina Pfarr <111446107+julia-pfarr@users.noreply.github.com> * [ENH] Add `PIPELINE_VARIABLES` to global config (#548) * add `PIPELINE_VARIABLES` field to global config * add PIPELINE_TYPE: PipelineTypeEnum in pipeline configs * use pipeline variables instead of substitutions for replacement in pipeline files * increase test coverage * address PR review comments * [ENH] Deprecate `DATASET_NAME`, `VISIT_IDS` and `SESSION_IDS` fields from `Config` (#553) * deprecate field from Config class * update StatusWorkflow and fix tests * rename test file * clean up BaseWorkflow tests * clean up DicomReorgWorkflow tests * clean up DoughnutWorkflow tests * fix autodoc warning * increase linkcheck timeout limit * [ENH] Rename pipeline store subdirectories (#561) * rename directories * fix tests * fix docs * [ENH] Force pipeline step config files to only have relative paths (#560) * Remove TRACKER_CONFIG_FILE from ProcPipelineConfig (should only be in ProcPipelineStepConfig) * update validation for BasePipelineStepConfig * update test for invocation/descriptor file fields * add tests for absolute path check and fix bug * update sample config files * refactor pipeline config loading + update relevant tests * handle relative paths in BasePipelineWorkflow * revert pipeline store directory names * fix some tests * fix remaining tests * fix doc warnings * increase test coverage + add test case * fix test * add helper function for getting pipeline bundle path * rename "proc" -> "processing" in test * [ENH] Split `BaseWorkflow` into `BaseWorkflow` and `BaseDatasetWorkflow` (#556) * rename `BaseWorkflow` -> `BaseDatasetWorkflow` * Extract `BaseWorkflow` from `BaseDatasetWorkflow` * move/fix tests * make validate_layout an init argument instead of class attribute * rename _skip_logging -> _skip_logfile * [ENH] Add `nipoppy pipeline validate` command (#558) * [ENH] Add `nipoppy pipeline install` command (#566) * add core validation functions + tests * Remove TRACKER_CONFIG_FILE from ProcPipelineConfig (should only be in ProcPipelineStepConfig) * add CLI command and substitutions * add substitutions test * add CLI test * rename test file * fix docs warnings * add test for PipelineValidateWorkflow * update validation for BasePipelineStepConfig * update test for invocation/descriptor file fields * add tests for absolute path check and fix bug * update sample config files * refactor pipeline config loading + update relevant tests * handle relative paths in BasePipelineWorkflow * revert pipeline store directory names * fix some tests * fix remaining tests * fix doc warnings * increase test coverage + add test case * fix test * adapt for pending relative path change * update log messages * fix test broken by merge * add helper function for getting pipeline bundle path * add base "pipeline install" functionality * Squashed commit of the following: commit e922f12 Merge: 788d3f1 5444535 Author: Michelle Wang <tomichellewang@gmail.com> Date: Mon Mar 24 14:01:07 2025 -0400 Merge remote-tracking branch 'upstream/dev-catalog' into pipeline_step_paths commit 5444535 Author: Michelle Wang <tomichellewang@gmail.com> Date: Sat Mar 22 13:04:20 2025 -0400 [ENH] Rename pipeline store subdirectories (#561) * rename directories * fix tests * fix docs * Squashed commit of the following: commit 853be29 Author: Michelle Wang <tomichellewang@gmail.com> Date: Mon Mar 24 13:53:19 2025 -0400 minor update to CLI * add CLI command and update logging * add variables to global config * add check that pipeline vars are not null * make sure existing fields in global config are not changed * add VARIABLES to default pipeline configs and add test * Squashed commit of the following: commit 34a5ddd Author: Michelle Wang <tomichellewang@gmail.com> Date: Fri Mar 28 11:15:59 2025 -0400 fix CLI test commit 8d17a8b Author: Michelle Wang <tomichellewang@gmail.com> Date: Thu Mar 27 16:51:37 2025 -0400 fix typo commit 7975bb4 Author: Michelle Wang <tomichellewang@gmail.com> Date: Thu Mar 27 16:33:59 2025 -0400 try to fix Pydantic deprecation errors commit c51fe30 Author: Michelle Wang <tomichellewang@gmail.com> Date: Thu Mar 27 16:25:09 2025 -0400 remove things related to dataset layout commit 3229e46 Merge: 9e75942 f437b59 Author: Michelle Wang <tomichellewang@gmail.com> Date: Thu Mar 27 15:59:53 2025 -0400 Merge remote-tracking branch 'upstream/dev-catalog' into 557/pipeline_validate commit 9e75942 Author: Michelle Wang <tomichellewang@gmail.com> Date: Thu Mar 27 11:17:46 2025 -0400 address most PR review comments commit 2e87f8d Merge: 9391ded fac5d2b Author: Michelle Wang <tomichellewang@gmail.com> Date: Thu Mar 27 11:06:40 2025 -0400 Merge remote-tracking branch 'upstream/dev-catalog' into 557/pipeline_validate commit 9391ded Author: Michelle Wang <tomichellewang@gmail.com> Date: Tue Mar 25 14:52:15 2025 -0400 make sure pipeline bundle content is flat list of file (no subdirectories) commit 42e904a Author: Michelle Wang <tomichellewang@gmail.com> Date: Tue Mar 25 12:40:06 2025 -0400 handle edge case with relative parent directories commit 853be29 Author: Michelle Wang <tomichellewang@gmail.com> Date: Mon Mar 24 13:53:19 2025 -0400 minor update to CLI * fix merge * fix PipelineInstallWorkflow * address PR review comments * fix CLI argument order * fix test * fix test again * do not catch exceptions * [ENH] api to download and upload pipelines from zenodo. (#506) * [ENH] api to download files from zenodo. * [FIX] add httpx dependency * [MAINT] remove requests dependency * [FIX] remove tqdm for now * [MAINT] Add dependabot for GitHub actions (#511) * Create dependabot.yml * add "bot" label * fix indentation * change schedule interval to monthly * [MAINT] Fix/update issue labeller (#509) * add new workflow and remove old ones * add required `enable-versioned-regex` field * try to fix labeler.yml * add write permissions for PRs * fix typo? * use pull_request_target insteade of pull_request event trigger * [MAINT] Use Nipoppy bot instead of PAT for projects automation workflow (#508) use bot instead of PAT * [MAINT] Add GitHub Actions workflow to automatically update `CITATION.cff` version after release (#512) * add DOI and update contact/authors in CITATION.cff * rename validation workflow * add initial version of update workflow * update workflow * update names * exclude `nipoppy-bot` PRs from changelog * Bump actions/stale from 9.0.0 to 9.1.0 (#515) Bumps [actions/stale](https://github.com/actions/stale) from 9.0.0 to 9.1.0. - [Release notes](https://github.com/actions/stale/releases) - [Changelog](https://github.com/actions/stale/blob/main/CHANGELOG.md) - [Commits](actions/stale@v9.0.0...v9.1.0) --- updated-dependencies: - dependency-name: actions/stale dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump actions/checkout from 3 to 4 (#514) Bumps [actions/checkout](https://github.com/actions/checkout) from 3 to 4. - [Release notes](https://github.com/actions/checkout/releases) - [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md) - [Commits](actions/checkout@v3...v4) --- updated-dependencies: - dependency-name: actions/checkout dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump actions/setup-python from 4 to 5 (#516) Bumps [actions/setup-python](https://github.com/actions/setup-python) from 4 to 5. - [Release notes](https://github.com/actions/setup-python/releases) - [Commits](actions/setup-python@v4...v5) --- updated-dependencies: - dependency-name: actions/setup-python dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * [MAINT] Bump codecov/codecov-action from 4 to 5 (#513) * Bump codecov/codecov-action from 4 to 5 Bumps [codecov/codecov-action](https://github.com/codecov/codecov-action) from 4 to 5. - [Release notes](https://github.com/codecov/codecov-action/releases) - [Changelog](https://github.com/codecov/codecov-action/blob/main/CHANGELOG.md) - [Commits](codecov/codecov-action@v4...v5) --- updated-dependencies: - dependency-name: codecov/codecov-action dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> * use `files` instead of deprecated `file` * add prefix to PR titles --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Michelle Wang <tomichellewang@gmail.com> * [MAINT] Clean up `.gitignore` (#521) * clean up .gitignore * add entries for Python venvs * [DOC] Add "how-to" guide for dataset initialization (#489) * add "how-to" guide for dataset init * address PR comments * split init doc in multiple pages for non-bids vs. bids start * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix error heading * fix heading and paths * fix header bids.md * update index and beginning of each page * update titles * minor changes --------- Co-authored-by: julia-pfarr <juliakatharina.pfarr@gmx.de> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Julia-Katharina Pfarr <111446107+julia-pfarr@users.noreply.github.com> * [pre-commit.ci] pre-commit autoupdate (#527) updates: - [github.com/pycqa/isort: 6.0.0 → 6.0.1](PyCQA/isort@6.0.0...6.0.1) Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * [DOC] Add a CLI reference page for `nipoppy status` (#529) * add "nipoppy status" page to docs * import StatusWorkflow in workflows module init * [DOC] Add tooltips for glossary terms (#526) try adding tooltip for glossary terms * [DOC] Do not include extractor scripts in autodocs (#525) do not include extractor scripts in autodocs (fix issue #502) * [DOC] Add external link checking for docs build + fix broken links (#524) * add linkcheck in Makefile * add linkcheck to readthedocs config * try to fix readthedocs config * fix broken links * add comment * address PR review comment * increase timeout limit * Update tests/test_zenodo.py Co-authored-by: Michelle Wang <tomichellewang@gmail.com> * Update nipoppy/zenodo.py Co-authored-by: Michelle Wang <tomichellewang@gmail.com> * Improve error message, no longer required access tokekn for download, improve test * add cli for downloading pipelines from zenodo * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update nipoppy/zenodo.py Co-authored-by: Michelle Wang <tomichellewang@gmail.com> * [ENH] Prototype for Zenodo upload * [TMP] started working on tests for upload. * Rename nipoppy pipeline add subcommand to install * add tests for authentication * Test for upload pipeline * Integrate new change for PR - Use new pipeline config for test - rename nipoppy pipeline add to download - Enforce nipoppy keywords on zenodo * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * pre-commit-ci modified the zenodo.json file for the example pipeline. * [no ci] cleanup gitignore conflict * Triger CI after adding secret to GH * Update nipoppy/cli.py Co-authored-by: Michelle Wang <tomichellewang@gmail.com> * Update nipoppy/zenodo.py Co-authored-by: Michelle Wang <tomichellewang@gmail.com> * fix pr comments * Seperated ZenodoAPI from workflow + tests for download workflow. * test get zenodo metadata from local pipeline. * test for zenodo upload workflow * rename zenodo_id to record_id * Add ZENODO as github variable * Remove and Prevent log file in history * Update nipoppy/workflows/zenodo.py Co-authored-by: Michelle Wang <tomichellewang@gmail.com> * pr comments * Fix PR comments * Fix env var name for zenodo ID * Handle empty ZENODO_TOKEN GitHub secret seems to be empty on PR * Update nipoppy/zenodo_api.py Co-authored-by: Michelle Wang <tomichellewang@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix after merge * try to fix failing test (Zenodo record not matching sample pipeline anymore) * fix new failing test * pre-commit fix * update test data directory name * fix again --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: Michelle Wang <tomichellewang@gmail.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: julia-pfarr <juliakatharina.pfarr@gmx.de> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Julia-Katharina Pfarr <111446107+julia-pfarr@users.noreply.github.com> * fix tests * [FIX] Fix Zenodo keywords not being set correctly (#598) * fix tests * remove docs file * use "subjects" metadata field instead of "keywords" * use "Nipoppy user" as default creator name * fix test * fix other tests * add pipeline name and version as keywords * [ENH] Add schema version to Zenodo record keywords (and pipeline config files) (#602) * fix tests * remove docs file * use "subjects" metadata field instead of "keywords" * use "Nipoppy user" as default creator name * fix test * fix other tests * add pipeline name and version as keywords * add mandatory schema version field and fix tests * [ENH] Implement `nipoppy pipeline search` (#599) * add search function and tests * add debug message * add search workflow and tests * add CLI command and test * minor changes * minor improvements * increase test coverage * address PR review comments * [ENH] Merge `nipoppy pipeline download` into `nipoppy pipeline install` (#594) * Merge `main` -> `dev-catalog` (#563) * [MAINT] Add dependabot for GitHub actions (#511) * Create dependabot.yml * add "bot" label * fix indentation * change schedule interval to monthly * [MAINT] Fix/update issue labeller (#509) * add new workflow and remove old ones * add required `enable-versioned-regex` field * try to fix labeler.yml * add write permissions for PRs * fix typo? * use pull_request_target insteade of pull_request event trigger * [MAINT] Use Nipoppy bot instead of PAT for projects automation workflow (#508) use bot instead of PAT * [MAINT] Add GitHub Actions workflow to automatically update `CITATION.cff` version after release (#512) * add DOI and update contact/authors in CITATION.cff * rename validation workflow * add initial version of update workflow * update workflow * update names * exclude `nipoppy-bot` PRs from changelog * Bump actions/stale from 9.0.0 to 9.1.0 (#515) Bumps [actions/stale](https://github.com/actions/stale) from 9.0.0 to 9.1.0. - [Release notes](https://github.com/actions/stale/releases) - [Changelog](https://github.com/actions/stale/blob/main/CHANGELOG.md) - [Commits](actions/stale@v9.0.0...v9.1.0) --- updated-dependencies: - dependency-name: actions/stale dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump actions/checkout from 3 to 4 (#514) Bumps [actions/checkout](https://github.com/actions/checkout) from 3 to 4. - [Release notes](https://github.com/actions/checkout/releases) - [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md) - [Commits](actions/checkout@v3...v4) --- updated-dependencies: - dependency-name: actions/checkout dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump actions/setup-python from 4 to 5 (#516) Bumps [actions/setup-python](https://github.com/actions/setup-python) from 4 to 5. - [Release notes](https://github.com/actions/setup-python/releases) - [Commits](actions/setup-python@v4...v5) --- updated-dependencies: - dependency-name: actions/setup-python dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * [MAINT] Bump codecov/codecov-action from 4 to 5 (#513) * Bump codecov/codecov-action from 4 to 5 Bumps [codecov/codecov-action](https://github.com/codecov/codecov-action) from 4 to 5. - [Release notes](https://github.com/codecov/codecov-action/releases) - [Changelog](https://github.com/codecov/codecov-action/blob/main/CHANGELOG.md) - [Commits](codecov/codecov-action@v4...v5) --- updated-dependencies: - dependency-name: codecov/codecov-action dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> * use `files` instead of deprecated `file` * add prefix to PR titles --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Michelle Wang <tomichellewang@gmail.com> * [MAINT] Clean up `.gitignore` (#521) * clean up .gitignore * add entries for Python venvs * [DOC] Add "how-to" guide for dataset initialization (#489) * add "how-to" guide for dataset init * address PR comments * split init doc in multiple pages for non-bids vs. bids start * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix error heading * fix heading and paths * fix header bids.md * update index and beginning of each page * update titles * minor changes --------- Co-authored-by: julia-pfarr <juliakatharina.pfarr@gmx.de> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Julia-Katharina Pfarr <111446107+julia-pfarr@users.noreply.github.com> * [pre-commit.ci] pre-commit autoupdate (#527) updates: - [github.com/pycqa/isort: 6.0.0 → 6.0.1](PyCQA/isort@6.0.0...6.0.1) Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * [DOC] Add a CLI reference page for `nipoppy status` (#529) * add "nipoppy status" page to docs * import StatusWorkflow in workflows module init * [DOC] Add tooltips for glossary terms (#526) try adding tooltip for glossary terms * [DOC] Do not include extractor scripts in autodocs (#525) do not include extractor scripts in autodocs (fix issue #502) * [DOC] Add external link checking for docs build + fix broken links (#524) * add linkcheck in Makefile * add linkcheck to readthedocs config * try to fix readthedocs config * fix broken links * add comment * address PR review comment * increase timeout limit * [MAINT] Add CI workflow for Read the docs preview link (#536) add Read the docs preview workflow * [ENH] Add `bids-validator` `2.0.3` (#534) * add `bids-validator` 2.0.3 * simplify container config * expose schema version field (use latest by default) in invocation * [ENH] group options in cli help. (#491) * [ENH] group options in cli help. * remove duplicate entry for --force flag * Update nipoppy/cli.py Co-authored-by: Michelle Wang <tomichellewang@gmail.com> * Remove redundant "options" suffix --------- Co-authored-by: Michelle Wang <tomichellewang@gmail.com> * [ENH] Remove Boutiques descriptor/invocation file content dumps (#542) * add `quiet` arg to `run_command` * silence bosh command * also overwrite subprocess error message * update error messages * fix regex in test * [ENH] Print container-pulling command and change global config container substitution (#540) * add test * add "apptainer pull" command to error message if URI is available * update default container store path * update container pull message * fix exception message in test * [BUG] Add validation for non-alphanumeric characters in `participant_id` and `session_id` in manifest (#546) * add failing tests * add validation * add test case with non-alphanumeric character in `visit_id` * clarify docstrings * update container path substitution pattern * [ENH] Improve `fs_stats` extractor pipeline (#550) * fix escape sequence warnings * make script crash if no subjects found + add troubleshooting suggestions for when FreeSurfer stats commands fail * make the default invocation have `"verbose": true` * rename script.py to fs_stats.py * update script * bump version to 0.2.1 * [MAINT] Fix Pydantic `model_fields` deprecations (#576) fix Pydantic `model_fields` deprecations * fix fs_stats version change * fix tests * fix test --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: julia-pfarr <juliakatharina.pfarr@gmx.de> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Julia-Katharina Pfarr <111446107+julia-pfarr@users.noreply.github.com> Co-authored-by: Mathieu Dugré <16450132+mathdugre@users.noreply.github.com> * more Zenodo workflow file into pipeline_store subdirectory * merge pipeline download with pipeline install * move tests * fix CLI tests * fix tests failing in 3.9 * increase test coverage * improve error message when Zenodo record doesn't have expected config file * address PR review comments * fix Zenodo ID for tests --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: julia-pfarr <juliakatharina.pfarr@gmx.de> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Julia-Katharina Pfarr <111446107+julia-pfarr@users.noreply.github.com> Co-authored-by: Mathieu Dugré <16450132+mathdugre@users.noreply.github.com> * [ENH] Automatically impute Zenodo record creator information (#601) * allow record_id in upload_pipeline() to start with "zenodo." * dynamically get creator info if user does not provide it * fix test * [ENH] Add `nipoppy pipeline list` command (#570) * Remove TRACKER_CONFIG_FILE from ProcPipelineConfig (should only be in ProcPipelineStepConfig) * update validation for BasePipelineStepConfig * update test for invocation/descriptor file fields * add tests for absolute path check and fix bug * update sample config files * refactor pipeline config loading + update relevant tests * handle relative paths in BasePipelineWorkflow * revert pipeline store directory names * fix some tests * fix remaining tests * fix doc warnings * increase test coverage + add test case * fix test * add helper function for getting pipeline bundle path * refactor test function args * add workflow and tests * fix failing tests * add CLI and test * switch to BaseDatasetWorkflow * update type annotation and refactor variable name * fix tests * fix tests * use defaultdict * reorder CLI subcommands * clean up sample global config * remove all sample pipeline files * remove default config tests * update init workflow * make tests pass * fix docs --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: julia-pfarr <juliakatharina.pfarr@gmx.de> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Julia-Katharina Pfarr <111446107+julia-pfarr@users.noreply.github.com> Co-authored-by: Mathieu Dugré <16450132+mathdugre@users.noreply.github.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Changes proposed in this pull request:
SCHEMA_VERSION=1Checklist (for reviewers)
This section is for the PR reviewer
[BUG],[DOC],[ENH],[MAINT])Refer to NumPy Development Guide for a full list
Closes #XXXXFor new features:
For bug fixes:
📚 Documentation preview 📚: https://nipoppy--602.org.readthedocs.build/en/602/