Schema refactor#48
Merged
Merged
Conversation
* linkml: add tool1 schema * linkml: add schema utils * linkml: add schema vignette + schema_to_mermaid.R * linkml: reorder tool1 schema * add schema_versions for mermaid diagrams * pkgdown fixes * gha: refactor deploy workflow for dev and main branches * pkgdown: use auto development mode * Bump version: 0.0.3 => 0.0.3.9000 * r-ulid: grab from umccr conda channel * makefile: add bump rule * makefile: add bump rule * Bump version: 0.0.3.9000 => 0.0.3.9001 * rattler-build upload anaconda: use channel, not label * Bump version: 0.0.3.9001 => 0.0.3.9002 * gha conda: drop umccr prefix to find dev label * Bump version: 0.0.3.9002 => 0.0.3.9003 * gha: use ssh-key for bot committing to protected branch * Bump version: 0.0.3.9003 => 0.0.3.9004 * gha conda pkgdown: drop umccr prefix to find dev label * Bump version: 0.0.3.9004 => 0.0.3.9005 * gha conda pkgdown: specify dev label * Bump version: 0.0.3.9005 => 0.0.3.9006 * [bot] Updating conda-lock files (v0.0.3.9006) * precommit: add air formatter * add CLAUDE.md * claude: add new nemotool skill * "Claude PR Assistant workflow" * "Claude Code Review workflow" * Change GitHub + Anaconda orgs (#39) * change gh org * change anaconda org * change anaconda org * GitHub Actions: use GitHub app for branch protection override (#40) * gha: use gh app for branch protection override * gha: use app email * gha: use same wf for dev + main (#41) * GitHub Actions: use reusable workflows for conda + pkgdown (#42) * gha: fix permissions (#43) * Add GHA-based version bumping workflow (#44) * Bump version: 0.0.3.9006 => 0.0.3.9007 * [bot] Updating conda-lock files (v0.0.3.9007) * precommit update * remove LinkML schema system (to be redesigned in separate PR) * gha: remove auto claude code review workflow * gha: restrict claude workflow to repo owners/collaborators/members --------- Co-authored-by: GitHub Actions <actions@github.com> Co-authored-by: tidywf-ci-bot[bot] <3171681+tidywf-ci-bot[bot]@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
This PR performs a major schema/config refactor across the nemo R6 tool/workflow framework, consolidating per-tool configs into a unified schema.yaml format with per-column versioning and extending the package with schema visualisation + UML tooling alongside documentation, example data, CI/deploy, and packaging updates.
Changes:
- Replaced split
raw.yaml/tidy.yamltool configs with a unifiedinst/config/tools/<tool>/schema.yamland updatedConfig/Tool/Tool1/WorkflowAPIs accordingly. - Added schema browsing/visualisation helpers (
nemo_schema_*,reactable_schema) plus a new schema vignette and pkgdown navigation updates. - Updated example/extdata (new table4/table5 samples), docs, tests (roxytest-generated), deployment pipelines, conda tooling, and repo metadata (URLs/org rename, version bump).
Reviewed changes
Copilot reviewed 71 out of 73 changed files in this pull request and generated 7 comments.
Show a summary per file
| File | Description |
|---|---|
| vignettes/structure.Rmd | Updated vignette to describe/use unified schema.yaml and new Config getters. |
| vignettes/schemas.qmd | Added vignette for interactive schema browsing via nemo_schema_reactable(). |
| vignettes/schema.qmd | Added detailed vignette documenting schema.yaml format, ftypes, and versioning. |
| vignettes/devnotes.qmd | Updated DVC notes / repo links to new org. |
| vignettes/contribute.qmd | Updated contributor steps for new schema layout and Tool scaffolding guidance. |
| vignettes/NEWS.qmd | Added dev release notes for schema refactor + tooling/CI changes and updated repo URL. |
| tests/testthat/test-roxytest-testexamples-utils.R | Updated generated roxytest line reference. |
| tests/testthat/test-roxytest-testexamples-schema_vis.R | Added generated tests for new schema visualisation API. |
| tests/testthat/test-roxytest-testexamples-parse.R | Updated generated tests to new Config$get_schemas_all() API. |
| tests/testthat/test-roxytest-testexamples-Workflow1.R | Updated generated tests for Workflow1 behavior and new file counts. |
| tests/testthat/test-roxytest-testexamples-Workflow.R | Updated generated tests for Workflow pipeline (filter/tidy/write/metadata). |
| tests/testthat/test-roxytest-testexamples-Tool1.R | Updated generated tests to cover new Tool1 tables and parsing/tidying. |
| tests/testthat/test-roxytest-testexamples-Tool.R | Added generated tests for new Tool dispatch/filter/write behavior. |
| tests/testthat/test-roxytest-testexamples-Config.R | Updated generated tests for new Config API (patterns, ftypes, schemas, col map). |
| pkgdown/_pkgdown.yml | Updated site URL, added schema article + schema visualisation reference section. |
| man/schema_guess.Rd | Updated docs/examples to new Config schema getter name. |
| man/reactable_schema.Rd | Added Rd for schema reactable renderer. |
| man/nemo_uml.Rd | Added Rd for UML generator. |
| man/nemo_schema_reactable.Rd | Added Rd for interactive schema explorer. |
| man/nemo_schema_data.Rd | Added Rd for schema data builder. |
| man/nemo-package.Rd | Updated repository + website links to new org. |
| man/config_sort_versions.Rd | Added internal Rd for version sorting helper. |
| man/config_prep_raw.Rd | Updated default type to "txt". |
| man/config_prep_multi.Rd | Updated examples to use "txt" types. |
| man/Workflow1.Rd | Updated Workflow1 description/examples + return section. |
| man/Workflow.Rd | Expanded Workflow docs/examples/returns and clarified include/exclude semantics. |
| man/Tool1.Rd | Updated Tool1 description/examples and method docs to new tables/dispatch changes. |
| man/Tool.Rd | Expanded Tool docs/examples and documented new dispatch helpers. |
| man/Config.Rd | Updated Config docs/examples to unified schema model + new API surface. |
| inst/scripts/uml.R | Switched UML script to call new nemo_uml() helper. |
| inst/scripts/file_to_yaml.R | Refactored config generator script to emit unified schema skeletons under _tmp/. |
| inst/extdata/tool1/v4.5.6/sampleA.tool1.table1.tsv | Added example input for an additional tool version. |
| inst/extdata/tool1/latest/sampleA.tool1.table5.csv | Added example csv-nohead-long input. |
| inst/extdata/tool1/latest/sampleA.tool1.table4.tsv | Added example txt-nohead input. |
| inst/extdata/tool1/latest/sampleA.tool1.table1.tsv | Updated latest example (dropped metricX, kept Y/Z). |
| inst/documentation/notes/pixi.md | Updated pixi channel references to new org. |
| inst/documentation/installation/_r.qmd | Updated GitHub install instructions and version reference. |
| inst/documentation/installation/_pixi.qmd | Updated pixi channel/package version examples. |
| inst/documentation/installation/_conda.qmd | Updated conda channel URLs and version example. |
| inst/config/tools/tool1/tidy.yaml | Removed legacy tidy config. |
| inst/config/tools/tool1/schema.yaml | Added unified schema for Tool1 including ftypes and per-column versions. |
| inst/config/tools/tool1/raw.yaml | Removed legacy raw config. |
| deploy/conda/recipe/recipe.yaml | Bumped version and added htmltools/reactable runtime deps; updated project URLs. |
| deploy/conda/env/yaml/uml.yaml | Added dedicated UML conda env definition. |
| deploy/conda/env/yaml/pkgdown.yaml | Updated channels and bumped r-nemo version. |
| deploy/conda/env/yaml/nemo.yaml | Updated channels and bumped r-nemo version. |
| deploy/conda/env/lock/conda-linux-64.lock | Refreshed lockfile for new version/channel changes. |
| data-raw/fake_tool1.R | Extended fake Tool1 generator for multi-version + new table formats (table4/table5). |
| README.qmd | Updated badge/docs links to new org URLs. |
| README.md | Regenerated README with updated URLs and CLI help formatting. |
| R/utils.R | Minor internal change (schema_type_remap assertion style) + workflow dispatcher addition. |
| R/uml.R | Added exported nemo_uml() helper to generate UML diagrams. |
| R/schema_vis.R | Added schema visualisation API (nemo_schema_data, nemo_schema_reactable, reactable_schema). |
| R/parse.R | Updated examples to new Config schema getter name. |
| R/Workflow1.R | Updated docs/tests and return annotations for Workflow1. |
| R/Workflow.R | Expanded docs/tests and clarified include/exclude semantics + returns. |
| R/Tool1.R | Updated Tool1 to new dispatch model; added table5 parse/tidy and table3 tidy override. |
| R/Tool.R | Added parse/tidy dispatch helpers, ftype-based parsing, updated docs/tests. |
| R/Config.R | Reworked Config to read unified schema.yaml; added schema/ftype/colmap APIs and version helper. |
| NAMESPACE | Exported new schema visualisation and UML functions. |
| Makefile | Added phony targets, bump/check targets, and adjusted build behavior. |
| LICENSE.md | Updated copyright year. |
| LICENSE | Updated YEAR field. |
| DESCRIPTION | Bumped version, updated URLs, added suggests/remotes for new features. |
| .pre-commit-config.yaml | Switched to air formatter hook and reorganized hook config. |
| .gitignore | Updated ignore patterns (notably _tmp output). |
| .github/workflows/deploy.yaml | Refactored deploy pipeline to reusable tidywf workflows and added UML step. |
| .github/workflows/claude.yml | Restricted Claude workflow triggers to trusted author associations. |
| .github/dependabot.yml | Added Dependabot config for GitHub Actions updates. |
| .claude/skills/nemotool/SKILL.md | Added Claude “nemotool” scaffolding skill documentation. |
| .claude/CLAUDE.md | Added repository guidance for Claude Code (architecture, workflows, conventions). |
| .bumpversion.toml | Bumped current version and updated bump workflow search/replace patterns. |
| .Rbuildignore | Ignored .claude directory in package builds. |
Comments suppressed due to low confidence (1)
R/schema_vis.R:136
reactable_schema()builds HTML by concatenating unescaped values fromschema_data(e.g.names(schema_data)andas.character(row)). If any schema strings contain<,&, quotes, etc., this can break the HTML or allow injection in rendered outputs (pkgdown/vignettes). Escape dynamic values (e.g. viahtmltools::htmlEscape()) before inserting them into HTML.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Major change in the schema structure:
raw.yaml/tidy.yamlconfig files with a singleschema.yamlper tool, using a flattablesmap with per-column versions for multi-version supportConfig,Tool, andTool1updated to work with the new schemaToolTool1