Skip to content

Add --continue-on-error flag for map-data CLI#163

Merged
amc-corey-cox merged 3 commits intomainfrom
continue-on-error
Mar 27, 2026
Merged

Add --continue-on-error flag for map-data CLI#163
amc-corey-cox merged 3 commits intomainfrom
continue-on-error

Conversation

@amc-corey-cox
Copy link
Copy Markdown
Contributor

Summary

  • Adds TransformationError dataclass exception with structured context (class_derivation_name, slot_derivation_name, source_row, row_index)
  • Single try/except in map_object's slot derivation loop converts exceptions to TransformationError with full context
  • transform_spec() gains an on_error callback parameter: None = fail-fast (default), callback = collect and continue
  • --continue-on-error CLI flag collects all row-level errors, reports summary to stderr, exits 1

Closes #162

Test plan

  • Engine-level: bad rows collected via on_error callback, good rows yielded, fail-fast preserved when no callback
  • CLI-level: --continue-on-error reports to stderr with exit 1; no errors = exit 0; without flag = fail-fast
  • Existing tests pass (526 total, including updated exception expectations)
  • just test full suite green

🤖 Generated with Claude Code

Introduce TransformationError with structured context (class_derivation,
slot_derivation, row_index) and on_error callback in transform_spec()
so row-level errors can be collected instead of aborting. CLI reports
error summary to stderr and exits 1 when errors occurred.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds structured, row-level error handling to the transformation engine and exposes it via a new --continue-on-error flag in the map-data CLI, enabling “collect and continue” behavior while preserving default fail-fast semantics.

Changes:

  • Introduces TransformationError carrying structured context (class/slot/row/row_index) for transformation failures.
  • Adds an on_error callback to transform_spec() to collect row errors and continue processing.
  • Extends map-data CLI with --continue-on-error to report a summary of errors and exit with code 1 when any occur.

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
src/linkml_map/transformer/errors.py Adds TransformationError dataclass with structured context + formatting.
src/linkml_map/transformer/object_transformer.py Wraps slot-derivation exceptions as TransformationError with context.
src/linkml_map/transformer/engine.py Adds on_error callback to collect/enrich row-level errors and continue.
src/linkml_map/cli/cli.py Adds --continue-on-error flag; collects errors, prints summary, exits 1 if any.
tests/test_transformer/test_object_transformer.py Updates exception expectation to TransformationError.
tests/test_transformer/test_continue_on_error.py Adds engine/error-type unit tests for continue-on-error behavior.
tests/test_compliance/test_compliance_suite.py Allows compliance tests to accept TransformationError wrapping.
tests/test_cli/test_cli_tabular.py Adds CLI-level tests covering fail-fast vs continue-on-error behavior.

Comment thread tests/test_transformer/test_object_transformer.py Outdated
Comment thread src/linkml_map/transformer/object_transformer.py
Comment thread src/linkml_map/transformer/errors.py Outdated
Comment thread tests/test_transformer/test_continue_on_error.py Outdated
- Restore match= assertion on test_derive_from_expr_restricted_raises
- Fix errors.py docstring: row_index only added with on_error callback
- Fix test_continue_on_error.py docstring to match actual field names

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Copy link
Copy Markdown
Collaborator

@madanucd madanucd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! All 25 tests pass including the 3 new CLI tests and 8 new error handling tests. Fail-fast default is preserved and --continue-on-error is a clean opt-in for batch jobs where one bad row shouldn't kill the entire run. All Copilot feedback addressed.

Resolve conflict in object_transformer.py: keep try/except error
wrapping from continue-on-error around the refactored populated_from
logic (joins, _resolve_fk_or_literal) from main.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@amc-corey-cox amc-corey-cox merged commit b840a70 into main Mar 27, 2026
7 checks passed
@amc-corey-cox amc-corey-cox deleted the continue-on-error branch March 27, 2026 21:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add --continue-on-error flag to map-data CLI

3 participants