Skip to content

Per-element runtime failures can produce silent short results #8233

Description

@carloea2

Feature Summary

MainLoop._process_data_element catches per-element runtime failures and writes only a worker log. It never updates the exception manager, so failed rows can disappear from a batch while the workflow continues and may report success with a short result.

Before: row processing fails -> worker log only -> workflow can complete with missing rows

After: row processing fails -> exception is reported -> remaining batch rows are attempted -> worker enters exception handling

Expected behavior:

Every per-element runtime failure should use the normal exception reporting path so the workflow cannot silently succeed with missing output rows.

Reproduction evidence:

Process a two-row DataElement while making input statistics update raise for each row. Both rows are attempted, but the exception manager remains empty.

Observed on 70c21145887920528d7d5540e3fb790b43e8b759:

attempted_rows=[0, 1]
logged_failures=2
exception_manager_has_exception=False

The focused regression failed with:

assert main_loop.context.exception_manager.has_exception()
E assert False

Version and commit evidence:

1.3.0-incubating-SNAPSHOT (main)

Commit Hash (Optional)

70c21145887920528d7d5540e3fb790b43e8b759

Proposed Solution or Design

Report every per-element runtime failure through the normal exception path so the workflow cannot silently succeed with missing output rows.

Affected Area

Workflow Engine (Amber)

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions