Skip to content

fix(frontend): drop stale attribute references when an operator's input schema changes#5294

Merged
kunwp1 merged 4 commits into
apache:mainfrom
kunwp1:fix/3070-aggregate-copy-paste-stale-attributes
May 30, 2026
Merged

fix(frontend): drop stale attribute references when an operator's input schema changes#5294
kunwp1 merged 4 commits into
apache:mainfrom
kunwp1:fix/3070-aggregate-copy-paste-stale-attributes

Conversation

@kunwp1
Copy link
Copy Markdown
Contributor

@kunwp1 kunwp1 commented May 29, 2026

What changes were proposed in this PR?

Fixes a bug where copy-pasting a schema-propagated Aggregate operator and connecting it to a different upstream operator caused a compile error that persisted even after clearing all properties.

Why it is a bug: An Aggregate stores property values that reference input column names — groupByKeys and aggregations[].attribute. Copy-paste clones operatorProperties, so the pasted operator keeps references to the old source's columns. When it is wired to a different operator, the backend AggregateOpDesc schema propagation calls inputSchema.getAttribute("<old col>"), which throws. These old column values were never removed. DynamicSchemaService.setDynamicSchema is documented to drop properties invalidated by a new schema, but the code never actually did it.

What changed: When schema propagation produces an input-attribute list, WorkflowCompilingService now drops property values that the new schema invalidates via a new pure helper dropInvalidAttributeValues:

  • autofill: "attributeName" -> reset to "" if not in the enum
  • autofill: "attributeNameList" -> filter out entries not in the enum
  • recurses through nested objects/arrays (so aggregations[].attribute is handled) and never mutates the input

The misleading doc comment on setDynamicSchema now points to where the contract is fulfilled.

Any related issues, documentation, discussions?

Resolves #3070.

How was this PR tested?

Manually tested:

Screen.Recording.2026-05-29.at.4.58.05.PM.mov

Was this PR authored or co-authored using generative AI tooling?

Generated-by: Claude Code (Claude Opus 4.8)

…ange

Copy-pasting a schema-propagated Aggregate (or any operator with
@AutofillAttributeName/@AutofillAttributeNameList properties) and wiring it
to a different upstream left property values referencing columns that no
longer exist in the new input schema. This caused a compile error that
persisted even after clearing properties, since invalid values were never
removed.

When schema propagation produces an authoritative input-attribute list,
WorkflowCompilingService now drops property values that the new schema
invalidates: single attribute references are reset and attribute lists are
filtered. This honors the contract previously only documented on
DynamicSchemaService.setDynamicSchema.

Resolves apache#3070.
@github-actions github-actions Bot added fix frontend Changes related to the frontend GUI labels May 29, 2026
@codecov-commenter
Copy link
Copy Markdown

codecov-commenter commented May 30, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 49.39%. Comparing base (953e2c4) to head (6d7aac1).

Additional details and impacted files
@@             Coverage Diff              @@
##               main    #5294      +/-   ##
============================================
+ Coverage     49.16%   49.39%   +0.23%     
  Complexity     2384     2384              
============================================
  Files          1051     1051              
  Lines         40350    40388      +38     
  Branches       4279     4292      +13     
============================================
+ Hits          19837    19949     +112     
+ Misses        19353    19261      -92     
- Partials       1160     1178      +18     
Flag Coverage Δ *Carryforward flag
access-control-service 41.89% <ø> (ø) Carriedforward from 607ccc5
agent-service 33.76% <ø> (ø) Carriedforward from 607ccc5
amber 51.66% <ø> (ø) Carriedforward from 607ccc5
computing-unit-managing-service 0.00% <ø> (ø) Carriedforward from 607ccc5
config-service 0.00% <ø> (ø) Carriedforward from 607ccc5
file-service 38.42% <ø> (ø) Carriedforward from 607ccc5
frontend 41.66% <100.00%> (+0.58%) ⬆️
python 90.79% <ø> (ø) Carriedforward from 607ccc5
workflow-compiling-service 56.81% <ø> (ø) Carriedforward from 607ccc5

*This pull request uses carry forward flags. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

kunwp1 added 2 commits May 29, 2026 17:20
Add a TestBed integration test driving applySchemaPropagationResult so the
getOperator/dropInvalidAttributeValues/setOperatorProperty path is exercised
(both the changed and unchanged branches), plus helper tests for the
non-object/nullish guard and the absent-property branch.
@kunwp1
Copy link
Copy Markdown
Contributor Author

kunwp1 commented May 30, 2026

/request-review @bobbai00

@github-actions github-actions Bot requested a review from bobbai00 May 30, 2026 00:27
@bobbai00 bobbai00 changed the title fix: drop stale attribute references when an operator's input schema changes fix(frontend): drop stale attribute references when an operator's input schema changes May 30, 2026
@kunwp1 kunwp1 added this pull request to the merge queue May 30, 2026
Merged via the queue into apache:main with commit a1045ce May 30, 2026
18 checks passed
@kunwp1 kunwp1 deleted the fix/3070-aggregate-copy-paste-stale-attributes branch May 30, 2026 19:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

fix frontend Changes related to the frontend GUI

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Copy Pasting Schema propagated Aggregate operation causes Error when connecting with different operator

3 participants