[Bug][AUTHZ] Data masking fails with MISSING_ATTRIBUTES on a UNION-ALL view over a masked view

[j1wonpark]

Code of Conduct

• I agree to follow this project's Code of Conduct

Search before asking

• I have searched in the issues and found no similar issues.

Describe the bug

Reading a permanent view that UNION-ALLs a masked permanent view fails analysis, while reading the masked view directly works (#3581):

[MISSING_ATTRIBUTES.RESOLVED_ATTRIBUTE_APPEAR_IN_OPERATION] Resolved attribute(s) "value2"
missing from "value2" in operator !Project [cast(value2#613 as string) AS value2#611].

With a view-level, type-preserving MASK policy on perm_view_masked.value2:

CREATE OR REPLACE VIEW default.perm_view_masked AS SELECT key, value2 FROM default.src;
CREATE OR REPLACE VIEW default.perm_view_union AS
  SELECT key, value2 FROM default.perm_view_masked WHERE key = 1
  UNION ALL
  SELECT key, value2 FROM default.perm_view_masked WHERE key = 10;

-- as the masked user
SELECT key, value2 FROM default.perm_view_union WHERE key = 1;   -- FAIL
SELECT key, value2 FROM default.perm_view_masked WHERE key = 1;  -- OK

The same view structure without a masking policy resolves fine, so the union/view shape itself is handled by Spark — only the masking rewrite breaks it.

Root cause (confirmed with spark.sql.planChangeLog): both UNION branches expand the masked view with identical output exprIds. After RuleApplyDataMaskingStage0 marks both branches, DeduplicateRelations re-instances the second branch's PermanentViewMarker, but DataMaskingStage0Marker.scan is not a child and keeps the pre-dedup ids, so the second branch's exprToMaskers() becomes keyed on ids now owned by the first branch. RuleApplyDataMaskingStage1 then merges all marker maps into one toMap (duplicate keys silently last-wins) and substitutes unconditionally, rewiring the outer view's schema compensation Project — above the Union — to attributes the Union never outputs.

Fix: scope the Stage1 substitution — rewrite the children first, then only substitute maskers the rewritten children actually expose, merging the per-marker maps after that filtering. Branch-local healing keeps working; the cross-branch leak is blocked at the Union boundary. All five data masking suites and the full kyuubi-spark-authz module pass with the fix.

Affects Version(s)

master

Kyuubi Server Log Output

No response

Kyuubi Engine Log Output

No response

Kyuubi Server Configurations

No response

Kyuubi Engine Configurations

No response

Additional context

No response

Are you willing to submit PR?

• Yes. I would be willing to submit a PR with guidance from the Kyuubi community to fix.
• No. I cannot submit a PR at this time.