Skip to content

Hudi MIT doesn't support the target table reference in extra MATCHED condition #17303

@hudi-bot

Description

@hudi-bot

Below MIT query fails with Hudi - 


{{MERGE INTO hudi_table_mor_2_partition_columns t
USING comprehensive_merge_source s
ON t.id = s.id
WHEN MATCHED 
AND s.operation = 'UPDATE_DEPT_MATCH' 
AND t.department = s.department 
THEN
UPDATE SET *
WHEN MATCHED 
AND s.operation = 'UPDATE_SALARY' THEN
UPDATE SET * ;}}

{{```}}

{{{}if we just remove `{}}}{{{}t.department = s.department{}}}{{{}` from above query it works. It also works if we just remove reference for target table from the same. {}}}{{{}`{}}}{{{}t.department = 'A'{}}}{{{}` doesn't work but {}}}{{{}`s{}}}{{{}.department = 'A'{}}}{{{}` works well.{}}}

 

GIthub issue have reproducible code - [https://github.com/apache/hudi/issues/12180]

Confirmed that DELTA supports these kind of MIT queries but Hudi fails with the Exception.  Also checked with master code and issue still exists.

## JIRA info

- Link: https://issues.apache.org/jira/browse/HUDI-8457
- Type: Sub-task
- Parent: https://issues.apache.org/jira/browse/HUDI-9109
- Fix version(s):
  - 1.1.0


---


## Comments

05/Jan/25 23:40;daviszhang;[https://github.com/apache/hudi/issues/12180] the issue has some initial triage;;;

---

06/Jan/25 19:08;yihua;We should mention the constraints of MATCHED conditions as a limitation of MERGE INTO in Hudi Spark SQL, for now.  We should triage how much work it takes to actually support such matching condition.;;;

Metadata

Metadata

Assignees

Labels

Type

No type
No fields configured for issues without a type.

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions