Skip to content

Investigate record projection for spark records #16668

@hudi-bot

Description

@hudi-bot

If a record is projected multiple times it can cause an OOM. Even for a write with only 10 records. Investigate if there is a memory leak or inefficiency with how HoodieSparkRecord does projection (rewrite with new schema)

JIRA info

Metadata

Metadata

Assignees

Labels

area:readerReader core functionalityarea:writerWrite client and core write operationsengine:sparkSpark integrationfrom-jirapriority:highSignificant impact; potential bugstype:devtaskDevelopment tasks and maintenance work

Type

No type
No fields configured for issues without a type.

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions