-
Notifications
You must be signed in to change notification settings - Fork 2.5k
Investigate record projection for spark records #16668
Copy link
Copy link
Open
Labels
area:readerReader core functionalityReader core functionalityarea:writerWrite client and core write operationsWrite client and core write operationsengine:sparkSpark integrationSpark integrationfrom-jirapriority:highSignificant impact; potential bugsSignificant impact; potential bugstype:devtaskDevelopment tasks and maintenance workDevelopment tasks and maintenance work
Milestone
Metadata
Metadata
Assignees
Labels
area:readerReader core functionalityReader core functionalityarea:writerWrite client and core write operationsWrite client and core write operationsengine:sparkSpark integrationSpark integrationfrom-jirapriority:highSignificant impact; potential bugsSignificant impact; potential bugstype:devtaskDevelopment tasks and maintenance workDevelopment tasks and maintenance work
Type
Fields
Give feedbackNo fields configured for issues without a type.
If a record is projected multiple times it can cause an OOM. Even for a write with only 10 records. Investigate if there is a memory leak or inefficiency with how HoodieSparkRecord does projection (rewrite with new schema)
JIRA info