Permalink
Show file tree
Hide file tree
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Browse files
[CARBONDATA-4194] Fixed presto read after update/delete from spark
Why is this PR needed? After update/delete with spark on the table which contains array/struct column, when we are trying to read from presto then it is throwing class cast exception. It is because when we perform update/delete then it contains vector of type ColumnarVectorWrapperDirectWithDeleteDelta which we are trying to typecast to CarbonColumnVectorImpl and because of this it is throwing typecast exception. After fixing this(added check for instanceOf) it started throwing IllegalArgumentException. It is because: 1. In case of local dictionary enable CarbondataPageSource.load is calling ComplexTypeStreamReader.putComplexObject before setting the correct number of rows(doesn't subtrat deleted rows). And it throws IllegalArgument while block building for child elements. 2. position count is wrong in the case of the struct. It should subtract the number of deleted rows in LocalDictDimensionDataChunkStore.fillVector. While this is not required to be changed in the case of the array because datalength of the array already taking care of deleted rows in ColumnVectorInfo.getUpdatedPageSizeForChildVector. What changes were proposed in this PR? First fixed class cast exception after putting instanceOf condition in if block. Then subtracted the deleted row count before calling ComplexTypeStreamReader.putComplexObject in DirectCompressCodec.decodeAndFillVector. Also handle deleted rows in case of struct in LocalDictDimensionDataChunkStore.fillVector Does this PR introduce any user interface change? No Is any new testcase added? No This Closes #4224
- Loading branch information
Showing
3 changed files
with
32 additions
and
18 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters