[CARBONDATA-3371] Fix ArrayIndexOutOfBoundsException of compaction after sort_columns modification #3201
Conversation
|
Build Failed with Spark 2.3.2, Please check CI http://136.243.101.176:8080/job/carbondataprbuilder2.3/11348/ |
|
Build Failed with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/3283/ |
|
Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/3084/ |
QiangCai
force-pushed
the
sort_columns_compaction_issue
branch
from
49fd2f5
to
d7fddd5
Compare
|
Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/3085/ |
|
Build Failed with Spark 2.3.2, Please check CI http://136.243.101.176:8080/job/carbondataprbuilder2.3/11349/ |
|
Build Failed with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/3284/ |
|
Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/3089/ |
|
Build Success with Spark 2.3.2, Please check CI http://136.243.101.176:8080/job/carbondataprbuilder2.3/11353/ |
|
Build Success with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/3288/ |
core/src/main/java/org/apache/carbondata/core/scan/result/iterator/RawResultIterator.java
Outdated
Show resolved
Hide resolved
core/src/main/java/org/apache/carbondata/core/scan/result/iterator/RawResultIterator.java
Outdated
Show resolved
Hide resolved
core/src/main/java/org/apache/carbondata/core/scan/result/iterator/RawResultIterator.java
Outdated
Show resolved
Hide resolved
|
@ravipesala please review again |
|
Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/3113/ |
|
Build Success with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/3312/ |
|
Build Success with Spark 2.3.2, Please check CI http://136.243.101.176:8080/job/carbondataprbuilder2.3/11377/ |
|
@QiangCai Please resolve conflicts |
QiangCai
force-pushed
the
sort_columns_compaction_issue
branch
from
f344fb7
to
dfcccb7
Compare
|
Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/3134/ |
|
Build Success with Spark 2.3.2, Please check CI http://136.243.101.176:8080/job/carbondataprbuilder2.3/11399/ |
|
Build Success with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/3333/ |
|
LGTM |
…ter sort_columns modification Modification: SegmentPropertiesWrapper should check the column order for different segments Because sort_columns modification can change the column order. dictionaryColumnChunkIndex of blockExecutionInfo should keep projection order for compaction if column drift happened, it should convert measure to dimension in RawResultIterator This closes #3201
…ter sort_columns modification Modification: SegmentPropertiesWrapper should check the column order for different segments Because sort_columns modification can change the column order. dictionaryColumnChunkIndex of blockExecutionInfo should keep projection order for compaction if column drift happened, it should convert measure to dimension in RawResultIterator This closes apache#3201
Modification:
Because sort_columns modification can change the column order.
Any interfaces changed?
no
Any backward compatibility impacted?
no
Document update required?
no
Testing done
Please provide details on
- Whether new unit test cases have been added or why no new tests are required?
added unit test cases
- How it is tested? Please attach test report.
unit test cases passed
- Is it a performance related change? Please attach the performance test report.
- Any additional information to help reviewers in testing this change.
For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.
small chagnes