[CARBONDATA-2960] SDK Reader fix with projection columns #2747
Conversation
|
Can one of the admins verify this patch? |
|
retest this please |
|
Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/421/ |
|
Build Failed with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/600/ |
|
Build Failed with Spark 2.3.1, Please check CI http://136.243.101.176:8080/job/carbondataprbuilder2.3/8670/ |
manishnalla1994
force-pushed
the
reader_fix
branch
from
b684ab2
to
b9d1f6e
Compare
|
retest this please |
| for (ColumnSchema cols : colList) { | ||
| if (cols.getSchemaOrdinal() != -1) { | ||
| projectColumn.add(cols.getColumnName()); | ||
| if (childDimCount == 0) { |
There was a problem hiding this comment.
added and updated
| @@ -205,6 +205,13 @@ public CarbonReaderBuilder setEndPoint(String value) { | |||
|
|
|||
| if (projectionColumns != null) { | |||
| // set the user projection | |||
| int len = projectionColumns.length; | |||
| for (int i = 0; i < len; i++) { | |||
| if (projectionColumns[i].contains(".")) { | |||
There was a problem hiding this comment.
Add some comments here
| val json = """ {"name":"bob", "age":10, "arrayRecord": [{"101": "Rahul", "102": "Pawan"}]} """.stripMargin | ||
| nonTransactionalCarbonTable.WriteFilesWithAvroWriter(2, mySchema, json) | ||
|
|
||
| val exception1 = intercept[Exception] { |
There was a problem hiding this comment.
Handle this in above test case itself. As we are testing reader. No need to write again and read it.
manishnalla1994
force-pushed
the
reader_fix
branch
from
b9d1f6e
to
c0f64cb
Compare
|
retest this please |
1 similar comment
|
retest this please |
|
Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/442/ |
|
add to whitelist |
|
Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/445/ |
|
Build Success with Spark 2.3.1, Please check CI http://136.243.101.176:8080/job/carbondataprbuilder2.3/8695/ |
|
Build Success with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/625/ |
|
LGTM |
SDK Reader was not working when all projection columns were given.
Added exception for Complex child projections too.
Be sure to do all of the following checklist to help us incorporate
your contribution quickly and easily:
Any interfaces changed?
Any backward compatibility impacted?
Document update required?
Testing done
For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.