[CARBONDATA-1170] Added condition to skip single_pass for first load #1031
Conversation
|
Can one of the admins verify this patch? |
|
Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/2467/ |
|
Refer to this link for build results (access rights to CI server needed): |
kunal642
force-pushed
the
single_pass_restrictions
branch
from
00d9d2d
to
c32ab73
Compare
|
Refer to this link for build results (access rights to CI server needed): Failed Tests: 2carbondata-pr-spark-1.6/org.apache.carbondata:carbondata-core: 2 |
|
Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/2471/ |
|
retest this please |
|
Refer to this link for build results (access rights to CI server needed): |
|
Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/2476/ |
kunal642
force-pushed
the
single_pass_restrictions
branch
from
c32ab73
to
30b6586
Compare
|
Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/2478/ |
|
Refer to this link for build results (access rights to CI server needed): |
| @@ -496,6 +496,15 @@ case class LoadTable( | |||
| val dictFolderPath = carbonTablePath.getMetadataDirectoryPath | |||
| val dimensions = carbonTable.getDimensionByTableName( | |||
| carbonTable.getFactTableName).asScala.toArray | |||
| if (null == carbonLoadModel.getLoadMetadataDetails) { | |||
| CommonUtil.readLoadMetadataDetails(carbonLoadModel, storePath) | |||
There was a problem hiding this comment.
Can use single pass for first load, but it will automatically fallback to twopass, which should be logged and continue.
|
LGTM |
The user should not be able to load using "single_pass"="true" for the first load instead the load would be restarted with "single_pass"="false"