[Documentation] Editorial review comment fixed #2603
Conversation
|
Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/7739/ |
|
Build Failed with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/6465/ |
|
retest this please |
|
SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/6129/ |
sgururajshetty
force-pushed
the
editorial_review1
branch
from
b5f3308
to
cc4dfb6
Compare
| @@ -44,7 +44,7 @@ A Bloom filter is a space-efficient probabilistic data structure that is used to | |||
| Carbondata introduce BloomFilter as an index datamap to enhance the performance of querying with precise value. | |||
| @@ -351,7 +351,7 @@ public CarbonWriter buildWriterForCSVInput() throws IOException, InvalidLoadOpti | |||
| * @throws IOException | |||
| * @throws InvalidLoadOptionException | |||
| */ | |||
| public CarbonWriter buildWriterForAvroInput() throws IOException, InvalidLoadOptionException; | |||
| public CarbonWriter buildWriterForAvroInput(org.apache.avro.Schema schema) throws IOException, InvalidLoadOptionException; | |||
| ``` | |||
|
|
|||
There was a problem hiding this comment.
TestSdkJson example code needs to be corrected. testJsonSdkWriter should be static and IOException should be handled
import java.io.IOException;
import org.apache.carbondata.common.exceptions.sql.InvalidLoadOptionException;
import org.apache.carbondata.core.metadata.datatype.DataTypes;
import org.apache.carbondata.core.util.CarbonProperties;
import org.apache.carbondata.sdk.file.CarbonWriter;
import org.apache.carbondata.sdk.file.CarbonWriterBuilder;
import org.apache.carbondata.sdk.file.Field;
import org.apache.carbondata.sdk.file.Schema;
public class TestSdkJson {
public static void main(String[] args) throws InvalidLoadOptionException {
testJsonSdkWriter();
}
public void testJsonSdkWriter() throws InvalidLoadOptionException {
String path = "./target/testJsonSdkWriter";
Field[] fields = new Field[2];
fields[0] = new Field("name", DataTypes.STRING);
fields[1] = new Field("age", DataTypes.INT);
Schema CarbonSchema = new Schema(fields);
CarbonWriterBuilder builder = CarbonWriter.builder().outputPath(path);
// initialize json writer with carbon schema
CarbonWriter writer = builder.buildWriterForJsonInput(CarbonSchema);
// one row of json Data as String
String JsonRow = "{"name":"abcd", "age":10}";
int rows = 5;
for (int i = 0; i < rows; i++) {
writer.write(JsonRow);
}
writer.close();
}
}
8.2
| @@ -140,7 +140,7 @@ This section provides the details of all the configurations required for CarbonD | |||
| | carbon.enableMinMax | true | Min max is feature added to enhance query performance. To disable this feature, set it false. | | |||
| | carbon.dynamicallocation.schedulertimeout | 5 | Specifies the maximum time (unit in seconds) the scheduler can wait for executor to be active. Minimum value is 5 sec and maximum value is 15 sec. | | |||
| | carbon.scheduler.minregisteredresourcesratio | 0.8 | Specifies the minimum resource (executor) ratio needed for starting the block distribution. The default value is 0.8, which indicates 80% of the requested resource is allocated for starting block distribution. The minimum value is 0.1 min and the maximum value is 1.0. | | |||
| | carbon.search.enabled | false | If set to true, it will use CarbonReader to do distributed scan directly instead of using compute framework like spark, thus avoiding limitation of compute framework like SQL optimizer and task scheduling overhead. | | |||
| | carbon.search.enabled (Alpha Feature) | false | If set to true, it will use CarbonReader to do distributed scan directly instead of using compute framework like spark, thus avoiding limitation of compute framework like SQL optimizer and task scheduling overhead. | | |||
|
|
|||
| * **Global Dictionary Configurations** | |||
There was a problem hiding this comment.
In S3 section.
- there should not be any Space in parameter . Should be carbon.storelocation.
- "Concurrent queries are not supported" should be changed to "Only concurrent put (data management operations like load,insert,update)are supported."
- The "Another way of setting the authentication parameters is as follows" should be removed.
There was a problem hiding this comment.
This issue is handled in a different PR #2576
| @@ -140,7 +140,7 @@ This section provides the details of all the configurations required for CarbonD | |||
| | carbon.enableMinMax | true | Min max is feature added to enhance query performance. To disable this feature, set it false. | | |||
| | carbon.dynamicallocation.schedulertimeout | 5 | Specifies the maximum time (unit in seconds) the scheduler can wait for executor to be active. Minimum value is 5 sec and maximum value is 15 sec. | | |||
| | carbon.scheduler.minregisteredresourcesratio | 0.8 | Specifies the minimum resource (executor) ratio needed for starting the block distribution. The default value is 0.8, which indicates 80% of the requested resource is allocated for starting block distribution. The minimum value is 0.1 min and the maximum value is 1.0. | | |||
| | carbon.search.enabled | false | If set to true, it will use CarbonReader to do distributed scan directly instead of using compute framework like spark, thus avoiding limitation of compute framework like SQL optimizer and task scheduling overhead. | | |||
| | carbon.search.enabled (Alpha Feature) | false | If set to true, it will use CarbonReader to do distributed scan directly instead of using compute framework like spark, thus avoiding limitation of compute framework like SQL optimizer and task scheduling overhead. | | |||
|
|
|||
| * **Global Dictionary Configurations** | |||
There was a problem hiding this comment.
In Local Dictionary section the following updates needs to be done.
-
Remove the line: “44ad8fb40… Updated documentation on Local Dictionary Supoort |” in Page no: 7 at the Local Dictionary Configuration section in the Opensource PDF. -
Change the description for “Local dictionary threshold” from: “The maximum cardinality for local dictionary generation (maximum - 100000)”
to
“The maximum cardinality for local dictionary generation (maximum value is 100000 and minimum value is 1000. If the “local_dictionary_threshold” value is set below 1000 or above 100000, then it would take the default value 10000)”
There was a problem hiding this comment.
The minimum value need not be mentioned now
|
Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/7760/ |
|
Build Failed with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/6487/ |
|
Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/7763/ |
|
LGTM |
|
Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/7769/ |
|
Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/6493/ |
|
LGTM |
|
SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/6146/ |
|
SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/6152/ |
Minor issues fixed (spelling, syntax, and missing info) This closes #2603
Minor issues fixed (spelling, syntax, and missing info)