Skip to content

Commit

Permalink
[CARBONDATA-3166]Updated Document and added Column Compressor used in…
Browse files Browse the repository at this point in the history
… Describe Formatted

Updated Document and added column compressor used in Describe Formatted Command

This closes #2986
  • Loading branch information
shardul-cr7 authored and xuchuanyin committed Dec 14, 2018
1 parent 82adc50 commit ebdd548
Show file tree
Hide file tree
Showing 2 changed files with 4 additions and 2 deletions.
2 changes: 1 addition & 1 deletion docs/configuration-parameters.md
Expand Up @@ -91,7 +91,7 @@ This section provides the details of all the configurations required for the Car
| carbon.dictionary.server.port | 2030 | Single Pass Loading enables single job to finish data loading with dictionary generation on the fly. It enhances performance in the scenarios where the subsequent data loading after initial load involves fewer incremental updates on the dictionary. Single pass loading can be enabled using the option ***carbon.options.single.pass***. When this option is specified, a dictionary server will be internally started to handle the dictionary generation and query requests. This configuration specifies the port on which the server need to listen for incoming requests. Port value ranges between 0-65535 |
| carbon.load.directWriteToStorePath.enabled | false | During data load, all the carbondata files are written to local disk and finally copied to the target store location in HDFS/S3. Enabling this parameter will make carbondata files to be written directly onto target HDFS/S3 location bypassing the local disk.**NOTE:** Writing directly to HDFS/S3 saves local disk IO(once for writing the files and again for copying to HDFS/S3) there by improving the performance. But the drawback is when data loading fails or the application crashes, unwanted carbondata files will remain in the target HDFS/S3 location until it is cleared during next data load or by running *CLEAN FILES* DDL command |
| carbon.options.serialization.null.format | \N | Based on the business scenarios, some columns might need to be loaded with null values. As null value cannot be written in csv files, some special characters might be adopted to specify null values. This configuration can be used to specify the null values format in the data being loaded. |
| carbon.column.compressor | snappy | CarbonData will compress the column values using the compressor specified by this configuration. Currently CarbonData supports 'snappy' and 'zstd' compressors. |
| carbon.column.compressor | snappy | CarbonData will compress the column values using the compressor specified by this configuration. Currently CarbonData supports 'snappy', 'zstd' and 'gzip' compressors. |
| carbon.minmax.allowed.byte.count | 200 | CarbonData will write the min max values for string/varchar types column using the byte count specified by this configuration. Max value is 1000 bytes(500 characters) and Min value is 10 bytes(5 characters). **NOTE:** This property is useful for reducing the store size thereby improving the query performance but can lead to query degradation if value is not configured properly. | |

## Compaction Configuration
Expand Down
Expand Up @@ -92,7 +92,9 @@ private[sql] case class CarbonDescribeFormattedCommand(
Strings.formatSize(
tblProps.getOrElse(CarbonCommonConstants.CARBON_LOAD_MIN_SIZE_INMB,
CarbonCommonConstants.CARBON_LOAD_MIN_SIZE_INMB_DEFAULT).toFloat), ""),

("Data File Compressor ", tblProps
.getOrElse(CarbonCommonConstants.COMPRESSOR,
CarbonCommonConstants.DEFAULT_COMPRESSOR), ""),
//////////////////////////////////////////////////////////////////////////////
// Index Information
//////////////////////////////////////////////////////////////////////////////
Expand Down

0 comments on commit ebdd548

Please sign in to comment.