[CARBONDATA-2098]Add Documentation for Pre-Aggregate tables #1886
Conversation
|
Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/2036/ |
|
Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/3273/ |
|
SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/3227/ |
| @@ -703,6 +704,194 @@ This tutorial is going to introduce all commands and data operations on CarbonDa | |||
| * The partitioned column can be excluded from SORT_COLUMNS, this will let other columns to do the efficient sorting. | |||
| * When writing SQL on a partition table, try to use filters on the partition column. | |||
|
|
|||
| ## PRE-AGGREGATE TABLES | |||
There was a problem hiding this comment.
Please add some example to show the plan matching mechanism, like what query will hit which datamap
|
@sraghunandan please add pre-agg example, it is better to have the performance comparison inside example. |
sraghunandan
force-pushed
the
partition_preagg_documentation
branch
from
49ead42
to
850dde6
Compare
|
@jackylk @chenliang613 added examples. pls review |
sraghunandan
force-pushed
the
partition_preagg_documentation
branch
from
850dde6
to
c4e67b1
Compare
|
Build Failed with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/2167/ |
|
Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/3406/ |
|
Build Failed with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/2171/ |
|
SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/3283/ |
sraghunandan
force-pushed
the
partition_preagg_documentation
branch
from
c4e67b1
to
dcfcc53
Compare
|
Build Failed with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/2201/ |
|
Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/3441/ |
|
SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/3305/ |
| | MAX | Yes | | ||
| | MIN | Yes | | ||
| | COUNT | Yes | | ||
| | DISTINCT COUNT | No | |
There was a problem hiding this comment.
Currently we are not supporting distinct count and in future only for timeseries we can support
sraghunandan
force-pushed
the
partition_preagg_documentation
branch
from
dcfcc53
to
2ab494e
Compare
|
Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/3471/ |
|
Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/2231/ |
|
|
||
| import scala.util.Random | ||
| val r = new Random() | ||
| val df = spark.sparkContext.parallelize(1 to 10 * 1000 * 1000 ) |
There was a problem hiding this comment.
please reduce the data size
| CarbonProperties.getInstance() | ||
| .addProperty(CarbonCommonConstants.CARBON_TIMESTAMP_FORMAT, "yyyy-MM-dd HH:mm:ss") | ||
|
|
||
| // spark.sql(s"LOAD DATA LOCAL INPATH '$testData' into table timeSeriesTable") |
sraghunandan
force-pushed
the
partition_preagg_documentation
branch
from
2ab494e
to
54a03a2
Compare
|
verified, LGTM |
|
Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/2233/ |
|
Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/3473/ |
Add Documentation for Pre-Aggregate tables This closes apache#1886
Be sure to do all of the following checklist to help us incorporate
your contribution quickly and easily:
No
No
Updated documentation
Please provide details on
- Whether new unit test cases have been added or why no new tests are required?
- How it is tested? Please attach test report.
- Is it a performance related change? Please attach the performance test report.
- Any additional information to help reviewers in testing this change.
NA
NA