Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CARBONDATA-2929][DataMap] Add block skipped info for explain command #2711

Closed
wants to merge 1 commit into from

Conversation

kevinjmh
Copy link
Member

This pr will add block skipped info by counting distinct file path from hit blocklet. It shows like below:

|== CarbonData Profiler ==
Table Scan on test
 - total: 125 blocks, 250 blocklets
 - filter: (l_partkey <> null and l_partkey = 1006)
 - pruned by Main DataMap
    - skipped: 119 blocks, 238 blocklets
 - pruned by CG DataMap
    - name: dm
    - provider: bloomfilter
    - skipped: 6 blocks, 12 blocklets
|== CarbonData Profiler ==
Table Scan on test
 - total: 125 blocks, 250 blocklets
 - filter: TEXT_MATCH('l_shipmode:AIR')
 - pruned by Main DataMap
    - skipped: 0 blocks, 0 blocklets
 - pruned by FG DataMap
    - name: dm
    - provider: lucene
    - skipped: 12 blocks, 80 blocklets

Be sure to do all of the following checklist to help us incorporate
your contribution quickly and easily:

  • Any interfaces changed?

  • Any backward compatibility impacted?

  • Document update required?

  • Testing done
    Please provide details on
    - Whether new unit test cases have been added or why no new tests are required?
    - How it is tested? Please attach test report.
    - Is it a performance related change? Please attach the performance test report.
    - Any additional information to help reviewers in testing this change.

  • For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.

@CarbonDataQA
Copy link

Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/243/

@CarbonDataQA
Copy link

Build Failed with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/412/

@CarbonDataQA
Copy link

Build Failed with Spark 2.3.1, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.3/8482/

@jackylk
Copy link
Contributor

jackylk commented Sep 12, 2018

LGTM. Thanks for your contribution

@CarbonDataQA
Copy link

Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/249/

@asfgit asfgit closed this in 22958d9 Sep 12, 2018
@CarbonDataQA
Copy link

Build Success with Spark 2.3.1, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.3/8487/

@CarbonDataQA
Copy link

Build Failed with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/417/

@xuchuanyin
Copy link
Contributor

I think the message can be optimized from
- total: 125 blocks, 250 blocklets
to
- total: 125 blocks containing 250 blocklets

@kevinjmh kevinjmh deleted the explain_block_skip branch September 14, 2018 01:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants