Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CARBONDATA-3476] Fix Read time and scan time stats in executor log for filter query #3333

Closed
wants to merge 1 commit into from

Conversation

@vikramahuja1001
Copy link
Contributor

vikramahuja1001 commented Jul 24, 2019

Problem: Read time and scan time stats shown wrong in executor log for filter query

Root cause: Projection read time is added in scan time because of this scan time and read time is not correct in stats

Solution: Added projection read time for both measure and dimension column in read stats

Be sure to do all of the following checklist to help us incorporate
your contribution quickly and easily:

  • Any interfaces changed? NA

  • Any backward compatibility impacted? NA

  • Document update required? NA

  • Testing done
    Please provide details on
    - Whether new unit test cases have been added or why no new tests are required?
    - How it is tested? Please attach test report.
    - Is it a performance related change? Please attach the performance test report.
    - Any additional information to help reviewers in testing this change.

  • For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.

@vikramahuja1001

This comment has been minimized.

Copy link
Contributor Author

vikramahuja1001 commented Jul 24, 2019

@brijoobopanna @kunal642 , please review.

@CarbonDataQA

This comment has been minimized.

Copy link

CarbonDataQA commented Jul 24, 2019

Build Failed with Spark 2.1.0, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.1/120/

@CarbonDataQA

This comment has been minimized.

Copy link

CarbonDataQA commented Jul 24, 2019

Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/3864/

@CarbonDataQA

This comment has been minimized.

Copy link

CarbonDataQA commented Jul 24, 2019

Build Failed with Spark 2.3.2, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/119/

@CarbonDataQA

This comment has been minimized.

Copy link

CarbonDataQA commented Jul 24, 2019

Build Failed with Spark 2.2.1, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.2/114/

@CarbonDataQA

This comment has been minimized.

Copy link

CarbonDataQA commented Jul 24, 2019

Build Failed with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/4069/

@CarbonDataQA

This comment has been minimized.

Copy link

CarbonDataQA commented Jul 24, 2019

Build Success with Spark 2.3.2, Please check CI http://136.243.101.176:8080/job/carbondataprbuilder2.3/12137/

@vikramahuja1001

This comment has been minimized.

Copy link
Contributor Author

vikramahuja1001 commented Jul 30, 2019

retest this please

@CarbonDataQA

This comment has been minimized.

Copy link

CarbonDataQA commented Jul 30, 2019

Build Success with Spark 2.1.0, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.1/158/

@CarbonDataQA

This comment has been minimized.

Copy link

CarbonDataQA commented Jul 30, 2019

Build Failed with Spark 2.2.1, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.2/152/

@CarbonDataQA

This comment has been minimized.

Copy link

CarbonDataQA commented Jul 30, 2019

Build Failed with Spark 2.3.2, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/156/

@vikramahuja1001

This comment has been minimized.

Copy link
Contributor Author

vikramahuja1001 commented Jul 31, 2019

retest this please

@CarbonDataQA

This comment has been minimized.

Copy link

CarbonDataQA commented Jul 31, 2019

Build Success with Spark 2.1.0, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.1/164/

@CarbonDataQA

This comment has been minimized.

Copy link

CarbonDataQA commented Jul 31, 2019

Build Success with Spark 2.2.1, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.2/158/

@CarbonDataQA

This comment has been minimized.

Copy link

CarbonDataQA commented Jul 31, 2019

Build Failed with Spark 2.3.2, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/162/

@vikramahuja1001

This comment has been minimized.

Copy link
Contributor Author

vikramahuja1001 commented Jul 31, 2019

retest this please

@CarbonDataQA

This comment has been minimized.

Copy link

CarbonDataQA commented Jul 31, 2019

Build Success with Spark 2.1.0, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.1/172/

@CarbonDataQA

This comment has been minimized.

Copy link

CarbonDataQA commented Jul 31, 2019

Build Failed with Spark 2.2.1, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.2/166/

@CarbonDataQA

This comment has been minimized.

Copy link

CarbonDataQA commented Jul 31, 2019

Build Failed with Spark 2.3.2, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/170/

@vikramahuja1001

This comment has been minimized.

Copy link
Contributor Author

vikramahuja1001 commented Aug 1, 2019

retest this please

@CarbonDataQA

This comment has been minimized.

Copy link

CarbonDataQA commented Aug 1, 2019

Build Success with Spark 2.1.0, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.1/183/

@CarbonDataQA

This comment has been minimized.

Copy link

CarbonDataQA commented Aug 1, 2019

Build Failed with Spark 2.3.2, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/182/

@CarbonDataQA

This comment has been minimized.

Copy link

CarbonDataQA commented Aug 1, 2019

Build Success with Spark 2.2.1, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.2/177/

@vikramahuja1001

This comment has been minimized.

Copy link
Contributor Author

vikramahuja1001 commented Aug 1, 2019

retest this please

@CarbonDataQA

This comment has been minimized.

Copy link

CarbonDataQA commented Aug 1, 2019

Build Success with Spark 2.1.0, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.1/189/

@CarbonDataQA

This comment has been minimized.

Copy link

CarbonDataQA commented Aug 1, 2019

Build Failed with Spark 2.2.1, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.2/183/

@CarbonDataQA

This comment has been minimized.

Copy link

CarbonDataQA commented Aug 1, 2019

Build Success with Spark 2.3.2, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/188/

@brijoobopanna

This comment has been minimized.

Copy link
Contributor

brijoobopanna commented Aug 5, 2019

retest this please

@CarbonDataQA

This comment has been minimized.

Copy link

CarbonDataQA commented Aug 5, 2019

Build Success with Spark 2.1.0, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.1/220/

@CarbonDataQA

This comment has been minimized.

Copy link

CarbonDataQA commented Aug 5, 2019

Build Failed with Spark 2.2.1, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.2/215/

@CarbonDataQA

This comment has been minimized.

Copy link

CarbonDataQA commented Aug 5, 2019

Build Success with Spark 2.3.2, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/218/

@kunal642

This comment has been minimized.

Copy link
Contributor

kunal642 commented Aug 6, 2019

LGTM

@asfgit asfgit closed this in 88ec830 Aug 6, 2019
kevinjmh added a commit to kevinjmh/carbondata that referenced this pull request Sep 5, 2019
…or filter query

Root cause: Projection read time is added in scan time because of this scan time and
read time is not correct in stats

Solution: Added projection read time for both measure and dimension column in read
stats seperately.

This closes apache#3333
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
4 participants
You can’t perform that action at this time.