Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[HOTFIX] Modified code to fix the degrade in compaction performance #2613

Closed
wants to merge 1 commit into from
Closed

[HOTFIX] Modified code to fix the degrade in compaction performance #2613

wants to merge 1 commit into from

Conversation

manishgupta88
Copy link
Contributor

Problem
Compaction performance for 3.5 billion degraded by 16-20%

Analysis:
Code modification in RawResultIerator.java has caused the problem wherein few extra checks are performed as compared to previous code

Fix
Revert the changes in RawResultIerator class

  • Any interfaces changed?
    No
  • Any backward compatibility impacted?
    NA
  • Document update required?
    No
  • Testing done
    Verified in cluster testing
  • For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.
    NA

@ravipesala
Copy link
Contributor

SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/6184/

@CarbonDataQA
Copy link

Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/7807/

@CarbonDataQA
Copy link

Build Failed with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/6531/

@ravipesala
Copy link
Contributor

SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/6189/

jackylk
jackylk reviewed Aug 7, 2018
View reviewed changes
core/src/main/java/org/apache/carbondata/core/scan/result/iterator/RawResultIterator.java Outdated
} else {
currentIdxInBuffer = 0;
new RowsFetcher(false).call();
@Override public Object[] next() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Move @OverRide to previous line

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok

@ravipesala
Copy link
Contributor

SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/6191/

@CarbonDataQA
Copy link

Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/7815/

@CarbonDataQA
Copy link

Build Failed with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/6539/

@ravipesala
Copy link
Contributor

SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/6196/

@CarbonDataQA
Copy link

Build Failed with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/6546/

@CarbonDataQA
Copy link

Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/7822/

@manishgupta88
Problem
70201bf 
Compaction performance for 3.5 billion degraded by 16-20%

Analysis:
Code modification in RawResultIerator.java has caused the problem wherein few extra checks are performed as compared to previous code

Fix
Revert the changes in RawResultIerator class
@ravipesala
Copy link
Contributor

SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/6201/

@CarbonDataQA
Copy link

Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/7827/

@CarbonDataQA
Copy link

Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/6553/

@ravipesala
Copy link
Contributor

LGTM

@asfgit asfgit closed this in 45b8d18 Aug 8, 2018
asfgit pushed a commit that referenced this pull request Aug 9, 2018
@ravipesala
[HOTFIX] Modified code to fix the degrade in compaction performance
7535d46 
Problem
Compaction performance for 3.5 billion degraded by 16-20%

Analysis:
Code modification in RawResultIerator.java has caused the problem wherein few extra checks are performed as compared to previous code

Fix
Revert the changes in RawResultIerator class

This closes #2613
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jackylk jackylk jackylk left review comments

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
4 participants
@manishgupta88 @ravipesala @CarbonDataQA @jackylk