Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[CARBONDATA-3106] Written_by_APPNAME not serialized in executor with global sort #2928

Closed
wants to merge 1 commit into from
Closed

[CARBONDATA-3106] Written_by_APPNAME not serialized in executor with global sort #2928

wants to merge 1 commit into from

Conversation

Indhumathi27
Copy link
Contributor

@Indhumathi27 Indhumathi27 commented Nov 16, 2018 •
edited

Problem:
Written_By_APPNAME when added in carbonproperty is not serialized in executor with global sort

Solution:
Add Written_by_APPNAME in hadoop conf and in executor side get it from configuration and add to carbonproperty

  • Any interfaces changed?

  • Any backward compatibility impacted?

  • Document update required?

  • Testing done
    Please provide details on
    - Whether new unit test cases have been added or why no new tests are required?
    - How it is tested? Please attach test report.
    - Is it a performance related change? Please attach the performance test report.
    - Any additional information to help reviewers in testing this change.

  • For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.

@CarbonDataQA
Copy link

Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/1447/

@CarbonDataQA
Copy link

Build Success with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/1657/

@CarbonDataQA
Copy link

Build Success with Spark 2.3.1, Please check CI http://136.243.101.176:8080/job/carbondataprbuilder2.3/9705/

@akashrn5
Copy link
Contributor

In CarbonFactDataWriterImplV3.java , writeFootrToFile , if appName after getting from carbonProperties is null,you can fail the dataload with dataload exception, please handle that also

@CarbonDataQA
Copy link

Build Failed with Spark 2.3.1, Please check CI http://136.243.101.176:8080/job/carbondataprbuilder2.3/9709/

@CarbonDataQA
Copy link

Build Failed with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/1661/

@CarbonDataQA
Copy link

Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/1451/

@CarbonDataQA
Copy link

Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/1452/

@CarbonDataQA
Copy link

Build Success with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/1662/

@CarbonDataQA
Copy link

Build Success with Spark 2.3.1, Please check CI http://136.243.101.176:8080/job/carbondataprbuilder2.3/9710/

@manishgupta88
Copy link
Contributor

LGTM

@asfgit asfgit closed this in da91d4c Nov 20, 2018
asfgit pushed a commit that referenced this pull request Nov 21, 2018
@ravipesala
[CARBONDATA-3106] WrittenbyAPI not serialized in executor with global…
2e0153b 
…sort

Problem:
Written_By_APPNAME when added in carbonproperty is not serialized in executor with global sort

Solution:
Add Written_by_APPNAME in hadoop conf and in executor side get it from configuration and add to carbonproperty

This closes #2928
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
4 participants
@Indhumathi27 @CarbonDataQA @akashrn5 @manishgupta88