Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[CARBONDATA-2513][32K] Support write long string from dataframe #2382

Closed
wants to merge 1 commit into from
Closed

[CARBONDATA-2513][32K] Support write long string from dataframe #2382

wants to merge 1 commit into from

Conversation

xuchuanyin
Copy link
Contributor

@xuchuanyin xuchuanyin commented Jun 19, 2018 •
edited

Support write long string from dataframe

Sample for usage:

longStringDF.write
  .format("carbondata")
  .option("tableName", longStringTable)
  .option("single_pass", "false")
  .option("sort_columns", "name")
  .option("long_string_columns", "description, note")
  .mode(SaveMode.Overwrite)
  .save()

Be sure to do all of the following checklist to help us incorporate
your contribution quickly and easily:

  • Any interfaces changed?
    NO

  • Any backward compatibility impacted?
    NO

  • Document update required?
    NO

  • Testing done
    Please provide details on
    - Whether new unit test cases have been added or why no new tests are required?
    Tests added
    - How it is tested? Please attach test report.
    Tested in local machine
    - Is it a performance related change? Please attach the performance test report.
    - Any additional information to help reviewers in testing this change.

  • For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.

@xuchuanyin
Copy link
Contributor Author

This PR depends on #2379

@xuchuanyin xuchuanyin mentioned this pull request Jun 19, 2018
Closed
5 tasks
@CarbonDataQA
Copy link

Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/6370/

@CarbonDataQA
Copy link

Build Failed with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/5209/

@ravipesala
Copy link
Contributor

SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/5319/

@xuchuanyin
Copy link
Contributor Author

retest this please

@xuchuanyin
Copy link
Contributor Author

retest it please

@CarbonDataQA
Copy link

Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/6381/

@CarbonDataQA
Copy link

Build Failed with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/5216/

@CarbonDataQA
Copy link

Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/6397/

@CarbonDataQA
Copy link

Build Failed with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/5232/

@xuchuanyin
Copy link
Contributor Author

retest this please

@ravipesala
Copy link
Contributor

SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/5345/

@CarbonDataQA
Copy link

Build Failed with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/5237/

@CarbonDataQA
Copy link

Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/6403/

@xuchuanyin
Copy link
Contributor Author

retest this please

@kumarvishal09
Copy link
Contributor

@xuchuanyin Please rebase

@CarbonDataQA
Copy link

Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/6408/

@xuchuanyin
Copy link
Contributor Author

rebased

@CarbonDataQA
Copy link

Build Failed with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/5244/

@xuchuanyin
Copy link
Contributor Author

retest this please

@CarbonDataQA
Copy link

Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/6410/

@ravipesala
Copy link
Contributor

SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/5350/

@CarbonDataQA
Copy link

Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/5248/

@CarbonDataQA
Copy link

Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/6414/

@CarbonDataQA
Copy link

Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/6418/

@ravipesala
Copy link
Contributor

SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/5356/

@xuchuanyin
Copy link
Contributor Author

retest this please

@CarbonDataQA
Copy link

Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/6430/

@CarbonDataQA
Copy link

Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/5262/

@kumarvishal09
Copy link
Contributor

LGTM

@asfgit asfgit closed this in 55f4bc6 Jun 21, 2018
anubhav100 pushed a commit to anubhav100/incubator-carbondata that referenced this pull request Jun 22, 2018
@xuchuanyin @anubhav100
[CARBONDATA-2513][32K] Support write long string from dataframe
2d11e81 
support write long string from dataframe

This closes apache#2382
asfgit pushed a commit that referenced this pull request Jul 30, 2018
@xuchuanyin @ravipesala
[CARBONDATA-2513][32K] Support write long string from dataframe
526d125 
support write long string from dataframe

This closes #2382
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
4 participants
@xuchuanyin @CarbonDataQA @ravipesala @kumarvishal09