Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SPARK-15473][SQL] CSV data source writes header for empty data to be read back as column names #13252

Closed
wants to merge 6 commits into from

Conversation

HyukjinKwon
Copy link
Member

@HyukjinKwon HyukjinKwon commented May 22, 2016

What changes were proposed in this pull request?

This PR lets the CSV data source write the header when header option is true even if given data is empty. Currently, this is not writing header in this case.

How was this patch tested?

Unit tests in CSVSuite.scala.

@SparkQA
Copy link

SparkQA commented May 22, 2016

Test build #59101 has finished for PR 13252 at commit 9e50552.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented May 24, 2016

Test build #59172 has finished for PR 13252 at commit f38569a.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds the following public classes (experimental):
    • case class ListFilesCommand(files: Seq[String] = Seq.empty[String]) extends RunnableCommand
    • case class ListJarsCommand(jars: Seq[String] = Seq.empty[String]) extends RunnableCommand

@SparkQA
Copy link

SparkQA commented May 26, 2016

Test build #59365 has finished for PR 13252 at commit 67d6ee7.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@HyukjinKwon HyukjinKwon reopened this Jun 10, 2016
@HyukjinKwon HyukjinKwon changed the title [SPARK-15473][SQL] CSV data source fails to write and read back empty data [SPARK-15473][SQL] CSV data source writes header when it is enabled and data is empty Jun 10, 2016
@HyukjinKwon HyukjinKwon changed the title [SPARK-15473][SQL] CSV data source writes header when it is enabled and data is empty [SPARK-15473][SQL] CSV data source fails to write and read back empty data unlike JSON data source Jun 10, 2016
@SparkQA
Copy link

SparkQA commented Jun 10, 2016

Test build #60284 has finished for PR 13252 at commit 67d6ee7.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@HyukjinKwon HyukjinKwon changed the title [SPARK-15473][SQL] CSV data source fails to write and read back empty data unlike JSON data source [SPARK-15473][SQL] CSV data source writes header for empty data to be read back as column names Aug 10, 2016
@SparkQA
Copy link

SparkQA commented Aug 10, 2016

Test build #63486 has finished for PR 13252 at commit 063a565.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented Aug 30, 2016

Test build #64633 has finished for PR 13252 at commit 031c9da.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented Aug 31, 2016

Test build #64693 has finished for PR 13252 at commit 031c9da.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented Oct 23, 2016

Test build #67421 has finished for PR 13252 at commit 031c9da.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@HyukjinKwon
Copy link
Member Author

Let me suggest a generalized way latter because it does not look a clean fix.

@HyukjinKwon HyukjinKwon closed this Jan 5, 2017
@jklewa
Copy link

jklewa commented Oct 18, 2017

Sorry to bring up a closed PR but has this issue been resolved? SPARK-15473 is "in progress" but hasn't been updated in ~7 months.

@HyukjinKwon
Copy link
Member Author

HyukjinKwon commented Oct 19, 2017

It's not resolved yet. I am not working on this for now. Please take over this if you are willing to do.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
3 participants