Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SPARK-27533][SQL][TEST] Date and timestamp CSV benchmarks #24429

Closed
wants to merge 6 commits into from

Conversation

MaxGekk
Copy link
Member

@MaxGekk MaxGekk commented Apr 21, 2019

What changes were proposed in this pull request?

Added new CSV benchmarks related to date and timestamps operations:

  • Write date/timestamp to CSV files
  • to_csv() and from_csv() for dates and timestamps
  • Read date/timestamps from CSV files, and infer schemas
  • Parse and infer schemas from Dataset[String]

Also existing CSV benchmarks are ported on NoOp datasource.

@MaxGekk
Copy link
Member Author

MaxGekk commented Apr 21, 2019

@HyukjinKwon Please, take a look at the PR.

@dongjoon-hyun dongjoon-hyun changed the title [SPARK-27533][SQL] Date and timestamp CSV benchmarks [SPARK-27533][SQL][TEST] Date and timestamp CSV benchmarks Apr 21, 2019
@SparkQA
Copy link

SparkQA commented Apr 21, 2019

Test build #104783 has finished for PR 24429 at commit 174c11c.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented Apr 21, 2019

Test build #104784 has finished for PR 24429 at commit f9a4912.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@HyukjinKwon
Copy link
Member

Merged to master.

@MaxGekk MaxGekk deleted the csv-timestamp-benchmark branch September 18, 2019 15:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
3 participants