Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SPARK-11970] [SQL] Add function SHOW to Dataset APIs #9955

Closed
wants to merge 2 commits into from

Conversation

gatorsmile
Copy link
Member

Add the function show() to Dataset APIs. The other functions were submitted in separate PRs.

@marmbrus @rxin @cloud-fan Could you help me review the code changes?

Thank you!

@rxin
Copy link
Contributor

rxin commented Nov 25, 2015

Can you put this together with your join and sample changes?

* @param _numRows Number of rows to show
* @param truncate Whether truncate long strings and align cells right
*/
override private[sql] def showString(_numRows: Int, truncate: Boolean = true): String = {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can't we share the code between dataset and dataframe?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When merging them into the same one, I hit a problem when calling the function take. The Dataset API take returns type Array[T], but Dataframe API take has a different return type Array[Row].

Do you have a better idea how to integrate them into the same function?

@gatorsmile
Copy link
Member Author

Sure, I can combine them into the same one when you think the implementation of show is ok.

Thank you!

@SparkQA
Copy link

SparkQA commented Nov 25, 2015

Test build #46662 has finished for PR 9955 at commit a4428c3.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@rxin
Copy link
Contributor

rxin commented Nov 25, 2015

I implemented show as part of this patch: #9964

I just used toDF().show(). Can you close this pull request?

@gatorsmile
Copy link
Member Author

Yeah, that is also the solution in my original solution in the PR for EXPLAIN.
As long as you do not think they are duplicate codes, I like it. : )

@gatorsmile gatorsmile closed this Nov 25, 2015
@gatorsmile gatorsmile deleted the showDS branch December 30, 2015 17:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
3 participants