Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[SPARK-3299][SQL]Public API in SQLContext to list tables #4547

Closed
wants to merge 6 commits into from
Closed

[SPARK-3299][SQL]Public API in SQLContext to list tables #4547

wants to merge 6 commits into from

Conversation

yhuai
Copy link
Contributor

@yhuai yhuai commented Feb 12, 2015

@yhuai yhuai mentioned this pull request Feb 12, 2015
Closed
@SparkQA
Copy link

SparkQA commented Feb 12, 2015

Test build #27316 has started for PR 4547 at commit aba2e88.

  • This patch merges cleanly.

rxin
rxin reviewed Feb 12, 2015
View reviewed changes
sql/hive/src/test/scala/org/apache/spark/sql/hive/ListTablesSuite.scala
sql("DROP DATABASE IF EXISTS db1")
}

test("get All Tables of current database") {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lower case all?

@SparkQA
Copy link

SparkQA commented Feb 12, 2015

Test build #27316 has finished for PR 4547 at commit aba2e88.

  • This patch fails Spark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@AmplabJenkins
Copy link

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/27316/
Test FAILed.

@yhuai
Copy link
Contributor Author

yhuai commented Feb 12, 2015

Actually, I cannot use TestSQLContext and TestHive in my test (they have lots of tables registered by other tests). Will find another way.

@marmbrus
Copy link
Contributor

You could just add and remove a table to make sure the list reflects this
change.
On Feb 11, 2015 8:29 PM, "Yin Huai" notifications@github.com wrote:

Actually, I cannot use TestSQLContext and TestHive in my test (they have
lots of tables registered by other tests). Will find another way.

—
Reply to this email directly or view it on GitHub
#4547 (comment).

@SparkQA
Copy link

SparkQA commented Feb 12, 2015

Test build #27324 has started for PR 4547 at commit 7793dcb.

  • This patch merges cleanly.

@SparkQA
Copy link

SparkQA commented Feb 12, 2015

Test build #27325 has started for PR 4547 at commit acbb281.

  • This patch merges cleanly.

@SparkQA
Copy link

SparkQA commented Feb 12, 2015

Test build #27324 has finished for PR 4547 at commit 7793dcb.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@AmplabJenkins
Copy link

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/27324/
Test PASSed.

@SparkQA
Copy link

SparkQA commented Feb 12, 2015

Test build #27325 has finished for PR 4547 at commit acbb281.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@AmplabJenkins
Copy link

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/27325/
Test PASSed.

@SparkQA
Copy link

SparkQA commented Feb 12, 2015

Test build #27395 has started for PR 4547 at commit 6c8f92e.

  • This patch merges cleanly.

@SparkQA
Copy link

SparkQA commented Feb 13, 2015

Test build #27395 has finished for PR 4547 at commit 6c8f92e.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@AmplabJenkins
Copy link

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/27395/
Test PASSed.

asfgit pushed a commit that referenced this pull request Feb 13, 2015
@yhuai @marmbrus
[SPARK-3299][SQL]Public API in SQLContext to list tables
edbac17 
https://issues.apache.org/jira/browse/SPARK-3299

Author: Yin Huai <yhuai@databricks.com>

Closes #4547 from yhuai/tables and squashes the following commits:

6c8f92e [Yin Huai] Add tableNames.
acbb281 [Yin Huai] Update Python test.
7793dcb [Yin Huai] Fix scala test.
572870d [Yin Huai] Address comments.
aba2e88 [Yin Huai] Format.
12c86df [Yin Huai] Add tables() to SQLContext to return a DataFrame containing existing tables.

(cherry picked from commit 1d0596a)
Signed-off-by: Michael Armbrust <michael@databricks.com>
@asfgit asfgit closed this in 1d0596a Feb 13, 2015
rxin
rxin reviewed Feb 13, 2015
View reviewed changes
sql/core/src/main/scala/org/apache/spark/sql/SQLContext.scala
@@ -734,6 +734,42 @@ class SQLContext(@transient val sparkContext: SparkContext)
def table(tableName: String): DataFrame =
DataFrame(this, catalog.lookupRelation(Seq(tableName)))

/**
* Returns a [[DataFrame]] containing names of existing tables in the given database.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the javadoc is actually wrong. this one should say "current database", and next one should say "given database"

@yhuai
Copy link
Contributor Author

yhuai commented Feb 13, 2015

I have created #4579 for the doc fix.

@yhuai yhuai deleted the tables branch February 13, 2015 02:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@yhuai @SparkQA @AmplabJenkins @marmbrus @rxin