Skip to content

Commit

Permalink
[SPARK-26897][SQL][TEST][FOLLOW-UP] Remove workaround for 2.2.0 and 2…
Browse files Browse the repository at this point in the history
….1.x in HiveExternalCatalogVersionsSuite

## What changes were proposed in this pull request?
This pr just removed workaround for 2.2.0 and 2.1.x in HiveExternalCatalogVersionsSuite.

## How was this patch tested?
Pass the Jenkins.

Closes #23817 from maropu/SPARK-26607-FOLLOWUP.

Authored-by: Takeshi Yamamuro <yamamuro@apache.org>
Signed-off-by: Hyukjin Kwon <gurwls223@apache.org>
(cherry picked from commit e2b8cc6)
Signed-off-by: Hyukjin Kwon <gurwls223@apache.org>
  • Loading branch information
maropu authored and HyukjinKwon committed Feb 18, 2019
1 parent 0d0c9ff commit d38a113
Showing 1 changed file with 4 additions and 13 deletions.
Expand Up @@ -257,19 +257,10 @@ object PROCESS_TABLES extends QueryTest with SQLTestUtils {

// SPARK-22356: overlapped columns between data and partition schema in data source tables
val tbl_with_col_overlap = s"tbl_with_col_overlap_$index"
// For Spark 2.2.0 and 2.1.x, the behavior is different from Spark 2.0, 2.2.1, 2.3+
if (testingVersions(index).startsWith("2.1") || testingVersions(index) == "2.2.0") {
spark.sql("msck repair table " + tbl_with_col_overlap)
assert(spark.table(tbl_with_col_overlap).columns === Array("i", "j", "p"))
checkAnswer(spark.table(tbl_with_col_overlap), Row(1, 1, 1) :: Row(1, 1, 1) :: Nil)
assert(sql("desc " + tbl_with_col_overlap).select("col_name")
.as[String].collect().mkString(",").contains("i,j,p"))
} else {
assert(spark.table(tbl_with_col_overlap).columns === Array("i", "p", "j"))
checkAnswer(spark.table(tbl_with_col_overlap), Row(1, 1, 1) :: Row(1, 1, 1) :: Nil)
assert(sql("desc " + tbl_with_col_overlap).select("col_name")
.as[String].collect().mkString(",").contains("i,p,j"))
}
assert(spark.table(tbl_with_col_overlap).columns === Array("i", "p", "j"))
checkAnswer(spark.table(tbl_with_col_overlap), Row(1, 1, 1) :: Row(1, 1, 1) :: Nil)
assert(sql("desc " + tbl_with_col_overlap).select("col_name")
.as[String].collect().mkString(",").contains("i,p,j"))
}
}
}

0 comments on commit d38a113

Please sign in to comment.