Skip to content

Commit

Permalink
[SQL][MINOR] Fix compiling for scala 2.12
Browse files Browse the repository at this point in the history
## What changes were proposed in this pull request?
Introduced by #21320 and #11744

```
$ sbt
> ++2.12.6
> project sql
> compile
...
[error] [warn] spark/sql/core/src/main/scala/org/apache/spark/sql/execution/ProjectionOverSchema.scala:41: match may not be exhaustive.
[error] It would fail on the following inputs: (_, ArrayType(_, _)), (_, _)
[error] [warn]         getProjection(a.child).map(p => (p, p.dataType)).map {
[error] [warn]
[error] [warn] spark/sql/core/src/main/scala/org/apache/spark/sql/execution/ProjectionOverSchema.scala:52: match may not be exhaustive.
[error] It would fail on the following input: (_, _)
[error] [warn]         getProjection(child).map(p => (p, p.dataType)).map {
[error] [warn]
...
```

And

```
$ sbt
> ++2.12.6
> project hive
> testOnly *ParquetMetastoreSuite
...
[error] /Users/rendong/wdi/spark/sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveSparkSubmitSuite.scala:22: object tools is not a member of package scala
[error] import scala.tools.nsc.Properties
[error]              ^
[error] /Users/rendong/wdi/spark/sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveSparkSubmitSuite.scala:146: not found: value Properties
[error]     val version = Properties.versionNumberString match {
[error]                   ^
[error] two errors found
...
```

## How was this patch tested?
Existing tests.

Closes #22260 from sadhen/fix_exhaustive_match.

Authored-by: 忍冬 <rendong@wacai.com>
Signed-off-by: hyukjinkwon <gurwls223@apache.org>
  • Loading branch information
忍冬 authored and HyukjinKwon committed Aug 30, 2018
1 parent 3a66a7f commit 56bc700
Show file tree
Hide file tree
Showing 2 changed files with 9 additions and 1 deletion.
Expand Up @@ -45,13 +45,21 @@ private[execution] case class ProjectionOverSchema(schema: StructType) {
projSchema.fieldIndex(a.field.name),
projSchema.size,
a.containsNull)
case (_, projSchema) =>
throw new IllegalStateException(
s"unmatched child schema for GetArrayStructFields: ${projSchema.toString}"
)
}
case GetMapValue(child, key) =>
getProjection(child).map { projection => GetMapValue(projection, key) }
case GetStructFieldObject(child, field: StructField) =>
getProjection(child).map(p => (p, p.dataType)).map {
case (projection, projSchema: StructType) =>
GetStructField(projection, projSchema.fieldIndex(field.name))
case (_, projSchema) =>
throw new IllegalStateException(
s"unmatched child schema for GetStructField: ${projSchema.toString}"
)
}
case _ =>
None
Expand Down
Expand Up @@ -19,7 +19,7 @@ package org.apache.spark.sql.hive

import java.io.{BufferedWriter, File, FileWriter}

import scala.tools.nsc.Properties
import scala.util.Properties

import org.apache.hadoop.fs.Path
import org.scalatest.{BeforeAndAfterEach, Matchers}
Expand Down

0 comments on commit 56bc700

Please sign in to comment.