[SPARK-33641][SQL] Invalidate new char/varchar types in public APIs that produce incorrect results #30586

yaooqinn · 2020-12-03T07:45:51Z

What changes were proposed in this pull request?

In this PR, we suppose to narrow the use cases of the char/varchar data types, of which are invalid now or later

Why are the changes needed?

udf

scala> spark.udf.register("abcd", () => "12345", org.apache.spark.sql.types.VarcharType(2))

scala> spark.sql("select abcd()").show
scala.MatchError: CharType(2) (of class org.apache.spark.sql.types.VarcharType)
  at org.apache.spark.sql.catalyst.encoders.RowEncoder$.externalDataTypeFor(RowEncoder.scala:215)
  at org.apache.spark.sql.catalyst.encoders.RowEncoder$.externalDataTypeForInput(RowEncoder.scala:212)
  at org.apache.spark.sql.catalyst.expressions.objects.ValidateExternalType.<init>(objects.scala:1741)
  at org.apache.spark.sql.catalyst.encoders.RowEncoder$.$anonfun$serializerFor$3(RowEncoder.scala:175)
  at scala.collection.TraversableLike.$anonfun$flatMap$1(TraversableLike.scala:245)
  at scala.collection.IndexedSeqOptimized.foreach(IndexedSeqOptimized.scala:36)
  at scala.collection.IndexedSeqOptimized.foreach$(IndexedSeqOptimized.scala:33)
  at scala.collection.mutable.ArrayOps$ofRef.foreach(ArrayOps.scala:198)
  at scala.collection.TraversableLike.flatMap(TraversableLike.scala:245)
  at scala.collection.TraversableLike.flatMap$(TraversableLike.scala:242)
  at scala.collection.mutable.ArrayOps$ofRef.flatMap(ArrayOps.scala:198)
  at org.apache.spark.sql.catalyst.encoders.RowEncoder$.serializerFor(RowEncoder.scala:171)
  at org.apache.spark.sql.catalyst.encoders.RowEncoder$.apply(RowEncoder.scala:66)
  at org.apache.spark.sql.Dataset$.$anonfun$ofRows$2(Dataset.scala:99)
  at org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:768)
  at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:96)
  at org.apache.spark.sql.SparkSession.$anonfun$sql$1(SparkSession.scala:611)
  at org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:768)
  at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:606)
  ... 47 elided

spark.createDataframe

scala> spark.createDataFrame(spark.read.text("README.md").rdd, new org.apache.spark.sql.types.StructType().add("c", "char(1)")).show
+--------------------+
|                   c|
+--------------------+
|      # Apache Spark|
|                    |
|Spark is a unifie...|
|high-level APIs i...|
|supports general ...|
|rich set of highe...|
|MLlib for machine...|
|and Structured St...|
|                    |
|<https://spark.ap...|
|                    |
|[![Jenkins Build]...|
|[![AppVeyor Build...|
|[![PySpark Covera...|
|                    |
|                    |

reader.schema

scala> spark.read.schema("a varchar(2)").text("./README.md").show(100)
+--------------------+
|                   a|
+--------------------+
|      # Apache Spark|
|                    |
|Spark is a unifie...|
|high-level APIs i...|
|supports general ...|

etc

Does this PR introduce any user-facing change?

NO, we intend to avoid protentical breaking change

How was this patch tested?

new tests

…produce incorrect results

SparkQA · 2020-12-03T09:01:52Z

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/36703/

SparkQA · 2020-12-03T09:04:54Z

Test build #132103 has finished for PR 30586 at commit 183db1a.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

SparkQA · 2020-12-03T09:29:33Z

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/36703/

yaooqinn · 2020-12-03T13:00:11Z

cc @cloud-fan @maropu thanks~

sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala

SparkQA · 2020-12-03T14:00:45Z

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/36735/

SparkQA · 2020-12-03T14:28:39Z

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/36735/

SparkQA · 2020-12-03T15:09:38Z

Test build #132134 has finished for PR 30586 at commit 9f24463.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

cloud-fan · 2020-12-03T17:02:59Z

sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/CharVarcharUtils.scala

+   */
+  def failWithCharLikeTypes(schema: StructType): StructType = {
+    if (schema.exists(f => hasCharVarchar(f.dataType) ||
+      f.metadata.contains(CHAR_VARCHAR_TYPE_STRING_METADATA_KEY))) {


when do we need to check the metadata?

We can assume that users won't set the very internal __CHAR_VARCHAR_TYPE_STRING metadata key, and only check with the explicit char/varchar type.

cloud-fan · 2020-12-03T17:07:52Z

sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/CharVarcharUtils.scala

+   * @param dt the given struct type
+   * @return
+   */
+  def failOrWarnWithCharLikeType(dt: DataType, fail: Boolean = true): DataType = {


let's have 2 methods instead of using a boolean flag.

def failIfHasCharVarchar(dt: DataType)... def warnIfHasCharVarchar(dt: DataType)...

SparkQA · 2020-12-03T17:07:53Z

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/36745/

SparkQA · 2020-12-03T17:38:39Z

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/36745/

SparkQA · 2020-12-03T17:38:44Z

Test build #132141 has finished for PR 30586 at commit 148ee63.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

SparkQA · 2020-12-03T18:16:57Z

Test build #132144 has finished for PR 30586 at commit 6bbebf6.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

SparkQA · 2020-12-04T03:31:01Z

Test build #132182 has finished for PR 30586 at commit 7ccb99b.

This patch fails to build.
This patch merges cleanly.
This patch adds no public classes.

SparkQA · 2020-12-04T04:07:49Z

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/36782/

SparkQA · 2020-12-04T04:31:04Z

Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/36782/

cloud-fan · 2020-12-04T05:57:40Z

sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/ExprUtils.scala

-        case s: UTF8String if s != null => DataType.fromDDL(s.toString)
+        case s: UTF8String if s != null =>
+          val dataType = DataType.fromDDL(s.toString)
+          CharVarcharUtils.replaceCharVarcharWithString(dataType)


shall we give a warning here?

SparkQA · 2020-12-04T06:27:06Z

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/36792/

SparkQA · 2020-12-04T06:55:29Z

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/36792/

SparkQA · 2020-12-04T07:24:10Z

Test build #132201 has finished for PR 30586 at commit f83c4af.

This patch fails Scala style tests.
This patch merges cleanly.
This patch adds no public classes.

SparkQA · 2020-12-05T06:28:45Z

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/36863/

SparkQA · 2020-12-05T06:41:08Z

Test build #132262 has finished for PR 30586 at commit 2ef9a88.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
class OneVsRest(Estimator, _OneVsRestParams, HasParallelism, MLReadable, MLWritable):
class OneVsRestReader(MLReader):
class OneVsRestWriter(MLWriter):
class OneVsRestModel(Model, _OneVsRestParams, MLReadable, MLWritable):
class OneVsRestModelReader(MLReader):
class OneVsRestModelWriter(MLWriter):
class CrossValidatorReader(MLReader):
class CrossValidatorWriter(MLWriter):
class CrossValidatorModelReader(MLReader):
class CrossValidatorModelWriter(MLWriter):
class TrainValidationSplitReader(MLReader):
class TrainValidationSplitWriter(MLWriter):
class TrainValidationSplitModelReader(MLReader):
class TrainValidationSplitModelWriter(MLWriter):
case class TemporaryViewRelation(tableMeta: CatalogTable) extends LeafNode
case class CreateNamedStruct(children: Seq[Expression]) extends Expression with NoThrow
case class DateFromUnixDate(child: Expression) extends UnaryExpression
case class UnixDate(child: Expression) extends UnaryExpression
case class UnixSeconds(child: Expression) extends TimestampToLongBase
case class UnixMillis(child: Expression) extends TimestampToLongBase
case class UnixMicros(child: Expression) extends TimestampToLongBase
case class MakeDate(
class JDBCTableCatalog extends TableCatalog with SupportsNamespaces with Logging
class StreamingQueryListenerBus(sparkListenerBus: Option[LiveListenerBus])
case class StateSchemaNotCompatible(message: String) extends Exception(message)
class StateSchemaCompatibilityChecker(
class StreamingQueryHistoryServerPlugin extends AppHistoryServerPlugin
class StreamingQueryStatusStore(store: KVStore)

SparkQA · 2020-12-05T07:03:34Z

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/36863/

yaooqinn · 2020-12-06T12:09:04Z

retest this please

SparkQA · 2020-12-06T13:09:32Z

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/36890/

SparkQA · 2020-12-06T13:36:20Z

Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/36890/

SparkQA · 2020-12-06T16:55:49Z

Test build #132289 has finished for PR 30586 at commit 2ef9a88.

This patch passes all tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
class OneVsRest(Estimator, _OneVsRestParams, HasParallelism, MLReadable, MLWritable):
class OneVsRestReader(MLReader):
class OneVsRestWriter(MLWriter):
class OneVsRestModel(Model, _OneVsRestParams, MLReadable, MLWritable):
class OneVsRestModelReader(MLReader):
class OneVsRestModelWriter(MLWriter):
class CrossValidatorReader(MLReader):
class CrossValidatorWriter(MLWriter):
class CrossValidatorModelReader(MLReader):
class CrossValidatorModelWriter(MLWriter):
class TrainValidationSplitReader(MLReader):
class TrainValidationSplitWriter(MLWriter):
class TrainValidationSplitModelReader(MLReader):
class TrainValidationSplitModelWriter(MLWriter):
case class TemporaryViewRelation(tableMeta: CatalogTable) extends LeafNode
case class CreateNamedStruct(children: Seq[Expression]) extends Expression with NoThrow
case class DateFromUnixDate(child: Expression) extends UnaryExpression
case class UnixDate(child: Expression) extends UnaryExpression
case class UnixSeconds(child: Expression) extends TimestampToLongBase
case class UnixMillis(child: Expression) extends TimestampToLongBase
case class UnixMicros(child: Expression) extends TimestampToLongBase
case class MakeDate(
class JDBCTableCatalog extends TableCatalog with SupportsNamespaces with Logging
class StreamingQueryListenerBus(sparkListenerBus: Option[LiveListenerBus])
case class StateSchemaNotCompatible(message: String) extends Exception(message)
class StateSchemaCompatibilityChecker(
class StreamingQueryHistoryServerPlugin extends AppHistoryServerPlugin
class StreamingQueryStatusStore(store: KVStore)

maropu · 2020-12-06T23:56:12Z

sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/CharVarcharUtils.scala

+   */
+  def failIfHasCharVarchar(dt: DataType): DataType = {
+    if (!SQLConf.get.charVarcharAsString && hasCharVarchar(dt)) {
+      throw new AnalysisException(s"char/varchar type can only be used in the table schema")


nit: remove s at the beginning.

maropu · 2020-12-07T00:00:04Z

sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala

+  val LEGACY_CHAR_VARCHAR_AS_STRING =
+    buildConf("spark.sql.legacy.charVarcharAsString")
+      .internal()
+      .doc("When true, the parser will not fail to parse char and varchar type but treat it as" +


It seems this config is not related only to the parser because failIfHasCharVarchar is used in many places. Could you update it accordingly?

seems that they are all related to the parser and actually char/varchar shouldn't exist after the parser phase

Ah, ok. If so, how about spark.sql. -> spark.sql.parser?

for example, spark.read.schema(StructType) has no parser involved.

maropu · 2020-12-07T00:06:05Z

sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/CharVarcharUtils.scala

+   */
+  def replaceCharVarcharWithStringForCast(dt: DataType): DataType = {
+    if (hasCharVarchar(dt)) {
+      logWarning("The Spark cast operator does not support char/varchar type and simply treat" +


nit: treat -> treats

maropu · 2020-12-07T00:07:47Z

sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/CharVarcharUtils.scala

+  def replaceCharVarcharWithStringForCast(dt: DataType): DataType = {
+    if (hasCharVarchar(dt)) {
+      logWarning("The Spark cast operator does not support char/varchar type and simply treat" +
+        " them as string type. Please use string type directly to avoid confusion.")


Could you add tests for this warning msg by using LogAppender?

cloud-fan · 2020-12-07T05:22:27Z

sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/CharVarcharUtils.scala

+   */
+  def failIfHasCharVarchar(dt: DataType): DataType = {
+    if (!SQLConf.get.charVarcharAsString && hasCharVarchar(dt)) {
+      throw new AnalysisException("char/varchar type can only be used in the table schema")


can we mention the config name?

cloud-fan · 2020-12-07T05:23:07Z

sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/CharVarcharUtils.scala

+   * warning message if it has char or varchar types
+   */
+  def replaceCharVarcharWithStringForCast(dt: DataType): DataType = {
+    if (hasCharVarchar(dt)) {


shall we skip warning if the config charVarcharAsString is true?

cloud-fan

LGTM except some minor comments.

SparkQA · 2020-12-07T07:17:55Z

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/36934/

SparkQA · 2020-12-07T07:22:51Z

Test build #132319 has finished for PR 30586 at commit 1f47ce0.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

SparkQA · 2020-12-07T07:43:04Z

Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/36934/

yaooqinn · 2020-12-07T07:56:09Z

retest this please

SparkQA · 2020-12-07T09:15:49Z

Test build #132345 has finished for PR 30586 at commit 10544a6.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

cloud-fan · 2020-12-07T09:20:52Z

retest this please

SparkQA · 2020-12-07T09:31:12Z

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/36945/

SparkQA · 2020-12-07T10:03:44Z

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/36945/

SparkQA · 2020-12-07T10:24:07Z

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/36953/

SparkQA · 2020-12-07T10:47:22Z

Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/36953/

SparkQA · 2020-12-07T10:59:57Z

Test build #132334 has finished for PR 30586 at commit 10544a6.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

cloud-fan · 2020-12-07T13:40:24Z

thanks, merging to master/3.1 as it's necessary for the char/varchar support.

…hat produce incorrect results ### What changes were proposed in this pull request? In this PR, we suppose to narrow the use cases of the char/varchar data types, of which are invalid now or later ### Why are the changes needed? 1. udf ```scala scala> spark.udf.register("abcd", () => "12345", org.apache.spark.sql.types.VarcharType(2)) scala> spark.sql("select abcd()").show scala.MatchError: CharType(2) (of class org.apache.spark.sql.types.VarcharType) at org.apache.spark.sql.catalyst.encoders.RowEncoder$.externalDataTypeFor(RowEncoder.scala:215) at org.apache.spark.sql.catalyst.encoders.RowEncoder$.externalDataTypeForInput(RowEncoder.scala:212) at org.apache.spark.sql.catalyst.expressions.objects.ValidateExternalType.<init>(objects.scala:1741) at org.apache.spark.sql.catalyst.encoders.RowEncoder$.$anonfun$serializerFor$3(RowEncoder.scala:175) at scala.collection.TraversableLike.$anonfun$flatMap$1(TraversableLike.scala:245) at scala.collection.IndexedSeqOptimized.foreach(IndexedSeqOptimized.scala:36) at scala.collection.IndexedSeqOptimized.foreach$(IndexedSeqOptimized.scala:33) at scala.collection.mutable.ArrayOps$ofRef.foreach(ArrayOps.scala:198) at scala.collection.TraversableLike.flatMap(TraversableLike.scala:245) at scala.collection.TraversableLike.flatMap$(TraversableLike.scala:242) at scala.collection.mutable.ArrayOps$ofRef.flatMap(ArrayOps.scala:198) at org.apache.spark.sql.catalyst.encoders.RowEncoder$.serializerFor(RowEncoder.scala:171) at org.apache.spark.sql.catalyst.encoders.RowEncoder$.apply(RowEncoder.scala:66) at org.apache.spark.sql.Dataset$.$anonfun$ofRows$2(Dataset.scala:99) at org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:768) at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:96) at org.apache.spark.sql.SparkSession.$anonfun$sql$1(SparkSession.scala:611) at org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:768) at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:606) ... 47 elided ``` 2. spark.createDataframe ``` scala> spark.createDataFrame(spark.read.text("README.md").rdd, new org.apache.spark.sql.types.StructType().add("c", "char(1)")).show +--------------------+ | c| +--------------------+ | # Apache Spark| | | |Spark is a unifie...| |high-level APIs i...| |supports general ...| |rich set of highe...| |MLlib for machine...| |and Structured St...| | | |<https://spark.ap...| | | |[![Jenkins Build]...| |[![AppVeyor Build...| |[![PySpark Covera...| | | | | ``` 3. reader.schema ``` scala> spark.read.schema("a varchar(2)").text("./README.md").show(100) +--------------------+ | a| +--------------------+ | # Apache Spark| | | |Spark is a unifie...| |high-level APIs i...| |supports general ...| ``` 4. etc ### Does this PR introduce _any_ user-facing change? NO, we intend to avoid protentical breaking change ### How was this patch tested? new tests Closes #30586 from yaooqinn/SPARK-33641. Authored-by: Kent Yao <yaooqinn@hotmail.com> Signed-off-by: Wenchen Fan <wenchen@databricks.com> (cherry picked from commit da72b87) Signed-off-by: Wenchen Fan <wenchen@databricks.com>

cloud-fan · 2020-12-07T13:42:15Z

After a second thought, although char/varchar is kind of hidden before 3.1, it's still possible for users to use it already. @yaooqinn can we create a followup PR to add a migration guide?

SparkQA · 2020-12-07T13:42:40Z

Test build #132353 has finished for PR 30586 at commit 10544a6.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

yaooqinn · 2020-12-07T14:04:59Z

After a second thought, although char/varchar is kind of hidden before 3.1, it's still possible for users to use it already. @yaooqinn can we create a followup PR to add a migration guide?
OK~

[SPARK-33641][SQL] Invalidate new char-like type in public APIs that …

183db1a

…produce incorrect results

github-actions bot added the SQL label Dec 3, 2020

fix test

9f24463

cloud-fan reviewed Dec 3, 2020

View reviewed changes

sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala Outdated Show resolved Hide resolved

yaooqinn added 2 commits December 3, 2020 23:28

refine

148ee63

allow cast

6bbebf6

cloud-fan reviewed Dec 3, 2020

View reviewed changes

adress cloudfan's comments

7ccb99b

yaooqinn changed the title ~~[SPARK-33641][SQL] Invalidate new char-like type in public APIs that produce incorrect results~~ [SPARK-33641][SQL] Invalidate new char-like types in public APIs that produce incorrect results Dec 4, 2020

fix build

7c1dff1

cloud-fan reviewed Dec 4, 2020

View reviewed changes

naming

f83c4af

maropu reviewed Dec 7, 2020

View reviewed changes

addr maropu's comment

1f47ce0

cloud-fan reviewed Dec 7, 2020

View reviewed changes

cloud-fan approved these changes Dec 7, 2020

View reviewed changes

add config in err

10544a6

cloud-fan closed this in da72b87 Dec 7, 2020

[SPARK-33641][SQL] Invalidate new char/varchar types in public APIs that produce incorrect results #30586

[SPARK-33641][SQL] Invalidate new char/varchar types in public APIs that produce incorrect results #30586

Conversation

yaooqinn commented Dec 3, 2020 • edited by maropu Loading

What changes were proposed in this pull request?

Why are the changes needed?

Does this PR introduce any user-facing change?

How was this patch tested?

SparkQA commented Dec 3, 2020

SparkQA commented Dec 3, 2020

SparkQA commented Dec 3, 2020

yaooqinn commented Dec 3, 2020

SparkQA commented Dec 3, 2020

SparkQA commented Dec 3, 2020

SparkQA commented Dec 3, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

SparkQA commented Dec 3, 2020

SparkQA commented Dec 3, 2020

SparkQA commented Dec 3, 2020

SparkQA commented Dec 3, 2020

SparkQA commented Dec 4, 2020

SparkQA commented Dec 4, 2020

SparkQA commented Dec 4, 2020

Choose a reason for hiding this comment

SparkQA commented Dec 4, 2020

SparkQA commented Dec 4, 2020

SparkQA commented Dec 4, 2020

SparkQA commented Dec 5, 2020

SparkQA commented Dec 5, 2020

SparkQA commented Dec 5, 2020

yaooqinn commented Dec 6, 2020

SparkQA commented Dec 6, 2020

SparkQA commented Dec 6, 2020

SparkQA commented Dec 6, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cloud-fan left a comment

Choose a reason for hiding this comment

SparkQA commented Dec 7, 2020

SparkQA commented Dec 7, 2020

SparkQA commented Dec 7, 2020

yaooqinn commented Dec 7, 2020

SparkQA commented Dec 7, 2020

cloud-fan commented Dec 7, 2020

SparkQA commented Dec 7, 2020

SparkQA commented Dec 7, 2020

SparkQA commented Dec 7, 2020

SparkQA commented Dec 7, 2020

SparkQA commented Dec 7, 2020

cloud-fan commented Dec 7, 2020

cloud-fan commented Dec 7, 2020

SparkQA commented Dec 7, 2020

yaooqinn commented Dec 7, 2020

yaooqinn commented Dec 3, 2020 •

edited by maropu

Loading