[HUDI-6198] Support Hudi on Spark 3.4.0 #8885

yihua · 2023-06-05T06:42:02Z

Change Logs

This PR adds the support of Apache Hudi on Spark 3.4.0.

There are a few changes in Spark 3.4.0 that the Hudi Spark integration needs to adjust to, particularly:

SQLConf.PARQUET_INFER_TIMESTAMP_NTZ_ENABLED (spark.sql.parquet.inferTimestampNTZ.enabled) is required for parquet reading
case class changes of MergeIntoTable (five to six arguments)

Before Spark 3.4.0 (five arguments):
 case class MergeIntoTable(
    targetTable: LogicalPlan,
    sourceTable: LogicalPlan,
    mergeCondition: Expression,
    matchedActions: Seq[MergeAction],
    notMatchedActions: Seq[MergeAction]) extends BinaryCommand with SupportsSubquery
   
Since Spark 3.4.0 (six arguments):
 case class MergeIntoTable(
    targetTable: LogicalPlan,
    sourceTable: LogicalPlan,
    mergeCondition: Expression,
    matchedActions: Seq[MergeAction],
    notMatchedActions: Seq[MergeAction],
    notMatchedBySourceActions: Seq[MergeAction]) extends BinaryCommand with SupportsSubquery

API change of SchemaUtils.checkColumnNameDuplication
PartitionedFile change

Before Spark 3.4.0,
case class PartitionedFile(
  partitionValues: InternalRow,
  filePath: String,
  start: Long,
  @transient locations: Array[String] = Array.empty)

Since Spark 3.4.0, the filePath is switch to [[SparkPath]] for type safety:
case class PartitionedFile(
  partitionValues: InternalRow,
  filePath: SparkPath,
  start: Long,
  length: Long,
  @transient locations: Array[String] = Array.empty,
  modificationTime: Long = 0L,
  fileSize: Long = 0L)

To support Apache Hudi on Spark 3.4.0, here the changes made:

Adds adapter support for Spark 3.4 in SparkAdapterSupport and HoodieSparkUtils.
Adds adapter framework changes and Hudi-related logic adjustment:
- To account for the config SQLConf.PARQUET_INFER_TIMESTAMP_NTZ_ENABLED (spark.sql.parquet.inferTimestampNTZ.enabled) required for parquet reading in Spark 3.4.0, the config is set now in HoodieSparkFileReaderFactory.
- To account for case class changes of MergeIntoTable (five to six arguments), a new function def unapplyMergeIntoTable(plan: LogicalPlan): Option[(LogicalPlan, LogicalPlan, Expression)] in HoodieCatalystPlansUtils` and corresponding implementation in each Spark version are added in the Spark adapter framework.
- To account for the API change of SchemaUtils.checkColumnNameDuplication, HoodieSchemaUtils and checkColumnNameDuplication function and corresponding implementation in each Spark version are added to the Spark adapter framework.
- To account for PartitionedFile change where the filePath is switch to [[SparkPath]] for type safety, the HoodieSparkPartitionedFileUtils and corresponding implementation in each Spark version are added to the Spark adapter framework.
- Creates BASE_PATH_PARAM = "basePath" as Spark 3.4.0 uses FileIndexOptions.BASE_PATH_PARAM instead PartitioningAwareFileIndex.BASE_PATH_PARAM
- Uses the above adapter functions to adjust logic in BaseFileOnlyRelation, HoodieBaseRelation, HoodieBootstrapRelation, HoodieDataSourceHelper, Iterators, MergeOnReadSnapshotRelation, HoodieCDCRDD, AlterHoodieTableAddColumnsCommand, and HoodieAnalysis.
Spark 3.4 now supports vectorized reader on nested fields. However, Hudi does not support this yet due to custom schema evolution logic. So we add logic to override supportBatch in HoodieParquetFileFormat for Spark 3.4.
Uses ColumnVector instead of WritableColumnVector to get general in Spark32PlusHoodieVectorizedParquetRecordReader
Adds hudi-spark3.4.x module by mainly copying the code from hudi-spark3.3.x module and making following adjustments (more details below on the code comparison):
- Spark32PlusHoodieParquetFileFormat is removed. Each spark version module has its own file format class. Spark34HoodieParquetFileFormat adds custom logic on when to enable vectorized reader based on various conditions, compared to Spark33HoodieParquetFileFormat.
- HoodieSpark34PartitionedFileUtils is different from HoodieSpark33PartitionedFileUtils because of PartitionedFile changes
- Spark34ResolveHudiAlterTableCommand adjusts logic based on AlterColumn class changes.
- HoodieSpark3_4ExtendedSqlAstBuilder adjusts logic based on Spark 3.4.
- HoodieSpark34CatalystExpressionUtils adjusts logic based on ParseToTimestamp change and removal of AnsiCast compared to HoodieSpark33CatalystExpressionUtils.
- HoodieSpark34CatalystPlanUtils adjusts logic based on MergeIntoTable change.
- HoodieSpark34SchemaUtils is different from HoodieSpark33SchemaUtils because of API change of SchemaUtils.checkColumnNameDuplication
Adds Github Actions to run Spark tests and bundle validation on Spark 3.4.0.
Adjusts tests due to changes in Spark 3.4.0.
- TestHoodieRecordSerialization: expected record size has changed.
- Error messages for Spark SQL have changed in a few tests in TestInsertTable, TestMergeIntoTable, TestIndexSyntax (table.identifier.quotedString now also has catalog name as the prefix), TestCopyToTempViewProcedure. New error messages in Spark SQL are presented in https://github.com/apache/spark/blob/master/core/src/main/resources/error/error-classes.json.

This PR address all compatibility issues in Hudi Spark integration across Spark versions on top of the #8826, which adds support of Hudi for Spark 3.4.0 on Hudi 0.13.1 release only. @mansipp, @rahil-c, @CTTY, and @umehrot2 have significantly contributed to #8826 to make Hudi 0.13.1 release work on Spark 3.4.0 for EMR release. This PR makes sure all recent changes in Hudi Spark integration since Hudi 0.13.1 are considered and taken care of, including:

More details

Major changes from hudi-spark3.3.x to hudi-spark3.4.x module:
Spark33HoodieParquetFileFormat vs Spark34HoodieParquetFileFormat:

HoodieSpark33PartitionedFileUtils vs HoodieSpark34PartitionedFileUtils:

Spark33ResolveHudiAlterTableCommand vs Spark34ResolveHudiAlterTableCommand:

HoodieSpark3_3ExtendedSqlAstBuilder vs HoodieSpark3_4ExtendedSqlAstBuilder:

HoodieSpark33CatalystExpressionUtils vs HoodieSpark34CatalystExpressionUtils

HoodieSpark33CatalystPlanUtils vs HoodieSpark34CatalystPlanUtils

HoodieSpark33SchemaUtils vs HoodieSpark34SchemaUtils

Impact

This enables Spark 3.4.0 to run Hudi workload.

Risk level

medium

The PR is thoroughly tested by the Java CI, Azure CI, and integration tests on EMR and S3.

Documentation Update

We'll update Hudi website on the Spark 3.4.0 support: HUDI-6341.

Contributor's checklist

Read through contributor's guide
Change Logs and Impact were stated clearly
Adequate tests were added if applicable
CI passed

melin · 2023-06-06T02:49:23Z

spark 2.4 and spark 3.0 are not supported by iceberg recently. Does hudi consider reducing the maintenance of the version

yihua · 2023-06-06T15:49:59Z

spark 2.4 and spark 3.0 are not supported by iceberg recently. Does hudi consider reducing the maintenance of the version

Maintaining compatibility across multiple Spark versions is indeed challenging and time-taking. However, there are still users using Hudi on Spark 2.4 and 3.0, so we cannot drop the support on Spark 2.4 and 3.0 at least for the next two releases.

pom.xml

hudi-common/pom.xml

hudi-common/src/main/java/org/apache/hudi/common/util/JsonUtils.java

...rce/hudi-spark3.2.x/src/main/scala/org/apache/spark/sql/HoodieSpark32CatalystPlanUtils.scala

...cala/org/apache/spark/sql/execution/datasources/parquet/Spark34HoodieParquetFileFormat.scala

yihua · 2023-06-08T23:50:47Z

.../main/scala/org/apache/spark/sql/execution/datasources/parquet/HoodieParquetFileFormat.scala

@@ -34,6 +34,15 @@ class HoodieParquetFileFormat extends ParquetFileFormat with SparkAdapterSupport

  override def toString: String = "Hoodie-Parquet"

+  override def supportBatch(sparkSession: SparkSession, schema: StructType): Boolean = {


FR: Spark 3.4 now supports vectorized reader on nested fields. However, Hudi does not support this yet due to custom schema evolution logic. So we add logic to override supportBatch in HoodieParquetFileFormat for Spark 3.4.

Can we add the JIRA for this as a comment here? https://issues.apache.org/jira/browse/HUDI-6262 for why we have this override, and that we should remove this once we support complex types in hudi?

JIRA to track: HUDI-6347

yihua · 2023-06-08T23:54:35Z

...urce/hudi-spark/src/test/scala/org/apache/spark/sql/hudi/command/index/TestIndexSyntax.scala

@@ -56,30 +58,37 @@ class TestIndexSyntax extends HoodieSparkSqlTestBase {

        var logicalPlan = sqlParser.parsePlan(s"show indexes from default.$tableName")
        var resolvedLogicalPlan = analyzer.execute(logicalPlan)
-        assertResult(s"`default`.`$tableName`")(resolvedLogicalPlan.asInstanceOf[ShowIndexesCommand].table.identifier.quotedString)


FR: table.identifier.quotedString now also has catalog name as the prefix.

yihua · 2023-06-09T00:10:41Z

hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieBaseRelation.scala

@@ -733,8 +734,8 @@ object HoodieBaseRelation extends SparkAdapterSupport {

    partitionedFile => {
      val hadoopConf = hadoopConfBroadcast.value.get()
-      val reader = new HoodieAvroHFileReader(hadoopConf, new Path(partitionedFile.filePath),
-        new CacheConfig(hadoopConf))
+      val filePath = sparkAdapter.getSparkPartitionedFileUtils.getPathFromPartitionedFile(partitionedFile)


For Reviewer (FR): all the changes in the common module of introducing new adapter support are because of Spark 3.4 class and API changes.

yihua · 2023-06-09T00:14:17Z

Hi @zhangyue19921010 @xiarixiaoyao @nsivabalan @xushiyan @danny0405, could you also review this PR?

…tFileFormat

…on and add checkColumnNameDuplication to fix AlterHoodieTableAddColumnsCommand

…w more compatibility issues

…ormat

yihua · 2023-06-09T05:19:39Z

@CTTY Thanks for the review. I addressed all your comments. @rahil-c @mansipp let me know if you have more comments.

rahil-c · 2023-06-09T17:55:22Z

hudi-client/hudi-spark-client/src/main/scala/org/apache/hudi/SparkJdbcUtils.scala

+    //      dialect: JdbcDialect,
+    //      alwaysNullable: Boolean = false,
+    //      isTimestampNTZ: Boolean = false): StructType
+    JdbcUtils.getSchema(resultSet, dialect, alwaysNullable)


For my own understanding, where does this get used?

This is used by UtilHelpers.getJDBCSchema for JDBC-based schema provider.

rahil-c · 2023-06-09T17:55:26Z

...-client/hudi-spark-client/src/main/scala/org/apache/spark/sql/HoodieCatalystPlansUtils.scala

+   * notMatchedActions: Seq[MergeAction],
+   * notMatchedBySourceActions: Seq[MergeAction]) extends BinaryCommand with SupportsSubquery
+   */
+  def unapplyMergeIntoTable(plan: LogicalPlan): Option[(LogicalPlan, LogicalPlan, Expression)]


For my own understanding, where does this get used?

This is ultimately used in HoodieAnalysis to match the plan.

rahil-c · 2023-06-09T18:10:40Z

...spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/BaseFileOnlyRelation.scala

  override def updatePrunedDataSchema(prunedSchema: StructType): Relation =
    this.copy(prunedDataSchema = Some(prunedSchema))

  override def imbueConfigs(sqlContext: SQLContext): Unit = {
    super.imbueConfigs(sqlContext)
    // TODO Issue with setting this to true in spark 332
-    if (!HoodieSparkUtils.gteqSpark3_3_2) {
+    if (HoodieSparkUtils.gteqSpark3_4 || !HoodieSparkUtils.gteqSpark3_3_2) {


@yihua Now when I think about it, I think the changes we made in 3.4(Specifically around supportBatch and supportsColumnar in ParquetFileFormat) should be brought to our 3.3.2 impl. This would then allow us to remove this if check all together, and allow vectorizedReader to be enabled regardless of Spark version.

I would say let's keep the logic for other spark versions as is and not balloon the scope of this PR.

I think if we sync I can try to help you get past this error for the other spark version.

JIRA to track: HUDI-6347

rahil-c · 2023-06-09T18:44:10Z

.../main/scala/org/apache/spark/sql/execution/datasources/parquet/HoodieParquetFileFormat.scala

@@ -34,6 +34,15 @@ class HoodieParquetFileFormat extends ParquetFileFormat with SparkAdapterSupport

  override def toString: String = "Hoodie-Parquet"

+  override def supportBatch(sparkSession: SparkSession, schema: StructType): Boolean = {
+    if (HoodieSparkUtils.gteqSpark3_4) {


Im wondering if this should be for greater than equal to spark 332, so that we can have this fix there as well.

Let's take this separately since this PR is not related to Spark 3.3.2. The code here is to make sure all other versions maintain the same logic.

The tests fail for other spark versions if I don't add this check.

Merge Hudi to Hudi *** FAILED *** 2023-06-06T23:38:24.7660935Z org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 3194.0 failed 1 times, most recent failure: Lost task 0.0 in stage 3194.0 (TID 3768) (fv-az1128-658 executor driver): java.lang.ClassCastException: org.apache.spark.sql.vectorized.ColumnarBatchRow cannot be cast to org.apache.spark.sql.vectorized.ColumnarBatch 2023-06-06T23:38:24.7662056Z at org.apache.spark.sql.execution.FileSourceScanExec$$anon$1.next(DataSourceScanExec.scala:560) 2023-06-06T23:38:24.7662628Z at org.apache.spark.sql.execution.FileSourceScanExec$$anon$1.next(DataSourceScanExec.scala:549) 2023-06-06T23:38:24.7663391Z at scala.collection.Iterator$$anon$11.nextCur(Iterator.scala:486)

...mon/src/main/scala/org/apache/spark/sql/hudi/command/AlterHoodieTableAddColumnsCommand.scala

rahil-c · 2023-06-09T19:12:30Z

...cala/org/apache/spark/sql/execution/datasources/parquet/Spark33HoodieParquetFileFormat.scala

+ * <li>Schema on-read</li>
+ * </ol>
+ */
+class Spark33HoodieParquetFileFormat(private val shouldAppendPartitionValues: Boolean) extends ParquetFileFormat {


Just to confirm, for this class we migrated logic as what spark32plus originally had? Are there any other changes we have added in this, and is this what spark 332 will use?

Im thinking if we should make a ParquertFileFormat for Spark 332 and add a similar supportBatch and supportColumnar so we can avoid cast issues instead of disabling vectorized reader for spark 332

This is copied from Spark32PlusHoodieParquetFileFormat. Let's take the Spark 3.3.2 improvement in a separate PR and not put everything in the same.

hudi-bot · 2023-06-10T02:22:08Z

CI report:

3419957 Azure: SUCCESS

Bot commands

@hudi-bot supports the following commands:

@hudi-bot run azure re-run the last Azure build

nsivabalan

LGTM. few minor comments. Good job folks on the 3.4.0 support 👏

nsivabalan · 2023-06-10T01:50:43Z

.../main/scala/org/apache/spark/sql/execution/datasources/HoodieSparkPartitionedFileUtils.scala

+   * @param partitionedFile Spark [[PartitionedFile]] instance.
+   * @return Hadoop [[Path]] instance.
+   */
+  def getPathFromPartitionedFile(partitionedFile: PartitionedFile): Path


may I know why do we need two methods, one to fetch string version and another to fetch Path. we can return string and the caller can construct the Path if need be?

SparkPath has toPath function to efficiently return the Path instance in Spark 3.4, so I opt to have two separate APIs here.

nsivabalan · 2023-06-10T02:10:07Z

...spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/BaseFileOnlyRelation.scala

  override def updatePrunedDataSchema(prunedSchema: StructType): Relation =
    this.copy(prunedDataSchema = Some(prunedSchema))

  override def imbueConfigs(sqlContext: SQLContext): Unit = {
    super.imbueConfigs(sqlContext)
    // TODO Issue with setting this to true in spark 332
-    if (!HoodieSparkUtils.gteqSpark3_3_2) {
+    if (HoodieSparkUtils.gteqSpark3_4 || !HoodieSparkUtils.gteqSpark3_3_2) {


should we introduce "HoodieSparkUtils.ltSpark3_3_2" instead of "!HoodieSparkUtils.gteqSpark3_3_2"

We can do that. I keep the existing code the same in this PR.

nsivabalan · 2023-06-10T02:14:07Z

hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieBaseRelation.scala

@@ -733,8 +734,8 @@ object HoodieBaseRelation extends SparkAdapterSupport {

    partitionedFile => {
      val hadoopConf = hadoopConfBroadcast.value.get()
-      val reader = new HoodieAvroHFileReader(hadoopConf, new Path(partitionedFile.filePath),
-        new CacheConfig(hadoopConf))
+      val filePath = sparkAdapter.getSparkPartitionedFileUtils.getPathFromPartitionedFile(partitionedFile)


yihua · 2023-06-10T05:53:49Z

Thank you folks for the review. I'll merge this PR now. Feel free to add more comments if there is more, given we're having ongoing discussion on the Hudi Spark integration.

rahil-c reviewed Jun 6, 2023

View reviewed changes

pom.xml Outdated Show resolved Hide resolved

rahil-c reviewed Jun 6, 2023

View reviewed changes

hudi-common/pom.xml Outdated Show resolved Hide resolved

yihua added dependencies Pull requests that update a dependency file priority:blocker spark Issues related to spark release-0.14.0 labels Jun 7, 2023

apache deleted a comment from hudi-bot Jun 7, 2023

yihua changed the title ~~[DNM][HUDI-6198] Support Hudi on Spark 3.4.0~~ [HUDI-6198] Support Hudi on Spark 3.4.0 Jun 8, 2023

CTTY reviewed Jun 8, 2023

View reviewed changes

hudi-common/src/main/java/org/apache/hudi/common/util/JsonUtils.java Outdated Show resolved Hide resolved

CTTY reviewed Jun 8, 2023

View reviewed changes

...rce/hudi-spark3.2.x/src/main/scala/org/apache/spark/sql/HoodieSpark32CatalystPlanUtils.scala Show resolved Hide resolved

CTTY reviewed Jun 8, 2023

View reviewed changes

...cala/org/apache/spark/sql/execution/datasources/parquet/Spark34HoodieParquetFileFormat.scala Outdated Show resolved Hide resolved

apache deleted a comment from hudi-bot Jun 8, 2023

yihua commented Jun 9, 2023

View reviewed changes

mansipp and others added 14 commits June 8, 2023 20:09

Spark 3.4.0 upgrade

f7a3b7d

Fix GH Java CI on Spark 3.4

1a8530e

Fix HoodieAnalysis and resolve MergeIntoTable signature change

52dbfdb

Fix configs in HoodieSparkFileReaderFactory

d75aac2

Revert Spark32PlusHoodieParquetFileFormat and fix Spark34HoodieParque…

9d1bf88

…tFileFormat

Fix BaseFileOnlyRelation

9f921d2

Add SparkPartitionedFileUtils to adapt PartitionedFile API changes

6f6f0d7

Add more APIs to SparkPartitionedFileUtils, fix HoodieBootstrapRelati…

a35227b

…on and add checkColumnNameDuplication to fix AlterHoodieTableAddColumnsCommand

Add build profile for Spark 3.4, fix parquet format classes, and a fe…

2f35ebe

…w more compatibility issues

Fix references of HoodieParquetFileFormat

37db2c9

Fix Spark34HoodieParquetFileFormat

663c8eb

Implement adapter APIs for Spark 3.4

edc2794

Fix build failure due to latest changes on top of 0.13.0 and 0.13.1

c2a1144

Fix HoodieSpark34CatalystExpressionUtils

9f39a94

yihua added 4 commits June 8, 2023 20:12

Add docs, fix nits and remove unnecessary changes

9660a7a

Add docs, fix nits and remove unnecessary changes - second batch

99e8877

Switch spark3 profile to Spark 3.4

70caf33

Fix bot.yml around Java 17

6bcf646

yihua force-pushed the HUDI-6198-spark34 branch from e2f44f2 to 6bcf646 Compare June 9, 2023 03:13

Use SQLConf.LEGACY_PARQUET_NANOS_AS_LONG in Spark34HoodieParquetFileF…

8fda233

…ormat

yihua assigned nsivabalan Jun 9, 2023

yihua requested review from nsivabalan, xushiyan, danny0405, xiarixiaoyao, rahil-c and CTTY June 9, 2023 05:20

rahil-c reviewed Jun 9, 2023

View reviewed changes

This was referenced Jun 9, 2023

[DO NOT MERGE][HUDI-6198] Testing Spark 3.4.0 Upgrade on 0.13.1 #8826

Closed

[DO NOT MERGE][HUDI-6198][0.13.0] Testing Spark 3.4.0 Upgrade #8813

Closed

[DO NOT MERGE] [HUDI-6198] Run gh actions with Spark 3.4.0 #8682

Closed

rahil-c reviewed Jun 9, 2023

View reviewed changes

...mon/src/main/scala/org/apache/spark/sql/hudi/command/AlterHoodieTableAddColumnsCommand.scala Outdated Show resolved Hide resolved

rahil-c reviewed Jun 9, 2023

View reviewed changes

...mon/src/main/scala/org/apache/spark/sql/hudi/command/AlterHoodieTableAddColumnsCommand.scala Show resolved Hide resolved

rahil-c reviewed Jun 9, 2023

View reviewed changes

Fix indentation

3419957

nsivabalan approved these changes Jun 10, 2023

View reviewed changes

rahil-c approved these changes Jun 10, 2023

View reviewed changes

yihua merged commit 96ca7aa into apache:master Jun 10, 2023
22 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[HUDI-6198] Support Hudi on Spark 3.4.0 #8885

[HUDI-6198] Support Hudi on Spark 3.4.0 #8885

yihua commented Jun 5, 2023 •

edited

Loading

melin commented Jun 6, 2023

yihua commented Jun 6, 2023

yihua Jun 8, 2023

rahil-c Jun 9, 2023 •

edited

Loading

yihua Jun 10, 2023

yihua Jun 8, 2023

yihua Jun 9, 2023

nsivabalan Jun 10, 2023

yihua commented Jun 9, 2023

yihua commented Jun 9, 2023

rahil-c Jun 9, 2023

yihua Jun 9, 2023

rahil-c Jun 9, 2023

yihua Jun 9, 2023

rahil-c Jun 9, 2023 •

edited

Loading

yihua Jun 9, 2023

rahil-c Jun 9, 2023

yihua Jun 9, 2023

rahil-c Jun 9, 2023

yihua Jun 9, 2023

yihua Jun 9, 2023

rahil-c Jun 9, 2023 •

edited

Loading

yihua Jun 9, 2023

hudi-bot commented Jun 10, 2023

nsivabalan left a comment

nsivabalan Jun 10, 2023

yihua Jun 10, 2023

nsivabalan Jun 10, 2023

yihua Jun 10, 2023

nsivabalan Jun 10, 2023

yihua commented Jun 10, 2023

		@@ -34,6 +34,15 @@ class HoodieParquetFileFormat extends ParquetFileFormat with SparkAdapterSupport

		override def toString: String = "Hoodie-Parquet"

		override def supportBatch(sparkSession: SparkSession, schema: StructType): Boolean = {

[HUDI-6198] Support Hudi on Spark 3.4.0 #8885

[HUDI-6198] Support Hudi on Spark 3.4.0 #8885

Conversation

yihua commented Jun 5, 2023 • edited Loading

Change Logs

More details

Impact

Risk level

Documentation Update

Contributor's checklist

melin commented Jun 6, 2023

yihua commented Jun 6, 2023

Choose a reason for hiding this comment

rahil-c Jun 9, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

yihua commented Jun 9, 2023

yihua commented Jun 9, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rahil-c Jun 9, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rahil-c Jun 9, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

hudi-bot commented Jun 10, 2023

CI report:

nsivabalan left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

yihua commented Jun 10, 2023

yihua commented Jun 5, 2023 •

edited

Loading

rahil-c Jun 9, 2023 •

edited

Loading

rahil-c Jun 9, 2023 •

edited

Loading

rahil-c Jun 9, 2023 •

edited

Loading