[SPARK-37287][SQL][FOLLOWUP] Only apply AQE on top of V1 write commands when planned write is enabled

[allisonwang-db]

What changes were proposed in this pull request?

This PR is a follow-up for SPARK-37287. We only want to apply AQE on top of V1 write commands when the planned write config is enabled.

Why are the changes needed?

To make the behavior consistent with before when spark.sql.optimizer.plannedWrite.enabled is disabled.

Does this PR introduce any user-facing change?

No

How was this patch tested?

New unit test.

[HyukjinKwon]

Failed test:

2022-07-26T23:16:22.6426634Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[32mV1WriteCommandSuite:�[0m�[0m
2022-07-26T23:16:23.2876266Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[32m- v1 write without partition columns (247 milliseconds)�[0m�[0m
2022-07-26T23:16:23.7915525Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[32m- v1 write with non-string partition columns (503 milliseconds)�[0m�[0m
2022-07-26T23:16:24.4310173Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[32m- v1 write with string partition columns (639 milliseconds)�[0m�[0m
2022-07-26T23:16:25.0168514Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[32m- v1 write with partition, bucketed and sort columns (584 milliseconds)�[0m�[0m
2022-07-26T23:16:25.4606518Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m- v1 write with already sorted plan - non-string partition column *** FAILED *** (421 milliseconds)�[0m�[0m
2022-07-26T23:16:25.4607721Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  false did not equal true Expect: true, Actual: false (V1WriteCommandSuite.scala:74)�[0m�[0m
2022-07-26T23:16:25.4608684Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  org.scalatest.exceptions.TestFailedException:�[0m�[0m
2022-07-26T23:16:25.4609457Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at org.scalatest.Assertions.newAssertionFailedException(Assertions.scala:472)�[0m�[0m
2022-07-26T23:16:25.4610564Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at org.scalatest.Assertions.newAssertionFailedException$(Assertions.scala:471)�[0m�[0m
2022-07-26T23:16:25.4611349Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at org.scalatest.Assertions$.newAssertionFailedException(Assertions.scala:1231)�[0m�[0m
2022-07-26T23:16:25.4612294Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at org.scalatest.Assertions$AssertionsHelper.macroAssert(Assertions.scala:1295)�[0m�[0m
2022-07-26T23:16:25.4613240Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at org.apache.spark.sql.execution.datasources.V1WriteCommandSuiteBase.executeAndCheckOrdering(V1WriteCommandSuite.scala:74)�[0m�[0m
2022-07-26T23:16:25.4614493Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at org.apache.spark.sql.execution.datasources.V1WriteCommandSuite.$anonfun$new$19(V1WriteCommandSuite.scala:148)�[0m�[0m
2022-07-26T23:16:25.4615303Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)�[0m�[0m
2022-07-26T23:16:25.4616213Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1490)�[0m�[0m
2022-07-26T23:16:25.4616964Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at org.apache.spark.sql.test.SQLTestUtilsBase.withTable(SQLTestUtils.scala:306)�[0m�[0m
2022-07-26T23:16:25.4618792Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at org.apache.spark.sql.test.SQLTestUtilsBase.withTable$(SQLTestUtils.scala:304)�[0m�[0m
2022-07-26T23:16:25.4619697Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at org.apache.spark.sql.execution.datasources.V1WriteCommandSuiteBase.withTable(V1WriteCommandSuite.scala:27)�[0m�[0m
2022-07-26T23:16:25.4620856Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at org.apache.spark.sql.execution.datasources.V1WriteCommandSuite.$anonfun$new$18(V1WriteCommandSuite.scala:141)�[0m�[0m
2022-07-26T23:16:25.4621804Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at org.apache.spark.sql.execution.datasources.V1WriteCommandSuite.$anonfun$new$18$adapted(V1WriteCommandSuite.scala:140)�[0m�[0m
2022-07-26T23:16:25.4623001Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at org.apache.spark.sql.execution.datasources.V1WriteCommandSuiteBase.$anonfun$withPlannedWrite$2(V1WriteCommandSuite.scala:49)�[0m�[0m
2022-07-26T23:16:25.4623888Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at org.apache.spark.sql.catalyst.plans.SQLHelper.withSQLConf(SQLHelper.scala:54)�[0m�[0m
2022-07-26T23:16:25.4624897Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at org.apache.spark.sql.catalyst.plans.SQLHelper.withSQLConf$(SQLHelper.scala:38)�[0m�[0m
2022-07-26T23:16:25.4625891Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at org.apache.spark.sql.execution.datasources.V1WriteCommandSuiteBase.org$apache$spark$sql$test$SQLTestUtilsBase$$super$withSQLConf(V1WriteCommandSuite.scala:27)�[0m�[0m
2022-07-26T23:16:25.4627211Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at org.apache.spark.sql.test.SQLTestUtilsBase.withSQLConf(SQLTestUtils.scala:247)�[0m�[0m
2022-07-26T23:16:25.4628007Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at org.apache.spark.sql.test.SQLTestUtilsBase.withSQLConf$(SQLTestUtils.scala:245)�[0m�[0m
2022-07-26T23:16:25.4629124Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at org.apache.spark.sql.execution.datasources.V1WriteCommandSuiteBase.withSQLConf(V1WriteCommandSuite.scala:27)�[0m�[0m
2022-07-26T23:16:25.4630132Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at org.apache.spark.sql.execution.datasources.V1WriteCommandSuiteBase.$anonfun$withPlannedWrite$1(V1WriteCommandSuite.scala:49)�[0m�[0m
2022-07-26T23:16:25.4631354Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at org.apache.spark.sql.execution.datasources.V1WriteCommandSuiteBase.$anonfun$withPlannedWrite$1$adapted(V1WriteCommandSuite.scala:47)�[0m�[0m
2022-07-26T23:16:25.4632279Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at scala.collection.immutable.List.foreach(List.scala:431)�[0m�[0m
2022-07-26T23:16:25.4633672Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at org.apache.spark.sql.execution.datasources.V1WriteCommandSuiteBase.withPlannedWrite(V1WriteCommandSuite.scala:47)�[0m�[0m
2022-07-26T23:16:25.4634736Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at org.apache.spark.sql.execution.datasources.V1WriteCommandSuite.$anonfun$new$17(V1WriteCommandSuite.scala:140)�[0m�[0m
2022-07-26T23:16:25.4636027Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)�[0m�[0m
2022-07-26T23:16:25.4636714Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at org.scalatest.OutcomeOf.outcomeOf(OutcomeOf.scala:85)�[0m�[0m
2022-07-26T23:16:25.4637589Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at org.scalatest.OutcomeOf.outcomeOf$(OutcomeOf.scala:83)�[0m�[0m
2022-07-26T23:16:25.4638247Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)�[0m�[0m
2022-07-26T23:16:25.4639116Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at org.scalatest.Transformer.apply(Transformer.scala:22)�[0m�[0m
2022-07-26T23:16:25.4639802Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at org.scalatest.Transformer.apply(Transformer.scala:20)�[0m�[0m
2022-07-26T23:16:25.4641126Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at org.scalatest.funsuite.AnyFunSuiteLike$$anon$1.apply(AnyFunSuiteLike.scala:190)�[0m�[0m
2022-07-26T23:16:25.4641725Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at org.apache.spark.SparkFunSuite.withFixture(SparkFunSuite.scala:204)�[0m�[0m
2022-07-26T23:16:25.4642360Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at org.scalatest.funsuite.AnyFunSuiteLike.invokeWithFixture$1(AnyFunSuiteLike.scala:188)�[0m�[0m
2022-07-26T23:16:25.4642997Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at org.scalatest.funsuite.AnyFunSuiteLike.$anonfun$runTest$1(AnyFunSuiteLike.scala:200)�[0m�[0m
2022-07-26T23:16:25.4643557Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at org.scalatest.SuperEngine.runTestImpl(Engine.scala:306)�[0m�[0m
2022-07-26T23:16:25.4644129Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at org.scalatest.funsuite.AnyFunSuiteLike.runTest(AnyFunSuiteLike.scala:200)�[0m�[0m
2022-07-26T23:16:25.4644727Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at org.scalatest.funsuite.AnyFunSuiteLike.runTest$(AnyFunSuiteLike.scala:182)�[0m�[0m
2022-07-26T23:16:25.4645377Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at org.apache.spark.SparkFunSuite.org$scalatest$BeforeAndAfterEach$$super$runTest(SparkFunSuite.scala:65)�[0m�[0m
2022-07-26T23:16:25.4645992Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at org.scalatest.BeforeAndAfterEach.runTest(BeforeAndAfterEach.scala:234)�[0m�[0m
2022-07-26T23:16:25.4646578Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at org.scalatest.BeforeAndAfterEach.runTest$(BeforeAndAfterEach.scala:227)�[0m�[0m
2022-07-26T23:16:25.4647135Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at org.apache.spark.SparkFunSuite.runTest(SparkFunSuite.scala:65)�[0m�[0m
2022-07-26T23:16:25.4647724Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at org.scalatest.funsuite.AnyFunSuiteLike.$anonfun$runTests$1(AnyFunSuiteLike.scala:233)�[0m�[0m
2022-07-26T23:16:25.4648293Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at org.scalatest.SuperEngine.$anonfun$runTestsInBranch$1(Engine.scala:413)�[0m�[0m
2022-07-26T23:16:25.4648820Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at scala.collection.immutable.List.foreach(List.scala:431)�[0m�[0m
2022-07-26T23:16:25.4649369Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at org.scalatest.SuperEngine.traverseSubNodes$1(Engine.scala:401)�[0m�[0m
2022-07-26T23:16:25.4649915Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at org.scalatest.SuperEngine.runTestsInBranch(Engine.scala:396)�[0m�[0m
2022-07-26T23:16:25.4650448Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at org.scalatest.SuperEngine.runTestsImpl(Engine.scala:475)�[0m�[0m
2022-07-26T23:16:25.4651024Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at org.scalatest.funsuite.AnyFunSuiteLike.runTests(AnyFunSuiteLike.scala:233)�[0m�[0m
2022-07-26T23:16:25.4651802Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at org.scalatest.funsuite.AnyFunSuiteLike.runTests$(AnyFunSuiteLike.scala:232)�[0m�[0m
2022-07-26T23:16:25.4652496Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at org.scalatest.funsuite.AnyFunSuite.runTests(AnyFunSuite.scala:1563)�[0m�[0m
2022-07-26T23:16:25.4652991Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at org.scalatest.Suite.run(Suite.scala:1112)�[0m�[0m
2022-07-26T23:16:25.4653486Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at org.scalatest.Suite.run$(Suite.scala:1094)�[0m�[0m
2022-07-26T23:16:25.4654075Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at org.scalatest.funsuite.AnyFunSuite.org$scalatest$funsuite$AnyFunSuiteLike$$super$run(AnyFunSuite.scala:1563)�[0m�[0m
2022-07-26T23:16:25.4654699Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at org.scalatest.funsuite.AnyFunSuiteLike.$anonfun$run$1(AnyFunSuiteLike.scala:237)�[0m�[0m
2022-07-26T23:16:25.4655233Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at org.scalatest.SuperEngine.runImpl(Engine.scala:535)�[0m�[0m
2022-07-26T23:16:25.4655772Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at org.scalatest.funsuite.AnyFunSuiteLike.run(AnyFunSuiteLike.scala:237)�[0m�[0m
2022-07-26T23:16:25.4656351Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at org.scalatest.funsuite.AnyFunSuiteLike.run$(AnyFunSuiteLike.scala:236)�[0m�[0m
2022-07-26T23:16:25.4656977Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at org.apache.spark.SparkFunSuite.org$scalatest$BeforeAndAfterAll$$super$run(SparkFunSuite.scala:65)�[0m�[0m
2022-07-26T23:16:25.4657592Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at org.scalatest.BeforeAndAfterAll.liftedTree1$1(BeforeAndAfterAll.scala:213)�[0m�[0m
2022-07-26T23:16:25.4658166Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at org.scalatest.BeforeAndAfterAll.run(BeforeAndAfterAll.scala:210)�[0m�[0m
2022-07-26T23:16:25.4658712Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at org.scalatest.BeforeAndAfterAll.run$(BeforeAndAfterAll.scala:208)�[0m�[0m
2022-07-26T23:16:25.4662209Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at org.apache.spark.SparkFunSuite.run(SparkFunSuite.scala:65)�[0m�[0m
2022-07-26T23:16:25.4662844Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at org.scalatest.tools.Framework.org$scalatest$tools$Framework$$runSuite(Framework.scala:318)�[0m�[0m
2022-07-26T23:16:25.4663435Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at org.scalatest.tools.Framework$ScalaTestTask.execute(Framework.scala:513)�[0m�[0m
2022-07-26T23:16:25.4663941Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at sbt.ForkMain$Run.lambda$runTest$1(ForkMain.java:413)�[0m�[0m
2022-07-26T23:16:25.4664437Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at java.util.concurrent.FutureTask.run(FutureTask.java:266)�[0m�[0m
2022-07-26T23:16:25.4665027Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)�[0m�[0m
2022-07-26T23:16:25.4665654Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)�[0m�[0m
2022-07-26T23:16:25.4666157Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at java.lang.Thread.run(Thread.java:750)�[0m�[0m
2022-07-26T23:16:26.0997843Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m- v1 write with already sorted plan - string partition column *** FAILED *** (641 milliseconds)�[0m�[0m
2022-07-26T23:16:26.1013963Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  false did not equal true Expect: true, Actual: false (V1WriteCommandSuite.scala:74)�[0m�[0m
2022-07-26T23:16:26.1021629Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  org.scalatest.exceptions.TestFailedException:�[0m�[0m
2022-07-26T23:16:26.1022502Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at org.scalatest.Assertions.newAssertionFailedException(Assertions.scala:472)�[0m�[0m
2022-07-26T23:16:26.1024077Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at org.scalatest.Assertions.newAssertionFailedException$(Assertions.scala:471)�[0m�[0m
2022-07-26T23:16:26.1024846Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at org.scalatest.Assertions$.newAssertionFailedException(Assertions.scala:1231)�[0m�[0m
2022-07-26T23:16:26.1025990Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at org.scalatest.Assertions$AssertionsHelper.macroAssert(Assertions.scala:1295)�[0m�[0m
2022-07-26T23:16:26.1027100Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at org.apache.spark.sql.execution.datasources.V1WriteCommandSuiteBase.executeAndCheckOrdering(V1WriteCommandSuite.scala:74)�[0m�[0m
2022-07-26T23:16:26.1028534Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at org.apache.spark.sql.execution.datasources.V1WriteCommandSuite.$anonfun$new$23(V1WriteCommandSuite.scala:163)�[0m�[0m
2022-07-26T23:16:26.1030600Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)�[0m�[0m
2022-07-26T23:16:26.1032119Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1490)�[0m�[0m
2022-07-26T23:16:26.1033009Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at org.apache.spark.sql.test.SQLTestUtilsBase.withTable(SQLTestUtils.scala:306)�[0m�[0m
2022-07-26T23:16:26.1034213Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at org.apache.spark.sql.test.SQLTestUtilsBase.withTable$(SQLTestUtils.scala:304)�[0m�[0m
2022-07-26T23:16:26.1035156Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at org.apache.spark.sql.execution.datasources.V1WriteCommandSuiteBase.withTable(V1WriteCommandSuite.scala:27)�[0m�[0m
2022-07-26T23:16:26.1036546Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at org.apache.spark.sql.execution.datasources.V1WriteCommandSuite.$anonfun$new$22(V1WriteCommandSuite.scala:156)�[0m�[0m
2022-07-26T23:16:26.1037620Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at org.apache.spark.sql.execution.datasources.V1WriteCommandSuite.$anonfun$new$22$adapted(V1WriteCommandSuite.scala:155)�[0m�[0m
2022-07-26T23:16:26.1039738Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at org.apache.spark.sql.execution.datasources.V1WriteCommandSuiteBase.$anonfun$withPlannedWrite$2(V1WriteCommandSuite.scala:49)�[0m�[0m
2022-07-26T23:16:26.1040623Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at org.apache.spark.sql.catalyst.plans.SQLHelper.withSQLConf(SQLHelper.scala:54)�[0m�[0m
2022-07-26T23:16:26.1042841Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at org.apache.spark.sql.catalyst.plans.SQLHelper.withSQLConf$(SQLHelper.scala:38)�[0m�[0m
2022-07-26T23:16:26.1043855Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at org.apache.spark.sql.execution.datasources.V1WriteCommandSuiteBase.org$apache$spark$sql$test$SQLTestUtilsBase$$super$withSQLConf(V1WriteCommandSuite.scala:27)�[0m�[0m
2022-07-26T23:16:26.1045198Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at org.apache.spark.sql.test.SQLTestUtilsBase.withSQLConf(SQLTestUtils.scala:247)�[0m�[0m
2022-07-26T23:16:26.1046030Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at org.apache.spark.sql.test.SQLTestUtilsBase.withSQLConf$(SQLTestUtils.scala:245)�[0m�[0m
2022-07-26T23:16:26.1047440Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at org.apache.spark.sql.execution.datasources.V1WriteCommandSuiteBase.withSQLConf(V1WriteCommandSuite.scala:27)�[0m�[0m
2022-07-26T23:16:26.1049397Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at org.apache.spark.sql.execution.datasources.V1WriteCommandSuiteBase.$anonfun$withPlannedWrite$1(V1WriteCommandSuite.scala:49)�[0m�[0m
2022-07-26T23:16:26.1050349Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at org.apache.spark.sql.execution.datasources.V1WriteCommandSuiteBase.$anonfun$withPlannedWrite$1$adapted(V1WriteCommandSuite.scala:47)�[0m�[0m
2022-07-26T23:16:26.1051022Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at scala.collection.immutable.List.foreach(List.scala:431)�[0m�[0m
2022-07-26T23:16:26.1051752Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at org.apache.spark.sql.execution.datasources.V1WriteCommandSuiteBase.withPlannedWrite(V1WriteCommandSuite.scala:47)�[0m�[0m
2022-07-26T23:16:26.1052572Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at org.apache.spark.sql.execution.datasources.V1WriteCommandSuite.$anonfun$new$21(V1WriteCommandSuite.scala:155)�[0m�[0m
2022-07-26T23:16:26.1053205Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)�[0m�[0m
2022-07-26T23:16:26.1054594Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at org.scalatest.OutcomeOf.outcomeOf(OutcomeOf.scala:85)�[0m�[0m
2022-07-26T23:16:26.1055115Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at org.scalatest.OutcomeOf.outcomeOf$(OutcomeOf.scala:83)�[0m�[0m
2022-07-26T23:16:26.1055750Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)�[0m�[0m
2022-07-26T23:16:26.1059872Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at org.scalatest.Transformer.apply(Transformer.scala:22)�[0m�[0m
2022-07-26T23:16:26.1062053Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at org.scalatest.Transformer.apply(Transformer.scala:20)�[0m�[0m
2022-07-26T23:16:26.1064017Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at org.scalatest.funsuite.AnyFunSuiteLike$$anon$1.apply(AnyFunSuiteLike.scala:190)�[0m�[0m
2022-07-26T23:16:26.1065537Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at org.apache.spark.SparkFunSuite.withFixture(SparkFunSuite.scala:204)�[0m�[0m
2022-07-26T23:16:26.1116254Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at org.scalatest.funsuite.AnyFunSuiteLike.invokeWithFixture$1(AnyFunSuiteLike.scala:188)�[0m�[0m
2022-07-26T23:16:26.1139960Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at org.scalatest.funsuite.AnyFunSuiteLike.$anonfun$runTest$1(AnyFunSuiteLike.scala:200)�[0m�[0m
2022-07-26T23:16:26.1140688Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at org.scalatest.SuperEngine.runTestImpl(Engine.scala:306)�[0m�[0m
2022-07-26T23:16:26.1141361Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at org.scalatest.funsuite.AnyFunSuiteLike.runTest(AnyFunSuiteLike.scala:200)�[0m�[0m
2022-07-26T23:16:26.1142038Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at org.scalatest.funsuite.AnyFunSuiteLike.runTest$(AnyFunSuiteLike.scala:182)�[0m�[0m
2022-07-26T23:16:26.1142768Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at org.apache.spark.SparkFunSuite.org$scalatest$BeforeAndAfterEach$$super$runTest(SparkFunSuite.scala:65)�[0m�[0m
2022-07-26T23:16:26.1143497Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at org.scalatest.BeforeAndAfterEach.runTest(BeforeAndAfterEach.scala:234)�[0m�[0m
2022-07-26T23:16:26.1144163Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at org.scalatest.BeforeAndAfterEach.runTest$(BeforeAndAfterEach.scala:227)�[0m�[0m
2022-07-26T23:16:26.1144807Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at org.apache.spark.SparkFunSuite.runTest(SparkFunSuite.scala:65)�[0m�[0m
2022-07-26T23:16:26.1145478Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at org.scalatest.funsuite.AnyFunSuiteLike.$anonfun$runTests$1(AnyFunSuiteLike.scala:233)�[0m�[0m
2022-07-26T23:16:26.1146128Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at org.scalatest.SuperEngine.$anonfun$runTestsInBranch$1(Engine.scala:413)�[0m�[0m
2022-07-26T23:16:26.1146726Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at scala.collection.immutable.List.foreach(List.scala:431)�[0m�[0m
2022-07-26T23:16:26.1147335Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at org.scalatest.SuperEngine.traverseSubNodes$1(Engine.scala:401)�[0m�[0m
2022-07-26T23:16:26.1147953Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at org.scalatest.SuperEngine.runTestsInBranch(Engine.scala:396)�[0m�[0m
2022-07-26T23:16:26.1148563Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at org.scalatest.SuperEngine.runTestsImpl(Engine.scala:475)�[0m�[0m
2022-07-26T23:16:26.1149212Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at org.scalatest.funsuite.AnyFunSuiteLike.runTests(AnyFunSuiteLike.scala:233)�[0m�[0m
2022-07-26T23:16:26.1149891Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at org.scalatest.funsuite.AnyFunSuiteLike.runTests$(AnyFunSuiteLike.scala:232)�[0m�[0m
2022-07-26T23:16:26.1150544Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at org.scalatest.funsuite.AnyFunSuite.runTests(AnyFunSuite.scala:1563)�[0m�[0m
2022-07-26T23:16:26.1151118Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at org.scalatest.Suite.run(Suite.scala:1112)�[0m�[0m
2022-07-26T23:16:26.1151637Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at org.scalatest.Suite.run$(Suite.scala:1094)�[0m�[0m
2022-07-26T23:16:26.1152302Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at org.scalatest.funsuite.AnyFunSuite.org$scalatest$funsuite$AnyFunSuiteLike$$super$run(AnyFunSuite.scala:1563)�[0m�[0m
2022-07-26T23:16:26.1153119Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at org.scalatest.funsuite.AnyFunSuiteLike.$anonfun$run$1(AnyFunSuiteLike.scala:237)�[0m�[0m
2022-07-26T23:16:26.1153738Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at org.scalatest.SuperEngine.runImpl(Engine.scala:535)�[0m�[0m
2022-07-26T23:16:26.1154530Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at org.scalatest.funsuite.AnyFunSuiteLike.run(AnyFunSuiteLike.scala:237)�[0m�[0m
2022-07-26T23:16:26.1155176Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at org.scalatest.funsuite.AnyFunSuiteLike.run$(AnyFunSuiteLike.scala:236)�[0m�[0m
2022-07-26T23:16:26.1155946Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at org.apache.spark.SparkFunSuite.org$scalatest$BeforeAndAfterAll$$super$run(SparkFunSuite.scala:65)�[0m�[0m
2022-07-26T23:16:26.1156652Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at org.scalatest.BeforeAndAfterAll.liftedTree1$1(BeforeAndAfterAll.scala:213)�[0m�[0m
2022-07-26T23:16:26.1157294Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at org.scalatest.BeforeAndAfterAll.run(BeforeAndAfterAll.scala:210)�[0m�[0m
2022-07-26T23:16:26.1157918Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at org.scalatest.BeforeAndAfterAll.run$(BeforeAndAfterAll.scala:208)�[0m�[0m
2022-07-26T23:16:26.1158536Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at org.apache.spark.SparkFunSuite.run(SparkFunSuite.scala:65)�[0m�[0m
2022-07-26T23:16:26.1159195Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at org.scalatest.tools.Framework.org$scalatest$tools$Framework$$runSuite(Framework.scala:318)�[0m�[0m
2022-07-26T23:16:26.1159846Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at org.scalatest.tools.Framework$ScalaTestTask.execute(Framework.scala:513)�[0m�[0m
2022-07-26T23:16:26.1160432Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at sbt.ForkMain$Run.lambda$runTest$1(ForkMain.java:413)�[0m�[0m
2022-07-26T23:16:26.1160997Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at java.util.concurrent.FutureTask.run(FutureTask.java:266)�[0m�[0m
2022-07-26T23:16:26.1161652Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)�[0m�[0m
2022-07-26T23:16:26.1162353Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)�[0m�[0m
2022-07-26T23:16:26.1162918Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[31m  at java.lang.Thread.run(Thread.java:750)�[0m�[0m
2022-07-26T23:16:26.9690187Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[32m- v1 write with null and empty string column values (850 milliseconds)�[0m�[0m
2022-07-26T23:16:27.3509661Z �[0m[�[0m�[0minfo�[0m] �[0m�[0m�[32m- v1 write with AQE changing SMJ to BHJ (388 milliseconds)�[0m�[0m

[allisonwang-db]

cc @cloud-fan

[cloud-fan]

thanks, merging to master!