Commits on Feb 16, 2024

1. [SPARK-45357][CONNECT][TESTS] Normalize dataframeId when comparing … …

   …`CollectMetrics` in `SparkConnectProtoSuite`
   
   ### What changes were proposed in this pull request?
   This PR add a new function `normalizeDataframeId` to sets the `dataframeId` to the constant 0 of `CollectMetrics`  before comparing `LogicalPlan` in the test case of `SparkConnectProtoSuite`.
   
   ### Why are the changes needed?
   The test scenario in `SparkConnectProtoSuite` does not need to compare the `dataframeId` in `CollectMetrics`
   
   ### Does this PR introduce _any_ user-facing change?
   No
   
   ### How was this patch tested?
   - Manually check
   
   run
   
   ```
   build/mvn clean install -pl connector/connect/server -am -DskipTests
   build/mvn test -pl connector/connect/server
   ```
   
   **Before**
   
   ```
   - Test observe *** FAILED ***
     == FAIL: Plans do not match ===
     !CollectMetrics my_metric, [min(id#0) AS min_val#0, max(id#0) AS max_val#0, sum(id#0) AS sum(id)#0L], 0   CollectMetrics my_metric, [min(id#0) AS min_val#0, max(id#0) AS max_val#0, sum(id#0) AS sum(id)#0L], 53
      +- LocalRelation <empty>, [id#0, name#0]                                                                 +- LocalRelation <empty>, [id#0, name#0] (PlanTest.scala:179)
   ```
   
   **After**
   
   ```
   Run completed in 41 seconds, 631 milliseconds.
   Total number of tests run: 882
   Suites: completed 24, aborted 0
   Tests: succeeded 882, failed 0, canceled 0, ignored 0, pending 0
   All tests passed.
   ```
   
   ### Was this patch authored or co-authored using generative AI tooling?
   No
   
   Closes apache#43155 from LuciferYang/SPARK-45357.
   
   Authored-by: yangjie01 <yangjie01@baidu.com>
   Signed-off-by: Ruifeng Zheng <ruifengz@apache.org>

   

   LuciferYang committed Feb 16, 2024
   Configuration menu

   • View commit details
   • Copy full SHA for 91f7e85
   • Browse repository at this point

   Copy the full SHA

   91f7e85 View commit details

   Browse the repository at this point in the history