[VL] fix wrong result for delta deletion vector

[zhli1142015]

What changes were proposed in this pull request?

DV is not supported, before scan with DV would fallback. After #4538, scan with _delta_internal_is_row_deleted:tinyint is offloaded but generats wrong results. This change fallback this case.

- deletion vector *** FAILED ***
  Results do not match for query:
  Timezone: sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]]
  Timezone Env:

  == Parsed Logical Plan ==
  Relation [id#16425] parquet

  == Analyzed Logical Plan ==
  id: int
  Relation [id#16425] parquet

  == Optimized Logical Plan ==
  Project [id#16425]
  +- Filter UDF(__delta_internal_is_row_deleted#16585)
     +- Relation [id#16425,__delta_internal_is_row_deleted#16585] parquet

  == Physical Plan ==
  VeloxColumnarToRowExec
  +- ^(198) ProjectExecTransformer [id#16425]
     +- ^(198) InputIteratorTransformer[id#16425, __delta_internal_is_row_deleted#16585]
        +- RowToVeloxColumnar
           +- *(1) Filter UDF(__delta_internal_is_row_deleted#16585)
              +- VeloxColumnarToRowExec
                 +- ^(197) NativeFileScan parquet [id#16425,__delta_internal_is_row_deleted#16585] Batched: true, DataFilters: [], Format: Parquet, Location: PreparedDeltaFileIndex(1 paths)[file:/tmp/spark-410fb7f8-a994-407e-acee-03c0fc50b571], PartitionFilters: [], PushedFilters: [], ReadSchema: struct<id:int,__delta_internal_is_row_deleted:tinyint>

  == Results ==

  == Results ==
  !== Correct Answer - 5 ==   == Gluten Answer - 10 ==
   struct<>                   struct<>
  ![1]                        [10]
  ![2]                        [1]
  ![3]                        [2]
  ![4]                        [3]
  ![5]                        [4]
  !                           [5]
  !                           [6]
  !                           [7]
  !                           [8]
  !                           [9] (GlutenQueryTest.scala:273)

How was this patch tested?

UT

[github-actions]

Thanks for opening a pull request!

Could you open an issue for this pull request on Github Issues?

https://github.com/oap-project/gluten/issues

Then could you also rename commit message and pull request title in the following format?

[GLUTEN-${ISSUES_ID}][COMPONENT]feat/fix: ${detailed message}

See also:

• Other pull requests

[zhli1142015]

cc @zhouyuan , @yma11 , @liujiayi771, @YannByron , could you help to review this PR? Thanks.

[liujiayi771]

LGTM. Thanks.

[yma11]

Why limits to 3.4.2?

[zhli1142015]

This feature is only available after delta 2.3. for spark 3.2 and 3.3 we use delta 2.0 and 2.2.
https://docs.delta.io/latest/delta-deletion-vectors.html