[VL] Fall back collect_set, min and max when input is complex type

[zhli1142015]

What changes were proposed in this pull request?

#4763 (comment)

Caused by: java.lang.RuntimeException: Exception: VeloxUserError
Error Source: USER
Error Code: INVALID_ARGUMENT
Reason: ROW comparison not supported for values that contain nulls
Retriable: False
Expression: !decoded.base()->containsNullAt(indices[index])
Function: checkNestedNulls
File: /var/git/gluten/ep/build-velox/build/velox_ep/velox/functions/lib/CheckNestedNulls.cpp
Line: 34

Now we use Presto Velox impl for these 3 functions, there is behavior mismatch between Presto and Spark for complex types, we should disable them before implementing them separately for Spark in Velox.

How was this patch tested?

UT.

[github-actions]

Thanks for opening a pull request!

Could you open an issue for this pull request on Github Issues?

https://github.com/apache/incubator-gluten/issues

Then could you also rename commit message and pull request title in the following format?

[GLUTEN-${ISSUES_ID}][COMPONENT]feat/fix: ${detailed message}

See also:

• Other pull requests

[rui-mo]

Looks good. Thanks.

[zhli1142015]

Spark35 failed twice but different errors. and both are related to CSV, cc @jinchengchenghh , thanks.

2024-05-31T07:36:17.3309580Z - change column type from short to int/long
2024-05-31T07:36:17.3335304Z 07:36:17.333 WARN org.apache.spark.util.HadoopFSUtils: The directory file:/tmp/spark-de6cf04d-262a-4ae9-8796-b0bdadc19214 was not found. Was it deleted very recently?
2024-05-31T07:36:17.3443877Z 07:36:17.344 WARN org.apache.spark.util.HadoopFSUtils: The directory file:/tmp/spark-de6cf04d-262a-4ae9-8796-b0bdadc19214 was not found. Was it deleted very recently?
2024-05-31T07:36:17.3552623Z 07:36:17.354 WARN org.apache.spark.util.HadoopFSUtils: The directory file:/tmp/spark-de6cf04d-262a-4ae9-8796-b0bdadc19214 was not found. Was it deleted very recently?
2024-05-31T07:36:17.3663075Z 07:36:17.365 WARN org.apache.spark.util.HadoopFSUtils: The directory file:/tmp/spark-de6cf04d-262a-4ae9-8796-b0bdadc19214 was not found. Was it deleted very recently?
2024-05-31T07:36:17.3771612Z 07:36:17.376 WARN org.apache.spark.util.HadoopFSUtils: The directory file:/tmp/spark-de6cf04d-262a-4ae9-8796-b0bdadc19214 was not found. Was it deleted very recently?
2024-05-31T07:36:17.3881802Z 07:36:17.387 WARN org.apache.spark.util.HadoopFSUtils: The directory file:/tmp/spark-de6cf04d-262a-4ae9-8796-b0bdadc19214 was not found. Was it deleted very recently?
2024-05-31T07:36:17.3991443Z 07:36:17.398 WARN org.apache.spark.util.HadoopFSUtils: The directory file:/tmp/spark-de6cf04d-262a-4ae9-8796-b0bdadc19214 was not found. Was it deleted very recently?
2024-05-31T07:36:17.4099566Z 07:36:17.409 WARN org.apache.spark.util.HadoopFSUtils: The directory file:/tmp/spark-de6cf04d-262a-4ae9-8796-b0bdadc19214 was not found. Was it deleted very recently?
2024-05-31T07:36:17.4208063Z 07:36:17.420 WARN org.apache.spark.util.HadoopFSUtils: The directory file:/tmp/spark-de6cf04d-262a-4ae9-8796-b0bdadc19214 was not found. Was it deleted very recently?
2024-05-31T07:36:17.4317770Z 07:36:17.431 WARN org.apache.spark.util.HadoopFSUtils: The directory file:/tmp/spark-de6cf04d-262a-4ae9-8796-b0bdadc19214 was not found. Was it deleted very recently?
2024-05-31T07:36:17.4426988Z 07:36:17.442 WARN org.apache.spark.util.HadoopFSUtils: The directory file:/tmp/spark-de6cf04d-262a-4ae9-8796-b0bdadc19214 was not found. Was it deleted very recently?
2024-05-31T07:36:17.4802653Z 07:36:17.479 WARN org.apache.spark.util.HadoopFSUtils: The directory file:/tmp/spark-de6cf04d-262a-4ae9-8796-b0bdadc19214 was not found. Was it deleted very recently?
2024-05-31T07:36:17.4847845Z #
2024-05-31T07:36:17.4848564Z # A fatal error has been detected by the Java Runtime Environment:
2024-05-31T07:36:17.4849328Z #
2024-05-31T07:36:17.4849925Z #  SIGSEGV (0xb) at pc=0x00007f2b1f3d88f4, pid=45579, tid=0x00007f2b18ffd700
2024-05-31T07:36:17.4850721Z #
2024-05-31T07:36:17.4851608Z # JRE version: OpenJDK Runtime Environment (8.0_362-b08) (build 1.8.0_362-b08)
2024-05-31T07:36:17.4853188Z # Java VM: OpenJDK 64-Bit Server VM (25.362-b08 mixed mode linux-amd64 compressed oops)
2024-05-31T07:36:17.4854127Z # Problematic frame:
2024-05-31T07:36:17.4855143Z # C  [jnilib-1977168895266751603.tmp+0x14a48f4]  arrow::PoolBuffer::~PoolBuffer()+0x44
2024-05-31T07:36:17.4856029Z #
2024-05-31T07:36:17.4857122Z # Core dump written. Default location: /__w/incubator-gluten/incubator-gluten/gluten-ut/spark35/core or core.45579
2024-05-31T07:36:17.4858365Z #
2024-05-31T07:36:17.4858843Z # An error report file with more information is saved as:
2024-05-31T07:36:17.4859970Z # /__w/incubator-gluten/incubator-gluten/gluten-ut/spark35/hs_err_pid45579.log
2024-05-31T07:36:17.4912553Z 07:36:17.490 WARN org.apache.spark.util.HadoopFSUtils: The directory file:/tmp/spark-de6cf04d-262a-4ae9-8796-b0bdadc19214 was not found. Was it deleted very recently?
2024-05-31T07:36:17.4914354Z - change column type from int to long
2024-05-31T07:36:17.4923626Z [thread 139826895070976 also had an error]
2024-05-31T07:36:17.5317153Z #
2024-05-31T07:36:17.5318039Z # If you would like to submit a bug report, please visit:
2024-05-31T07:36:17.5319958Z #   https://bugzilla.redhat.com/enter_bug.cgi?product=Red%20Hat%20Enterprise%20Linux%208&component=java-1.8.0-openjdk
2024-05-31T07:36:17.5321395Z # The crash happened outside the Java Virtual Machine in native code.
2024-05-31T07:36:17.5322109Z # See problematic frame for where to report the bug.
2024-05-31T07:36:17.5322546Z #

/__w/incubator-gluten/incubator-gluten/ep/build-velox/build/velox_ep/_build/release/third_party/arrow_ep/src/arrow_ep/java/dataset/src/main/cpp/jni_util.cc:79: Failed to update reservation while freeing bytes: Java Exception: java.lang.IncompatibleClassChangeError

/tmp/jnilib-2388859298697668340.tmp(+0x12183e8)[0x7f989485a3e8]
/tmp/jnilib-2388859298697668340.tmp(_ZN5arrow4util8ArrowLogD1Ev+0xed)[0x7f989485a56d]
/tmp/jnilib-2388859298697668340.tmp(_ZN5arrow7dataset3jni31ReservationListenableMemoryPool4FreeEPhll+0x45d)[0x7f9893e5009d]
/tmp/jnilib-2388859298697668340.tmp(_ZN5arrow10PoolBufferD0Ev+0x47)[0x7f9894ae68f7]
/tmp/jnilib-2388859298697668340.tmp(_ZNSt6vectorISt10shared_ptrIN5arrow6BufferEESaIS3_EED1Ev+0x6b)[0x7f9893f641fb]
/tmp/jnilib-2388859298697668340.tmp(_ZNSt23_Sp_counted_ptr_inplaceIN5arrow9ArrayDataESaIS1_ELN9__gnu_cxx12_Lock_policyE2EE10_M_disposeEv+0xda)[0x7f9894071e0a]
/tmp/jnilib-2388859298697668340.tmp(_ZN5arrow17SimpleRecordBatchD1Ev+0x116)[0x7f98949903d6]
/tmp/jnilib-2388859298697668340.tmp(_ZNSt16_Sp_counted_baseILN9__gnu_cxx12_Lock_policyE2EE10_M_releaseEv+0x48)[0x7f9893e49758]
/tmp/jnilib-2388859298697668340.tmp(_ZN5arrow8internal6FnOnceIFvRKNS_10FutureImplEEE6FnImplINS_6FutureISt10shared_ptrINS_11RecordBatchEEE20WrapResultOnComplete8CallbackINSC_14ThenOnCompleteIZNS_23DefaultIfEmptyGeneratorISB_EclEvEUt_NSC_17PassthruOnFailureISI_EEEEEEED0Ev+0x2e)[0x7f9893e9a6ae]
/tmp/jnilib-2388859298697668340.tmp(_ZN5arrow18ConcreteFutureImpl22DoMarkFinishedOrFailedENS_11FutureStateE+0x3a4)[0x7f9894873124]
/tmp/jnilib-2388859298697668340.tmp(_ZN5arrow6FutureISt10shared_ptrINS_11RecordBatchEEE12MarkFinishedENS_6ResultIS3_EE+0xda)[0x7f9893ebac6a]
/tmp/jnilib-2388859298697668340.tmp(_ZN5arrow8internal6FnOnceIFvRKNS_10FutureImplEEE6FnImplINS_6FutureISt10shared_ptrINS_11RecordBatchEEE20WrapResultOnComplete8CallbackINS_6detail16MarkNextFinishedISC_SC_Lb0ELb0EEEEEE6invokeES4_+0x56)[0x7f9893ebb846]
/tmp/jnilib-2388859298697668340.tmp(_ZN5arrow18ConcreteFutureImpl21RunOrScheduleCallbackERKSt10shared_ptrINS_10FutureImplEEONS2_14CallbackRecordEb+0x46)[0x7f98948735f6]
/tmp/jnilib-2388859298697668340.tmp(_ZN5arrow10FutureImpl11AddCallbackENS_8internal6FnOnceIFvRKS0_EEENS_15CallbackOptionsE+0xcb)[0x7f989485eb0b]
/tmp/jnilib-2388859298697668340.tmp(_ZN5arrow8internal6FnOnceIFvRKNS_10FutureImplEEE6FnImplINS_6FutureISt8functionIFNS8_ISt10shared_ptrINS_11RecordBatchEEEEvEEE20WrapResultOnComplete8CallbackINSG_14ThenOnCompleteIZNS_20FutureFirstGeneratorISC_EclEvEUlRKSF_E_NSG_17PassthruOnFailureISO_EEEEEEE6invokeES4_+0xde)[0x7f9893ebbd8e]
/tmp/jnilib-2388859298697668340.tmp(_ZN5arrow18ConcreteFutureImpl22DoMarkFinishedOrFailedENS_11FutureStateE+0x397)[0x7f9894873117]
/tmp/jnilib-2388859298697668340.tmp(+0x83c23b)[0x7f9893e7e23b]
/tmp/jnilib-2388859298697668340.tmp(_ZN5arrow18ConcreteFutureImpl22DoMarkFinishedOrFailedENS_11FutureStateE+0x397)[0x7f9894873117]
/tmp/jnilib-2388859298697668340.tmp(+0x82d04e)[0x7f9893e6f04e]
/tmp/jnilib-2388859298697668340.tmp(_ZN5arrow18ConcreteFutureImpl22DoMarkFinishedOrFailedENS_11FutureStateE+0x397)[0x7f9894873117]
/tmp/jnilib-2388859298697668340.tmp(_ZN5arrow8internal6FnOnceIFvRKNS_10FutureImplEEE6FnImplINS_6FutureISt10shared_ptrINS_3csv15StreamingReaderEEE20WrapResultOnComplete8CallbackINS_6detail16MarkNextFinishedISD_SD_Lb0ELb0EEEEEE6invokeES4_+0x164)[0x7f9893e9be34]
/tmp/jnilib-2388859298697668340.tmp(_ZN5arrow18ConcreteFutureImpl22DoMarkFinishedOrFailedENS_11FutureStateE+0x397)[0x7f9894873117]
/tmp/jnilib-2388859298697668340.tmp(+0x119ba3b)[0x7f98947dda3b]
/tmp/jnilib-2388859298697668340.tmp(_ZN5arrow18ConcreteFutureImpl22DoMarkFinishedOrFailedENS_11FutureStateE+0x397)[0x7f9894873117]
/tmp/jnilib-2388859298697668340.tmp(_ZN5arrow6FutureINS_8internal5EmptyEE12MarkFinishedIS2_vEEvNS_6StatusE+0x94)[0x7f9893ea96e4]
/tmp/jnilib-2388859298697668340.tmp(_ZN5arrow8internal6FnOnceIFvRKNS_10FutureImplEEE6FnImplINS_6FutureINS0_5EmptyEE21WrapStatusyOnComplete8CallbackINS_6detail16MarkNextFinishedISA_SA_Lb1ELb1EEEEEE6invokeES4_+0x28)[0x7f9893ea9848]
/tmp/jnilib-2388859298697668340.tmp(_ZN5arrow18ConcreteFutureImpl21RunOrScheduleCallbackERKSt10shared_ptrINS_10FutureImplEEONS2_14CallbackRecordEb+0x46)[0x7f98948735f6]
/tmp/jnilib-2388859298697668340.tmp(_ZN5arrow10FutureImpl11AddCallbackENS_8internal6FnOnceIFvRKS0_EEENS_15CallbackOptionsE+0xcb)[0x7f989485eb0b]
/tmp/jnilib-2388859298697668340.tmp(_ZNK5arrow6FutureINS_8internal5EmptyEE11AddCallbackINS_6detail16MarkNextFinishedIS3_S3_Lb1ELb1EEENS3_21WrapStatusyOnComplete8CallbackIS7_EEEEvT_NS_15CallbackOptionsE+0x5b)[0x7f98948213bb]
/tmp/jnilib-2388859298697668340.tmp(+0x11c8543)[0x7f989480a543]
/tmp/jnilib-2388859298697668340.tmp(_ZN5arrow18ConcreteFutureImpl22DoMarkFinishedOrFailedENS_11FutureStateE+0x397)[0x7f9894873117]
/tmp/jnilib-2388859298697668340.tmp(_ZN5arrow6FutureISt10shared_ptrINS_6BufferEEE12MarkFinishedENS_6ResultIS3_EE+0xda)[0x7f98[9422](https://github.com/apache/incubator-gluten/actions/runs/9312670615/job/25635452082?pr=5937#step:11:9423)5c4a]
/tmp/jnilib-2388859298697668340.tmp(_ZN5arrow8internal6FnOnceIFvRKNS_10FutureImplEEE6FnImplINS_6FutureISt10shared_ptrINS_6BufferEEE20WrapResultOnComplete8CallbackINS_6detail16MarkNextFinishedISC_SC_Lb0ELb0EEEEEE6invokeES4_+0x56)[0x7f989422a706]
/tmp/jnilib-2388859298697668340.tmp(_ZN5arrow18ConcreteFutureImpl21RunOrScheduleCallbackERKSt10shared_ptrINS_10FutureImplEEONS2_14CallbackRecordEb+0x46)[0x7f98948735f6]
/tmp/jnilib-2388859298697668340.tmp(_ZN5arrow10FutureImpl11AddCallbackENS_8internal6FnOnceIFvRKS0_EEENS_15CallbackOptionsE+0xcb)[0x7f989485eb0b]
/tmp/jnilib-2388859298697668340.tmp(_ZN5arrow8internal6FnOnceIFvRKNS_10FutureImplEEE6FnImplINS_6FutureISt10shared_ptrINS_6BufferEEE20WrapResultOnComplete8CallbackINSC_14ThenOnCompleteIZNS_21TransformingGeneratorISB_SB_E26TransformingGeneratorStateclEvEUlRKSB_E_NSC_17PassthruOnFailureISL_EEEEEEE6invokeES4_+0x99)[0x7f989481fe29]
/tmp/jnilib-2388859298697668340.tmp(_ZN5arrow18ConcreteFutureImpl22DoMarkFinishedOrFailedENS_11FutureStateE+0x397)[0x7f9894873117]
/tmp/jnilib-2388859298697668340.tmp(_ZN5arrow6FutureISt10shared_ptrINS_6BufferEEE12MarkFinishedENS_6ResultIS3_EE+0xda)[0x7f9894225c4a]
/tmp/jnilib-2388859298697668340.tmp(_ZN5arrow8internal6FnOnceIFvvEE6FnImplIZZNS0_8Executor10DoTransferISt10shared_ptrINS_6BufferEENS_6FutureIS9_EENS_6ResultIS9_EEEENSA_IT_EESF_bENUlRKSD_E0_clESH_EUlvE_E6invokeEv+0x52)[0x7f989422a5e2]
/tmp/jnilib-2388859298697668340.tmp(+0x1210abd)[0x7f9894852abd]
/lib64/libstdc++.so.6(+0xc2b23)[0x7f98c7a89b23]
/lib64/libpthread.so.0(+0x81ca)[0x7f99634bb1ca]
/lib64/libc.so.6(clone+0x43)[0x7f9962cf98d3]

[jinchengchenghh]

This is because it does not use gluten modified and compile Arrow jar version, after this PR, it will not happen. #5850

[zhli1142015]

Get it, thanks.