What is the problem the feature request solves?
We disabled Comet shuffle for Spark core-1 test because negative ref count issue reported by Java Arrow. After fixing memory issues and some changes to more follow C Data interface, we may re-enable Comet shuffle for Spark core-1 test in CI.
Describe the potential solution
No response
Additional context
No response