[VL] Support GPU shuffle reader

[jinchengchenghh]

Description

1. Move the lock from WholeStageResultIterator constructor to shuffle reader, the threads can prepare the first batch in advance, now it is a small batch, for GPU, it will produce a big batch with bytes 1GB
2. Implement the decompression, convert buffer to cudf table, resize batch in GPU
3. Then the threads can prepare read buffer from file
4. The other thread to notify needs to fetch more data than batch size, prepare batches for GPU to process, reserve the bytes from pool, if reserve successfully, CPU threads fetch data in the background, save the batches in Vector.
5. The waited CPU may can do the decompression and resize batch and wait GPU to fetch.

Gluten version

None

[jinchengchenghh]

In CPU Gluten, we uses VeloxResizesBatchesExec to concat the batch after shuffle reader, this can help concat from different stream readers VS concat in ShuffleReader.
But in GPU gluten, if we split to cudf::table and concatenateTables, most of the concatenateTables operation is CPU work, because we add the GPU lock to restrict GPU threads before generating cudf::table, so we should do the concat in CPU side.

The lock is used for limited GPU memory. So most time the number of GPU threads should be less than CPU threads.

For example, the EC2 instance g4dn.2xlarge, CPU memory is 30G while GPU memory is 15G.

Future work:

• Operator CudfColumnarExchangeExec, save all the buffers to std::vector, wrap it to new class BufferColumnarBatch. For example, the flat Vector contains {BufferPtr nulls, BufferPtr values}, not it is {std::vector nulls, std::vector values}
• Operator VeloxBuffersBatchExec, The first thread uses GPU decompression while the other threads still uses CPU decompression, because the other threads cannot execute in GPU immediately, they need to wait for GPU execution.
• The other threads fetch the BufferColumnarBatch and concat to big batch continuously util there is not enough memory, save the Batches to std::vector and wait the downstream operator to fetch.
• After decompression, merge the buffer especially for the validity buffer which requires ByteForBits, different copy, and also different for the offset buffer, needs to recompute the offset.
• Lock and construct cudf::table

Question:

Is the first thread GPU decompression make senses? Looks like complex and may not benefit because the concat buffer is in single GPU thread in these cases.