Add host memory retries for GeneratedInternalRowToCudfRowIterator

[jbrennan333]

This PR adds host memory allocation retries for GeneratedInternalRowToCudfRowIterator.
This initial version only handles retries, it does not handle splitting the input data into batches.
It is the first of several PRs that will be used to address #8887.

I have run the existing unit and integration tests, and I verified no performance impact for nds on spark2a, although I do not think nds is exercising this code.

I am putting this up as a draft because it includes a change that is also in #9908, and to do additional testing.

[jbrennan333]

build

[jbrennan333]

build

[revans2]

Looks good to me.

[abellina]

nit,

Suggested change

	((long) numRowsEstimate + 1) * BYTES_PER_OFFSET, true);
	((long) numRowsEstimate + 1) * BYTES_PER_OFFSET, /*preferPinned*/ true);

[abellina]

nit,

Suggested change

	return HostAlloc$.MODULE$.alloc(dataLength, true);
	return HostAlloc$.MODULE$.alloc(dataLength, /*preferPinned*/ true);

[gerashegalov]

LGTM

[jbrennan333]

build

[jbrennan333]

I have been testing this locally, running nds query75 with the ShuffleExchangeExec operator disabled. This causes it to add GpuRowToColumnar targetsize(1073741824) to the plan. I am running with 16 executor cores, and limiting off heap memory. With these changes, I hit CpuSplitAndRetryOOM after reducing the offheap memory to about 24G.

As a comparison, when I run without these changes, but I change the HostMemoryBuffer.allocate() calls for dataBuffer and offsetBuffer to use HostAlloc.alloc(), I hit CpuRetryOOM with offheap memory at around 24G as well.

So it doesn't seem like the withRetryNoSplit calls are buying us much in this case, beyond the retries in HostAlloc.alloc(). @revans2 is does this make sense?

[revans2]

This indicates that we are still using a lot of non-spillable host memory. So we need to track down who is using the memory and not making it spillable. I'll see what I can come up with to be able to help debug this. In the short term I am fine with merging this in as it is not worse than what we have today.

[jbrennan333]

Thanks @revans2. I will put this in, but will continue to test it as I work on splitting the inputs.