Clean up buffers in case AssertionError

[razajafri]

In case ColumnView throws an AssertionError which it can after #13071, we should close out the open buffers.

fixes #13225

Description

Checklist

• I am familiar with the Contributing Guidelines.
• New or existing tests cover these changes.
• The documentation is up to date with these changes.

[abellina]

Calling .close() here is not what we want, and does not work (would throw here). This is similar to what @ttnghia brought up.

The problem is with sub classes like ColumnVector (what original issue was filed for). Since this subclass is not a ColumnView, calling close() from the constructor of ColumnView is going to call ColumnVector.close() at the wrong time, which does all kinds of bad things at this point of execution:

1. At the time when ColumnView constructor is entered refCount=0, as it gets incremented in the ColumnVector constructor only after the view has successfully constructed.

2. The close constructor will then decrement this refCount setting it to -1, and it should throw "Close called too many times"

  @Override
  public synchronized void close() {
    refCount--;
    offHeap.delRef();                                  
    if (eventHandler != null) {
      eventHandler.onClosed(refCount);
    }
    if (refCount == 0) {
      offHeap.clean(false);
    } else if (refCount < 0) {
      offHeap.logRefCountDebug("double free " + this);
      throw new IllegalStateException("Close called too many times " + this);
    }
  }

[abellina]

You want to treat this case differently since it is a special error on construction.

In the viewHandle constructor case, the code you have seems fine but I would also 0 it out for good measure.

In the OffHeapState constructor case, the state of concern is offHeap. This is created externally in ColumnVector and handed down to the view. I think the only thing you want to do is to close offHeap state directly if you are going to throw. I would also 0 out viewHandle for good measure, it's a pointer to nothing after offHeap closes.

I would put comments on both constructors that this is now happening to make it extra clear and we don't forget this in the future.

[razajafri]

@abellina do you have anything else to add? I think I have addressed all your concerns

[abellina]

small nit, since we know we are creating column vectors, I don't see the point of using the superclass here.

[abellina]

In addition to closing the column vectors/views in the array, we also have pending nativeHandles, sorry I just caught this one. In order to make it safer, we should also need to close the handles which are leaked. Should probably create a helper handles -> vectors, or handles -> views, if we don't have those already, since all the other places you changed are similar.

I would change the loop like so:

try {
  for (int i = 0; i < nativeHandles.length; i++) {
    columnVectors[i] = new ColumnVector(nativeHandles[i]);
    nativeHandles[i] = 0; // mark the handle as consumed
  }
} catch (Throwable t) {
  for (ColumnVector cv : columnVectors) {
    if (cv != null) {
      cv.close();
    }
  }
  for (long handle : nativeHandles) { 
    if (handle != 0) {
      ColumnVector.deleteCudfColumn(handle);
    }
  }
  throw t;
}

[razajafri]

Good catch! But why not close all the nativeHandles? We got them as a result of slice They should all be cleaned out since we have encountered an error.

[razajafri]

I believe the same thing should happen in splitAsViews

[abellina]

Yes, but since you have handed some handles to the vector/view, the expectation is that the vector/view will close the handle they were given. So you don't want to double delete these.

[razajafri]

@abellina thank you for taking a look and catching things that I had missed. I have covered the missed cases I think. I have also created a helper method in ColumnView and also replaced places where it should be called instead of the code we were using before. Please take a look again when you have a chance

[abellina]

I just had one typo change request, otherwise LGTM

[razajafri]

/merge