[C#] ArrayBuilder API to support writing nulls

[asfimport]

There is currently no API in the PrimitiveArrayBuilder class to support writing nulls.  See this TODO - https://github.com/apache/arrow/blob/1515fe10c039fb6685df2e282e2e888b773caa86/csharp/src/Apache.Arrow/Arrays/PrimitiveArrayBuilder.cs#L101.

 

Also see #5381.

 

We should add some APIs to support writing nulls.

Reporter: Eric Erhardt / @eerhardt
Assignee: Anthony Abate / @abbotware

Related issues:

• [C#] Array builders to use bit-packed buffer builder rather than boolean array builder for validity map (is related to)

PRs and other links:

• GitHub Pull Request #6121
• GitHub Pull Request #7032

_{Note: This issue was originally created as ARROW-6603. Please see the migration documentation for further details.}

[asfimport]

Anthony Abate / @abbotware:
I have a few extension methods that do this - on thing i noticed, the spec seems to refer to terms NullBitmap and ValidityBitmaps - I think ValidityBitmap might be the more correct term since 1 = valid, whereas NullBitmap sounds like 1 = null.   My first attempt at creating the nullbitmap inverted all the values

[asfimport]

Anthony Abate / @abbotware:
I added a PR

#6121

 

note that this does not change the existing API, but can be used in-lieu of when creating record batches.

[asfimport]

Anthony Abate / @abbotware:
The reason I did this is because as much as I tried to use the existing API - but I think it can't support null able correctly as many assumptions are baked into the generic always being non-nullable..

 If you see how I implement nullable, I am filling in dummy values in the Value Buffer for nulls, but correctly setting the value bitmap...   this results in a reader of the arrow file correctly interpreting the NULL.

It still might be possible a builder method called AppendNullable<T?>() into the existing builder code...   but I was able to get the code in the PR to work fairly quickly once I understood the flatbuffer spec

[asfimport]

Zachary Gramana:
I came across this conversation late, and after implementing an alternative approach which much more in-line with other Arrow implementations.

I have submitted GitHub Pull Request #7032 which includes:

• A newly added interface member, AppendNull, along with implementations for PrimitiveArrayBuilder and Binary.BuilderBase.
• Additional work to finish the previously stubbed support for NullBitmapBuffer in a few of the specialized Array classes.
• Several new and expanded tests.

[asfimport]

Eric Erhardt / @eerhardt:
Issue resolved by pull request 7032
#7032