[C++] Support for scalar value recycling in RecordBatch/Table creation

[asfimport]

Please can we have the capability to be able to recycle scalar values during table creation?  It would work as follows:

Upon creation of a new Table/RecordBatch, the length of each column is checked.  If:

• number of columns is > 1 and

• any columns have length 1 and

• not all columns have length 1

  then, the value in the length 1 column(s) should be repeated to make it as long as the other columns. 

  This should only occur if all columns either have length 1 or N (where N is some value greater than 1), and if any columns lengths are values other than 1 or N, we should still get an error as we do now.

Reporter: Nicola Crane / @thisisnic

Related issues:

• [R] Support scalar value recycling in RecordBatch/Table$create() (relates to)

_{Note: This issue was originally created as ARROW-12789. Please see the migration documentation for further details.}

[asfimport]

Joris Van den Bossche / @jorisvandenbossche:
I was going to comment that, alternatively, C++ could also provide the utility to easily/efficiently create an array of a given length from a scalar, and then leave it up to the bindings to check for scalars and create the appropriate array. But it seems (from the PR for the other issue) this is what is already being done with MakeArrayFromScalar.

[asfimport]

Weston Pace / @westonpace:
@jorisvandenbossche I've just opened ARROW-12813 for that as it has come up a few times.

[asfimport]

Weston Pace / @westonpace:
@bkietz  I'm going to tag you for input just because you've done some work with broadcasting in the past (e.g. in regards to projection) which seems quite similar.  Perhaps this PR could be achieved by a "broadcast" compute function which takes in a vector of arrays? (which I suppose is an odd shape for input into the compute layer).

[asfimport]

Nicola Crane / @thisisnic:
Also, feel free to push back on this if you don't think there's a huge amount of application beyond R - I can always look to implement it in the R package's C++ code if this is looking to be a particularly special case that won't be needed elsewhere?

[asfimport]

Ben Kietzman / @bkietz:
In C++ terms, a RecordBatch's column is strictly an array and not a scalar. I'm guessing you'd like to do something similar to

tibble(a = 1:10, s = 0)

that seems useful, but it's not something which makes sense to add in C++. I'd recommend adding a clause to RecordBatch$create which detects Scalars and invokes MakeArrayFromScalar to coerce to an Array

[github-actions]

This issue has been marked as stale because it has had no activity in the past 365 days. Please remove the stale label or comment below, or this issue will be closed in 14 days. If this improvement is still desired but has no current owner, please add the 'Status: needs champion' label.