ARROW-10640: [C++] An "if_else" kernel to combine two arrays based on a mask #10410

nirandaperera · 2021-05-26T18:19:27Z

Adding a preliminary impl for an if_else(cond: Datum, left: Datum, right: Datum) function. It works as follows,

def if_else(cond, left, right):
    for c, true_val, false_val in zip(cond, left, right):
        if c:
            yield true_val
        else:
            yield false_val

null values will be promoted to the output.

github-actions · 2021-05-26T18:19:48Z

https://issues.apache.org/jira/browse/ARROW-10640

nirandaperera · 2021-05-26T19:06:40Z

@bkietz I think the bitmap ops approach is simpler than the bitmap visitor approach. WDYT?
bitmap ops -

arrow/cpp/src/arrow/compute/kernels/scalar_if_else.cc

Line 307 in 8c0e8d9

    
           /*Status PromoteNulls(KernelContext* ctx, const ArrayData& cond, const ArrayData& left,

onward

bitmap visitor -

arrow/cpp/src/arrow/compute/kernels/scalar_if_else.cc

Line 156 in 8c0e8d9

Status PromoteNullsVisitor(KernelContext* ctx, const ArrayData& cond,

onward

lidavidm

This looks good! I left some general comments.

As for visitor vs bitmap ops, I think the visitor-based one is fine. It does break down the cases nicely, even if it's a bit verbose.

cpp/src/arrow/compute/api_scalar.h

cpp/src/arrow/compute/kernels/scalar_if_else_test.cc

lidavidm · 2021-05-27T11:34:11Z

cpp/src/arrow/compute/kernels/scalar_if_else.cc

+                                 output->buffers[0]->mutable_data());
+      break;
+    case COND_ALL_VALID:  // = 1
+      // out_valid = 0 --> nothing to do; but requires out_valid to be a all-zero buffer


AllocateBitmap doesn't zero the allocation

arrow/cpp/src/arrow/compute/kernel.h

Lines 61 to 64 in 1769888

/// \brief Allocate buffer for bitmap from the context's memory pool. Like

/// Allocate, the contents of the buffer are not initialized but the last

/// byte is preemptively zeroed to help avoid ASAN or valgrind issues.

Result<std::shared_ptr<ResizableBuffer>> AllocateBitmap(int64_t num_bits);

I think there is a mismatch in the kernel comment and the impl.

arrow/cpp/src/arrow/compute/kernel.cc

Line 56 in 29130ca

// Since bitmaps are typically written bit by bit, we could leak uninitialized bits.

It looks like it is zeroed out for bitmaps

Hmm, we should probably update comment then. CC @bkietz given KernelContext::AllocateBitmap has to zero the allocation to satisfy Valgrind et al, should we just go ahead and guarantee that it returns a pre-cleared bitmap?

I'd prefer to leave this for a follow up where we can try to zero only the final byte (as described by the comment) and see if Valgrind is appeased. If not, the comment can be updated

cpp/src/arrow/compute/kernels/scalar_if_else.cc

cpp/src/arrow/compute/kernels/scalar_if_else_test.cc

lidavidm · 2021-05-27T15:27:37Z

cpp/src/arrow/compute/kernels/scalar_if_else.cc

+        // out_buf = ones
+        ARROW_ASSIGN_OR_RAISE(out_buf, ctx->AllocateBitmap(cond.length));
+        // filling with UINT8_MAX upto the buffer's size (in bytes)
+        arrow::compute::internal::SetMemory<UINT8_MAX>(out_buf.get());


You could just inline the memset call here - no need to put a helper in a header IMO unless we're going to use it a lot.

Please inline the call to memset here

cpp/src/arrow/compute/kernels/scalar_if_else.cc

cpp/src/arrow/compute/kernels/scalar_if_else_test.cc

cpp/src/arrow/compute/kernels/scalar_if_else.cc

nirandaperera · 2021-06-01T01:29:48Z

@github-actions autotune

cpp/src/arrow/compute/kernels/scalar_if_else_test.cc

cpp/src/arrow/compute/kernels/scalar_if_else.cc

cpp/src/arrow/compute/kernels/scalar_if_else_test.cc

cpp/src/arrow/compute/kernels/scalar_if_else.cc

nirandaperera · 2021-06-01T18:41:57Z

@bkietz @lidavidm I think I have added all the changes requested for this PR so far. Anything else?

bkietz

just two more improvements

cpp/src/arrow/compute/kernels/scalar_if_else_test.cc

bkietz · 2021-06-01T20:29:37Z

cpp/src/arrow/compute/kernels/scalar_if_else.cc

+        // out_buf = ones
+        ARROW_ASSIGN_OR_RAISE(out_buf, ctx->AllocateBitmap(cond.length));
+        // filling with UINT8_MAX upto the buffer's size (in bytes)
+        arrow::compute::internal::SetMemory<UINT8_MAX>(out_buf.get());


Please inline the call to memset here

cpp/src/arrow/compute/api_scalar.h

jorisvandenbossche · 2021-06-02T10:16:00Z

I did a quick comparison with np.where (using plain float64 arrays without nulls to have it comparable), and it seems this implementation is already doing quite good compared to numpy (using a release build):

In [19]: N = 10_000_000
    ...: arr1 = np.random.randn(N)
    ...: arr2 = np.random.randn(N)
    ...: mask = np.random.randint(0, 2, N).astype(bool)
    ...: 
    ...: pa_arr1 = pa.array(arr1)
    ...: pa_arr2 = pa.array(arr2)
    ...: pa_mask = pa.array(mask)
    ...: 

In [20]: %timeit np.where(mask, arr1, arr2)
82.3 ms ± 8.76 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)

In [21]: %timeit pc.if_else(pa_mask, pa_arr1, pa_arr2)
50.4 ms ± 6.08 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)

nirandaperera · 2021-06-02T12:56:53Z

I did a quick comparison with np.where (using plain float64 arrays without nulls to have it comparable), and it seems this implementation is already doing quite good compared to numpy (using a release build):

In [19]: N = 10_000_000
    ...: arr1 = np.random.randn(N)
    ...: arr2 = np.random.randn(N)
    ...: mask = np.random.randint(0, 2, N).astype(bool)
    ...: 
    ...: pa_arr1 = pa.array(arr1)
    ...: pa_arr2 = pa.array(arr2)
    ...: pa_mask = pa.array(mask)
    ...: 

In [20]: %timeit np.where(mask, arr1, arr2)
82.3 ms ± 8.76 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)

In [21]: %timeit pc.if_else(pa_mask, pa_arr1, pa_arr2)
50.4 ms ± 6.08 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)

We may be able to further improve this if we directly use vector operations inside the kernel (I haven't checked the compiled code yet, may be compiler does that already), because if-else use case directly map to mask_move operations in AVX512

Co-authored-by: Benjamin Kietzman <bengilgit@gmail.com>

Co-authored-by: Joris Van den Bossche <jorisvandenbossche@gmail.com>

bkietz

Excellent work, thanks for doing this!

lidavidm · 2021-06-02T20:29:46Z

It looks like there's an actual build failure on CentOS/GCC4.8 (expand "Dump install logs" to get the actual failure):

In file included from /tmp/Rtmpmpy1Y5/file7d77bb9a74/src/arrow/CMakeFiles/arrow_objlib.dir/Unity/unity_17_cxx.cxx:4:0:
/arrow/cpp/src/arrow/compute/kernels/scalar_if_else.cc: In function ‘arrow::Status arrow::compute::{anonymous}::PromoteNullsVisitor(arrow::compute::KernelContext*, const arrow::Datum&, const arrow::Datum&, const arrow::Datum&, arrow::ArrayData*)’:
/arrow/cpp/src/arrow/compute/kernels/scalar_if_else.cc:121:8: error: no matching function for call to ‘arrow::internal::Bitmap::VisitWords(<brace-enclosed initializer list>, arrow::compute::{anonymous}::PromoteNullsVisitor(arrow::compute::KernelContext*, const arrow::Datum&, const arrow::Datum&, const arrow::Datum&, arrow::ArrayData*)::__lambda44)’
       });
        ^
/arrow/cpp/src/arrow/compute/kernels/scalar_if_else.cc:121:8: note: candidate is:
In file included from /arrow/cpp/src/arrow/compute/kernels/scalar_if_else.cc:22:0,
                 from /tmp/Rtmpmpy1Y5/file7d77bb9a74/src/arrow/CMakeFiles/arrow_objlib.dir/Unity/unity_17_cxx.cxx:4:
/arrow/cpp/src/arrow/util/bitmap.h:128:18: note: template<long unsigned int N, class Visitor, class Word> static int64_t arrow::internal::Bitmap::VisitWords(const arrow::internal::Bitmap (&)[N], Visitor&&)
   static int64_t VisitWords(const Bitmap (&bitmaps_arg)[N], Visitor&& visitor) {
                  ^
/arrow/cpp/src/arrow/util/bitmap.h:128:18: note:   template argument deduction/substitution failed:
In file included from /tmp/Rtmpmpy1Y5/file7d77bb9a74/src/arrow/CMakeFiles/arrow_objlib.dir/Unity/unity_17_cxx.cxx:4:0:
/arrow/cpp/src/arrow/compute/kernels/scalar_if_else.cc:121:8: note:   couldn't deduce template parameter ‘N’
       });
        ^
/arrow/cpp/src/arrow/compute/kernels/scalar_if_else.cc:127:12: error: no matching function for call to ‘arrow::internal::Bitmap::VisitWords(<brace-enclosed initializer list>, arrow::compute::{anonymous}::PromoteNullsVisitor(arrow::compute::KernelContext*, const arrow::Datum&, const arrow::Datum&, const arrow::Datum&, arrow::ArrayData*)::__lambda45)’
           });
            ^
/arrow/cpp/src/arrow/compute/kernels/scalar_if_else.cc:127:12: note: candidate is:
In file included from /arrow/cpp/src/arrow/compute/kernels/scalar_if_else.cc:22:0,
                 from /tmp/Rtmpmpy1Y5/file7d77bb9a74/src/arrow/CMakeFiles/arrow_objlib.dir/Unity/unity_17_cxx.cxx:4:
/arrow/cpp/src/arrow/util/bitmap.h:128:18: note: template<long unsigned int N, class Visitor, class Word> static int64_t arrow::internal::Bitmap::VisitWords(const arrow::internal::Bitmap (&)[N], Visitor&&)
   static int64_t VisitWords(const Bitmap (&bitmaps_arg)[N], Visitor&& visitor) {
            ^
/arrow/cpp/src/arrow/compute/kernels/scalar_if_else.cc:157:12: note: candidate is:
In file included from /arrow/cpp/src/arrow/compute/kernels/scalar_if_else.cc:22:0,
                 from /tmp/Rtmpmpy1Y5/file7d77bb9a74/src/arrow/CMakeFiles/arrow_objlib.dir/Unity/unity_17_cxx.cxx:4:
/arrow/cpp/src/arrow/util/bitmap.h:128:18: note: template<long unsigned int N, class Visitor, class Word> static int64_t arrow::internal::Bitmap::VisitWords(const arrow::internal::Bitmap (&)[N], Visitor&&)
   static int64_t VisitWords(const Bitmap (&bitmaps_arg)[N], Visitor&& visitor) {
                  ^
/arrow/cpp/src/arrow/util/bitmap.h:128:18: note:   template argument deduction/substitution failed:
In file included from /tmp/Rtmpmpy1Y5/file7d77bb9a74/src/arrow/CMakeFiles/arrow_objlib.dir/Unity/unity_17_cxx.cxx:4:0:
/arrow/cpp/src/arrow/compute/kernels/scalar_if_else.cc:157:12: note:   couldn't deduce template parameter ‘N’
           });
            ^
/arrow/cpp/src/arrow/compute/kernels/scalar_if_else.cc:164:12: error: no matching function for call to ‘arrow::internal::Bitmap::VisitWords(<brace-enclosed initializer list>, arrow::compute::{anonymous}::PromoteNullsVisitor(arrow::compute::KernelContext*, const arrow::Datum&, const arrow::Datum&, const arrow::Datum&, arrow::ArrayData*)::__lambda50)’
           });
            ^
/arrow/cpp/src/arrow/compute/kernels/scalar_if_else.cc:164:12: note: candidate is:
In file included from /arrow/cpp/src/arrow/compute/kernels/scalar_if_else.cc:22:0,
                 from /tmp/Rtmpmpy1Y5/file7d77bb9a74/src/arrow/CMakeFiles/arrow_objlib.dir/Unity/unity_17_cxx.cxx:4:
/arrow/cpp/src/arrow/util/bitmap.h:128:18: note: template<long unsigned int N, class Visitor, class Word> static int64_t arrow::internal::Bitmap::VisitWords(const arrow::internal::Bitmap (&)[N], Visitor&&)
   static int64_t VisitWords(const Bitmap (&bitmaps_arg)[N], Visitor&& visitor) {
                  ^
/arrow/cpp/src/arrow/util/bitmap.h:128:18: note:   template argument deduction/substitution failed:
In file included from /tmp/Rtmpmpy1Y5/file7d77bb9a74/src/arrow/CMakeFiles/arrow_objlib.dir/Unity/unity_17_cxx.cxx:4:0:
/arrow/cpp/src/arrow/compute/kernels/scalar_if_else.cc:164:12: note:   couldn't deduce template parameter ‘N’
           });
            ^
/arrow/cpp/src/arrow/compute/kernels/scalar_if_else.cc:171:27: error: no matching function for call to ‘arrow::internal::Bitmap::VisitWords(<brace-enclosed initializer list>, arrow::compute::{anonymous}::PromoteNullsVisitor(arrow::compute::KernelContext*, const arrow::Datum&, const arrow::Datum&, const arrow::Datum&, arrow::ArrayData*)::__lambda51)’
                          });
                           ^
/arrow/cpp/src/arrow/compute/kernels/scalar_if_else.cc:171:27: note: candidate is:
In file included from /arrow/cpp/src/arrow/compute/kernels/scalar_if_else.cc:22:0,
                 from /tmp/Rtmpmpy1Y5/file7d77bb9a74/src/arrow/CMakeFiles/arrow_objlib.dir/Unity/unity_17_cxx.cxx:4:
/arrow/cpp/src/arrow/util/bitmap.h:128:18: note: template<long unsigned int N, class Visitor, class Word> static int64_t arrow::internal::Bitmap::VisitWords(const arrow::internal::Bitmap (&)[N], Visitor&&)
   static int64_t VisitWords(const Bitmap (&bitmaps_arg)[N], Visitor&& visitor) {
                  ^
/arrow/cpp/src/arrow/util/bitmap.h:128:18: note:   template argument deduction/substitution failed:
In file included from /tmp/Rtmpmpy1Y5/file7d77bb9a74/src/arrow/CMakeFiles/arrow_objlib.dir/Unity/unity_17_cxx.cxx:4:0:
/arrow/cpp/src/arrow/compute/kernels/scalar_if_else.cc:171:27: note:   couldn't deduce template parameter ‘N’
                          });

nirandaperera · 2021-06-03T12:37:16Z

It looks like there's an actual build failure on CentOS/GCC4.8 (expand "Dump install logs" to get the actual failure):

In file included from /tmp/Rtmpmpy1Y5/file7d77bb9a74/src/arrow/CMakeFiles/arrow_objlib.dir/Unity/unity_17_cxx.cxx:4:0:
/arrow/cpp/src/arrow/compute/kernels/scalar_if_else.cc: In function ‘arrow::Status arrow::compute::{anonymous}::PromoteNullsVisitor(arrow::compute::KernelContext*, const arrow::Datum&, const arrow::Datum&, const arrow::Datum&, arrow::ArrayData*)’:
/arrow/cpp/src/arrow/compute/kernels/scalar_if_else.cc:121:8: error: no matching function for call to ‘arrow::internal::Bitmap::VisitWords(<brace-enclosed initializer list>, arrow::compute::{anonymous}::PromoteNullsVisitor(arrow::compute::KernelContext*, const arrow::Datum&, const arrow::Datum&, const arrow::Datum&, arrow::ArrayData*)::__lambda44)’
       });
        ^
/arrow/cpp/src/arrow/compute/kernels/scalar_if_else.cc:121:8: note: candidate is:
In file included from /arrow/cpp/src/arrow/compute/kernels/scalar_if_else.cc:22:0,
                 from /tmp/Rtmpmpy1Y5/file7d77bb9a74/src/arrow/CMakeFiles/arrow_objlib.dir/Unity/unity_17_cxx.cxx:4:
/arrow/cpp/src/arrow/util/bitmap.h:128:18: note: template<long unsigned int N, class Visitor, class Word> static int64_t arrow::internal::Bitmap::VisitWords(const arrow::internal::Bitmap (&)[N], Visitor&&)
   static int64_t VisitWords(const Bitmap (&bitmaps_arg)[N], Visitor&& visitor) {
                  ^
/arrow/cpp/src/arrow/util/bitmap.h:128:18: note:   template argument deduction/substitution failed:
In file included from /tmp/Rtmpmpy1Y5/file7d77bb9a74/src/arrow/CMakeFiles/arrow_objlib.dir/Unity/unity_17_cxx.cxx:4:0:
/arrow/cpp/src/arrow/compute/kernels/scalar_if_else.cc:121:8: note:   couldn't deduce template parameter ‘N’
       });
        ^
/arrow/cpp/src/arrow/compute/kernels/scalar_if_else.cc:127:12: error: no matching function for call to ‘arrow::internal::Bitmap::VisitWords(<brace-enclosed initializer list>, arrow::compute::{anonymous}::PromoteNullsVisitor(arrow::compute::KernelContext*, const arrow::Datum&, const arrow::Datum&, const arrow::Datum&, arrow::ArrayData*)::__lambda45)’
           });
            ^
/arrow/cpp/src/arrow/compute/kernels/scalar_if_else.cc:127:12: note: candidate is:
In file included from /arrow/cpp/src/arrow/compute/kernels/scalar_if_else.cc:22:0,
                 from /tmp/Rtmpmpy1Y5/file7d77bb9a74/src/arrow/CMakeFiles/arrow_objlib.dir/Unity/unity_17_cxx.cxx:4:
/arrow/cpp/src/arrow/util/bitmap.h:128:18: note: template<long unsigned int N, class Visitor, class Word> static int64_t arrow::internal::Bitmap::VisitWords(const arrow::internal::Bitmap (&)[N], Visitor&&)
   static int64_t VisitWords(const Bitmap (&bitmaps_arg)[N], Visitor&& visitor) {
            ^
/arrow/cpp/src/arrow/compute/kernels/scalar_if_else.cc:157:12: note: candidate is:
In file included from /arrow/cpp/src/arrow/compute/kernels/scalar_if_else.cc:22:0,
                 from /tmp/Rtmpmpy1Y5/file7d77bb9a74/src/arrow/CMakeFiles/arrow_objlib.dir/Unity/unity_17_cxx.cxx:4:
/arrow/cpp/src/arrow/util/bitmap.h:128:18: note: template<long unsigned int N, class Visitor, class Word> static int64_t arrow::internal::Bitmap::VisitWords(const arrow::internal::Bitmap (&)[N], Visitor&&)
   static int64_t VisitWords(const Bitmap (&bitmaps_arg)[N], Visitor&& visitor) {
                  ^
/arrow/cpp/src/arrow/util/bitmap.h:128:18: note:   template argument deduction/substitution failed:
In file included from /tmp/Rtmpmpy1Y5/file7d77bb9a74/src/arrow/CMakeFiles/arrow_objlib.dir/Unity/unity_17_cxx.cxx:4:0:
/arrow/cpp/src/arrow/compute/kernels/scalar_if_else.cc:157:12: note:   couldn't deduce template parameter ‘N’
           });
            ^
/arrow/cpp/src/arrow/compute/kernels/scalar_if_else.cc:164:12: error: no matching function for call to ‘arrow::internal::Bitmap::VisitWords(<brace-enclosed initializer list>, arrow::compute::{anonymous}::PromoteNullsVisitor(arrow::compute::KernelContext*, const arrow::Datum&, const arrow::Datum&, const arrow::Datum&, arrow::ArrayData*)::__lambda50)’
           });
            ^
/arrow/cpp/src/arrow/compute/kernels/scalar_if_else.cc:164:12: note: candidate is:
In file included from /arrow/cpp/src/arrow/compute/kernels/scalar_if_else.cc:22:0,
                 from /tmp/Rtmpmpy1Y5/file7d77bb9a74/src/arrow/CMakeFiles/arrow_objlib.dir/Unity/unity_17_cxx.cxx:4:
/arrow/cpp/src/arrow/util/bitmap.h:128:18: note: template<long unsigned int N, class Visitor, class Word> static int64_t arrow::internal::Bitmap::VisitWords(const arrow::internal::Bitmap (&)[N], Visitor&&)
   static int64_t VisitWords(const Bitmap (&bitmaps_arg)[N], Visitor&& visitor) {
                  ^
/arrow/cpp/src/arrow/util/bitmap.h:128:18: note:   template argument deduction/substitution failed:
In file included from /tmp/Rtmpmpy1Y5/file7d77bb9a74/src/arrow/CMakeFiles/arrow_objlib.dir/Unity/unity_17_cxx.cxx:4:0:
/arrow/cpp/src/arrow/compute/kernels/scalar_if_else.cc:164:12: note:   couldn't deduce template parameter ‘N’
           });
            ^
/arrow/cpp/src/arrow/compute/kernels/scalar_if_else.cc:171:27: error: no matching function for call to ‘arrow::internal::Bitmap::VisitWords(<brace-enclosed initializer list>, arrow::compute::{anonymous}::PromoteNullsVisitor(arrow::compute::KernelContext*, const arrow::Datum&, const arrow::Datum&, const arrow::Datum&, arrow::ArrayData*)::__lambda51)’
                          });
                           ^
/arrow/cpp/src/arrow/compute/kernels/scalar_if_else.cc:171:27: note: candidate is:
In file included from /arrow/cpp/src/arrow/compute/kernels/scalar_if_else.cc:22:0,
                 from /tmp/Rtmpmpy1Y5/file7d77bb9a74/src/arrow/CMakeFiles/arrow_objlib.dir/Unity/unity_17_cxx.cxx:4:
/arrow/cpp/src/arrow/util/bitmap.h:128:18: note: template<long unsigned int N, class Visitor, class Word> static int64_t arrow::internal::Bitmap::VisitWords(const arrow::internal::Bitmap (&)[N], Visitor&&)
   static int64_t VisitWords(const Bitmap (&bitmaps_arg)[N], Visitor&& visitor) {
                  ^
/arrow/cpp/src/arrow/util/bitmap.h:128:18: note:   template argument deduction/substitution failed:
In file included from /tmp/Rtmpmpy1Y5/file7d77bb9a74/src/arrow/CMakeFiles/arrow_objlib.dir/Unity/unity_17_cxx.cxx:4:0:
/arrow/cpp/src/arrow/compute/kernels/scalar_if_else.cc:171:27: note:   couldn't deduce template parameter ‘N’
                          });

Ah! thanks @lidavidm I missed this... it seems to be a gcc 4.x thing, isn't it.

BTW @bkietz should I add string type support to this as well, or can I leave it for later development?

jorisvandenbossche · 2021-06-03T17:21:27Z

should I add string type support to this as well, or can I leave it for later development?

I think it is important to add support for strings as well, but it's probably a good idea to leave that for a follow-up PR so we can get this merged? (I can imagine the implementation to be a bit different / more complex for variable length array elements)

lidavidm · 2021-06-03T17:30:01Z

I agree, and also Niranda already filed ARROW-12955 to take care of that followup.

lidavidm

Looks like that fixed the builds, and the remaining failures are not relevant.

…s based on a mask Adding a preliminary impl for an `if_else(cond: Datum, left: Datum, right: Datum)` function. It works as follows, ```python def if_else(cond, left, right): for c, true_val, false_val in zip(cond, left, right): if c: yield true_val else: yield false_val ``` `null` values will be promoted to the output. Closes apache#10410 from nirandaperera/ARROW-10640 Authored-by: niranda perera <niranda.perera@gmail.com> Signed-off-by: David Li <li.davidm96@gmail.com>

github-actions bot added the Component: C++ label May 26, 2021

pitrou self-requested a review May 26, 2021 18:28

lidavidm reviewed May 27, 2021

View reviewed changes

lidavidm mentioned this pull request May 27, 2021

ARROW-9430: [C++] Implement replace_with_mask kernel #10412

Closed

lidavidm reviewed May 27, 2021

View reviewed changes

nirandaperera requested review from bkietz and lidavidm May 27, 2021 17:01

bkietz requested changes May 27, 2021

View reviewed changes

cpp/src/arrow/compute/kernels/scalar_if_else.cc Outdated Show resolved Hide resolved

cpp/src/arrow/compute/kernels/scalar_if_else.cc Show resolved Hide resolved

nirandaperera requested a review from bkietz May 27, 2021 20:32

nirandaperera marked this pull request as ready for review May 27, 2021 20:43

lidavidm reviewed May 27, 2021

View reviewed changes

cpp/src/arrow/compute/kernels/scalar_if_else.cc Outdated Show resolved Hide resolved

cpp/src/arrow/compute/kernels/scalar_if_else_test.cc Outdated Show resolved Hide resolved

cpp/src/arrow/compute/kernels/scalar_if_else.cc Outdated Show resolved Hide resolved

bkietz requested changes May 28, 2021

View reviewed changes

cpp/src/arrow/compute/kernels/scalar_if_else.cc Outdated Show resolved Hide resolved

nirandaperera requested review from bkietz and lidavidm May 28, 2021 14:21

lidavidm reviewed May 28, 2021

View reviewed changes

cpp/src/arrow/compute/kernels/scalar_if_else.cc Outdated Show resolved Hide resolved

cpp/src/arrow/compute/kernels/scalar_if_else.cc Outdated Show resolved Hide resolved

cpp/src/arrow/compute/kernels/scalar_if_else.cc Outdated Show resolved Hide resolved

nirandaperera requested a review from lidavidm June 1, 2021 01:32

lidavidm reviewed Jun 1, 2021

View reviewed changes

cpp/src/arrow/compute/kernels/scalar_if_else_test.cc Outdated Show resolved Hide resolved

cpp/src/arrow/compute/kernels/scalar_if_else.cc Outdated Show resolved Hide resolved

cpp/src/arrow/compute/kernels/scalar_if_else_test.cc Outdated Show resolved Hide resolved

bkietz requested changes Jun 1, 2021

View reviewed changes

cpp/src/arrow/compute/kernels/scalar_if_else.cc Outdated Show resolved Hide resolved

cpp/src/arrow/compute/kernels/scalar_if_else.cc Outdated Show resolved Hide resolved

cpp/src/arrow/compute/kernels/scalar_if_else.cc Outdated Show resolved Hide resolved

nirandaperera requested a review from bkietz June 1, 2021 17:18

bkietz requested changes Jun 1, 2021

View reviewed changes

jorisvandenbossche changed the title ~~ARROW-10640: [C++] A "where" kernel to combine two arrays based on a mask~~ ARROW-10640: [C++] A, "if_else" ("where") kernel to combine two arrays based on a mask Jun 2, 2021

jorisvandenbossche reviewed Jun 2, 2021

View reviewed changes

cpp/src/arrow/compute/api_scalar.h Outdated Show resolved Hide resolved

nirandaperera added 2 commits June 2, 2021 11:40

adding basic structure

b010b2f

working primitive types

c909293

nirandaperera and others added 17 commits June 2, 2021 11:40

cosmetic changes

68f065b

Update cpp/src/arrow/compute/kernels/scalar_if_else.cc

0c25c2a

Co-authored-by: Benjamin Kietzman <bengilgit@gmail.com>

moving test cases to a method

f3ddd66

adding datum level null promotion

54575a9

fixing slicing issue

e32d709

rewriting the tests

e49842a

Autoformat/render all the things [automated commit]

c54e88e

adding comments

4e58697

Update cpp/src/arrow/compute/kernels/scalar_if_else.cc

7e9dff5

Co-authored-by: Benjamin Kietzman <bengilgit@gmail.com>

minor bug fix

ee573a8

minor bug fix

c6d0cdb

adding discussed changes to the tests

e15b9df

removing offset test case

4fa62c3

Update cpp/src/arrow/compute/kernels/scalar_if_else_test.cc

6891211

Co-authored-by: Benjamin Kietzman <bengilgit@gmail.com>

adding PR review comments

b8802bb

Update cpp/src/arrow/compute/api_scalar.h

cee34c2

Co-authored-by: Joris Van den Bossche <jorisvandenbossche@gmail.com>

adding docs

4dbe076

nirandaperera force-pushed the ARROW-10640 branch from 6208b36 to 4dbe076 Compare June 2, 2021 15:45

bkietz approved these changes Jun 2, 2021

View reviewed changes

fixing gcc 4.8.1 compilation issue

52b6a83

lidavidm approved these changes Jun 3, 2021

View reviewed changes

lidavidm closed this in 5baebba Jun 3, 2021

ianmcook changed the title ~~ARROW-10640: [C++] A, "if_else" ("where") kernel to combine two arrays based on a mask~~ ARROW-10640: [C++] An "if_else" kernel to combine two arrays based on a mask Jun 3, 2021

pitrou mentioned this pull request Jun 10, 2021

ARROW-11044: [C++] Add "replace" kernel #9024

Closed

asfimport mentioned this pull request Jul 2, 2021

[C++] An "if_else" kernel to combine two arrays based on a mask #26596

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ARROW-10640: [C++] An "if_else" kernel to combine two arrays based on a mask #10410

ARROW-10640: [C++] An "if_else" kernel to combine two arrays based on a mask #10410

nirandaperera commented May 26, 2021

github-actions bot commented May 26, 2021

nirandaperera commented May 26, 2021

lidavidm left a comment

lidavidm May 27, 2021

nirandaperera May 27, 2021

lidavidm May 27, 2021

bkietz May 27, 2021

lidavidm May 27, 2021

bkietz Jun 1, 2021

nirandaperera commented Jun 1, 2021

nirandaperera commented Jun 1, 2021 •

edited

Loading

bkietz left a comment

bkietz Jun 1, 2021

jorisvandenbossche commented Jun 2, 2021

nirandaperera commented Jun 2, 2021

bkietz left a comment

lidavidm commented Jun 2, 2021

nirandaperera commented Jun 3, 2021

jorisvandenbossche commented Jun 3, 2021

lidavidm commented Jun 3, 2021

lidavidm left a comment

	/// \brief Allocate buffer for bitmap from the context's memory pool. Like
	/// Allocate, the contents of the buffer are not initialized but the last
	/// byte is preemptively zeroed to help avoid ASAN or valgrind issues.
	Result<std::shared_ptr<ResizableBuffer>> AllocateBitmap(int64_t num_bits);

ARROW-10640: [C++] An "if_else" kernel to combine two arrays based on a mask #10410

ARROW-10640: [C++] An "if_else" kernel to combine two arrays based on a mask #10410

Conversation

nirandaperera commented May 26, 2021

github-actions bot commented May 26, 2021

nirandaperera commented May 26, 2021

lidavidm left a comment

Choose a reason for hiding this comment

lidavidm May 27, 2021

Choose a reason for hiding this comment

nirandaperera May 27, 2021

Choose a reason for hiding this comment

lidavidm May 27, 2021

Choose a reason for hiding this comment

bkietz May 27, 2021

Choose a reason for hiding this comment

lidavidm May 27, 2021

Choose a reason for hiding this comment

bkietz Jun 1, 2021

Choose a reason for hiding this comment

nirandaperera commented Jun 1, 2021

nirandaperera commented Jun 1, 2021 • edited Loading

bkietz left a comment

Choose a reason for hiding this comment

bkietz Jun 1, 2021

Choose a reason for hiding this comment

jorisvandenbossche commented Jun 2, 2021

nirandaperera commented Jun 2, 2021

bkietz left a comment

Choose a reason for hiding this comment

lidavidm commented Jun 2, 2021

nirandaperera commented Jun 3, 2021

jorisvandenbossche commented Jun 3, 2021

lidavidm commented Jun 3, 2021

lidavidm left a comment

Choose a reason for hiding this comment

nirandaperera commented Jun 1, 2021 •

edited

Loading