[C++] In some cases ChunkedArray::Equals(std::shared_ptr<arrow::ChunkedArray>& other) does not treat NaN values as unequal consistently

[sgilmore10]

Describe the bug, including details regarding any error messages, version, and platform.

While working on #37448, I noticed the Equals() method of ChunkedArray treats NaNs as equal by default. I found this behavior surprising because the Equals() method of Array doesn't treat NaNs as equal by default.

Below is a program to reproduce this behavior:

#include <arrow/api.h>
#include <arrow/type.h>

#include <iostream>
#include <math.h>
#include <sstream>

arrow::Result<std::shared_ptr<arrow::ChunkedArray>> make_chunked_array() {
    arrow::NumericBuilder<arrow::DoubleType> builder;
    
    std::shared_ptr<arrow::Array> array;
    ARROW_RETURN_NOT_OK(builder.AppendValues({0, 1, NAN, 2, 4}));
    ARROW_RETURN_NOT_OK(builder.Finish(&array));
    
    return arrow::ChunkedArray::Make({array});
}

int main(int argc, char *argv[])
{
    auto maybe_chunked_array = make_chunked_array();
    if (!maybe_chunked_array.ok()) {
        return -1;
    }
    auto chunked_array = std::move(maybe_chunked_array).ValueUnsafe();
    auto array = chunked_array->chunk(0);
    
    std::stringstream stream;
    stream << "array contents: ";
    stream << "\n\n";
    stream << array->ToString();
    stream << "\n\n";
    stream << "array->Equals(array): ";
    stream << array->Equals(array);
    stream << "\n\n";

    stream << "chunked_array contents: ";
    stream << "\n\n";
    stream << chunked_array->ToString();
    stream << "\n\n";
    stream << "chunked_array->Equals(chunked_array): ";
    stream << chunked_array->Equals(chunked_array);
    
    std::cout << stream.str() << std::endl;
}

Here is the output of this program:

array contents: 

[
  0,
  1,
  nan,
  2,
  4
]

array->Equals(array): 0

chunked_array contents: 

[
  [
    0,
    1,
    nan,
    2,
    4
  ]
]

chunked_array->Equals(chunked_array): 1

Am I correct in that this is a bug?

Component(s)

C++

[js8544]

It doesn't treat NaNs as equal. But it does assumes that chunked arrays at the same address are equal, where Array doesn't have such assumption. See https://github.com/apache/arrow/blob/main/cpp/src/arrow/chunked_array.cc#L115

[sgilmore10]

Ah, thanks for pointing that out! It's a bit weird that Equals(ChunkedArray& other) and Equals(std::shared_ptr<ChunkedArray>& other) may return different results. For example, if you modified the program above to also call Equals(ChunkedArray& other), this would be the output:

chunked_array->Equals(chunked_array): 1
chunked_array->Equals(*chunked_array): 0

I wonder if it's worth removing this optimization for consistency? Or at least removing it for datatypes that can have NaNs?

[js8544]

I agree that this optimization can be removed for the sake of consistency.

[sgilmore10]

I'll work on this.

[sgilmore10]

take