[C++] MinGW Flight tests failing

[asfimport]

Noticed across several PRs

[ RUN      ] GrpcDataTest.TestDoExchangeError
D:/a/arrow/arrow/cpp/src/arrow/flight/test_definitions.cc:490: Failure
Value of: _st.IsNotImplemented()
  Actual: false
Expected: true
Expected 'writer->Close()' to fail with NotImplemented, but got IOError: Stream finished before first message sent. gRPC client debug context: UNKNOWN:Error received from peer ipv4:127.0.0.1:52323 {created_time:"2022-07-23T01:21:23.785644223+00:00", grpc_status:2, grpc_message:"Stream finished before first message sent"}. Client context: OK. Detail: Failed
D:/a/arrow/arrow/cpp/src/arrow/flight/test_definitions.cc:490: Failure
Value of: _st.ToString()
Expected: has substring "Expected error"
  Actual: "IOError: Stream finished before first message sent. gRPC client debug context: UNKNOWN:Error received from peer ipv4:127.0.0.1:52323 {created_time:\"2022-07-23T01:21:23.785644223+00:00\", grpc_status:2, grpc_message:\"Stream finished before first message sent\"}. Client context: OK. Detail: Failed"
[  FAILED  ] GrpcDataTest.TestDoExchangeError (5 ms)
[ RUN      ] GrpcDataTest.TestDoExchangeConcurrency
[       OK ] GrpcDataTest.TestDoExchangeConcurrency (5 ms)
[ RUN      ] GrpcDataTest.TestDoExchangeUndrained
[       OK ] GrpcDataTest.TestDoExchangeUndrained (4 ms)
[ RUN      ] GrpcDataTest.TestIssue5095
[       OK ] GrpcDataTest.TestIssue5095 (9 ms)
[----------] 17 tests from GrpcDataTest (891 ms total)
[----------] 7 tests from GrpcDoPutTest
[ RUN      ] GrpcDoPutTest.TestInts
D:/a/arrow/arrow/cpp/src/arrow/flight/test_definitions.cc:690: Failure
Failed
'writer->Close()' failed with Invalid: Expected app_metadata to be foo bar but got \0L���. gRPC client debug context: UNKNOWN:Error received from peer ipv4:127.0.0.1:52331 {grpc_message:"Expected app_metadata to be foo bar but got \x00L\xf4\x86\x02\xe0\xa1", grpc_status:3, created_time:"2022-07-23T01:21:23.810734286+00:00"}. Client context: OK
[  FAILED  ] GrpcDoPutTest.TestInts (4 ms)
[ RUN      ] GrpcDoPutTest.TestFloats
D:/a/arrow/arrow/cpp/src/arrow/flight/test_definitions.cc:690: Failure
Failed
'writer->Close()' failed with Invalid: Expected app_metadata to be foo bar but got \0<����. gRPC client debug context: UNKNOWN:Error received from peer ipv4:127.0.0.1:52333 {grpc_message:"Expected app_metadata to be foo bar but got \x00<\xee\xc6\x02\xe0\xa1", grpc_status:3, created_time:"2022-07-23T01:21:23.815439591+00:00"}. Client context: OK
[  FAILED  ] GrpcDoPutTest.TestFloats (4 ms)
[ RUN      ] GrpcDoPutTest.TestEmptyBatch
D:/a/arrow/arrow/cpp/src/arrow/flight/test_definitions.cc:690: Failure
Failed
'writer->Close()' failed with Invalid: Expected app_metadata to be foo bar but got \0����. gRPC client debug context: UNKNOWN:Error received from peer ipv4:127.0.0.1:52335 {grpc_message:"Expected app_metadata to be foo bar but got \x00\x9c\xef\xa6\x02\xe0\xa1", grpc_status:3, created_time:"2022-07-23T01:21:23.819872813+00:00"}. Client context: OK
[  FAILED  ] GrpcDoPutTest.TestEmptyBatch (4 ms)
[ RUN      ] GrpcDoPutTest.TestDicts
D:/a/arrow/arrow/cpp/src/arrow/flight/test_definitions.cc:690: Failure
Failed
'writer->Close()' failed with Invalid: Expected app_metadata to be foo bar but got \0\����. gRPC client debug context: UNKNOWN:Error received from peer ipv4:127.0.0.1:52337 {grpc_message:"Expected app_metadata to be foo bar but got \x00\\\xf0\xc6\x02\xe0\xa1", grpc_status:3, created_time:"2022-07-23T01:21:23.824172893+00:00"}. Client context: OK
[  FAILED  ] GrpcDoPutTest.TestDicts (4 ms)
[ RUN      ] GrpcDoPutTest.TestLargeBatch
D:/a/arrow/arrow/cpp/src/arrow/flight/test_definitions.cc:690: Failure
Failed
'writer->Close()' failed with Invalid: Expected app_metadata to be foo bar but got \0|���. gRPC client debug context: UNKNOWN:Error received from peer ipv4:127.0.0.1:52339 {created_time:"2022-07-23T01:21:24.001437714+00:00", grpc_status:3, grpc_message:"Expected app_metadata to be foo bar but got \x00|\xf2\xa6\x02\xe0\xa1"}. Client context: OK
[  FAILED  ] GrpcDoPutTest.TestLargeBatch (185 ms)
[ RUN      ] GrpcDoPutTest.TestSizeLimit
D:/a/arrow/arrow/cpp/src/arrow/flight/test_definitions.cc:802: Failure
Failed
'writer->Close()' failed with Invalid: Expected app_metadata to be foo bar but got \0\�,��. gRPC client debug context: UNKNOWN:Error received from peer ipv4:127.0.0.1:52341 {grpc_message:"Expected app_metadata to be foo bar but got \x00\\\xef,\x07\xe0\xa1", grpc_status:3, created_time:"2022-07-23T01:21:24.016917836+00:00"}. Client context: OK
[  FAILED  ] GrpcDoPutTest.TestSizeLimit (8 ms)
[ RUN      ] GrpcDoPutTest.TestUndrained
D:/a/arrow/arrow/cpp/src/arrow/flight/test_definitions.cc:690: Failure
Failed
'writer->Close()' failed with Invalid: Expected app_metadata to be foo bar but got \0����. gRPC client debug context: UNKNOWN:Error received from peer ipv4:127.0.0.1:52343 {created_time:"2022-07-23T01:21:24.022330532+00:00", grpc_status:3, grpc_message:"Expected app_metadata to be foo bar but got \x00\xac\xee\xa6\x02\xe0\xa1"}. Client context: OK
[  FAILED  ] GrpcDoPutTest.TestUndrained (4 ms) 

Reporter: David Li / @lidavidm
Assignee: David Li / @lidavidm

PRs and other links:

• GitHub Pull Request #13696

_{Note: This issue was originally created as ARROW-17191. Please see the migration documentation for further details.}

[asfimport]

David Li / @lidavidm:
What I currently see: calling make_shared and passing the existing shared_ptr<Buffer> to slice a buffer also overwrites the data in the buffer, since the newly allocated buffer (or shared_ptr control structure?) overlaps the allocated gRPC buffer. The reason is unclear…

[asfimport]

David Li / @lidavidm:
The reason is that: sometimes (most times!) gRPC hands us a noncontiguous set of slices, which we have to concatenate into a single memory region to use with Flight. We use a gRPC function to do this. Also, slices can inline the data into the structure directly for small slices. It turns out that after concatenation, we can still get an inlined slice, which we weren't properly accounting for. And then the inlined slice would have its data overwritten when we sliced the buffer. I'm unclear why this is the case, though.

[asfimport]

David Li / @lidavidm:

Thread 8 hit Watchpoint 2: *(uint8_t*) 0x509bfe569Old value = 26 '\032'
New value = 169 '▒'
0x00007ffa24604f88 in ?? () from C:\msys64\mingw64\bin\libstdc++-6.dll
(gdb) bt
#0  0x00007ffa24604f88 in ?? () from C:\msys64\mingw64\bin\libstdc++-6.dll
#1  0x00007ffa2470a95c in ?? () from C:\msys64\mingw64\bin\libstdc++-6.dll
#2  0x00007ff9dfb09f7d in std::__new_allocator<std::_Sp_counted_ptr_inplace<arrow::Buffer, std::allocator<void>, (__gnu_cxx::_Lock_policy)2> >::allocate (
    this=0x509bfe67f, __n=1)
    at C:/msys64/mingw64/include/c++/12.1.0/bits/new_allocator.h:137
#3  0x00007ff9dfb10eb6 in std::allocator_traits<std::allocator<std::_Sp_counted_ptr_inplace<arrow::Buffer, std::allocator<void>, (__gnu_cxx::_Lock_policy)2> > >::allocate (__a=..., __n=1)
    at C:/msys64/mingw64/include/c++/12.1.0/bits/alloc_traits.h:464
#4  0x00007ff9dfb30691 in std::__allocate_guarded<std::allocator<std::_Sp_counted_ptr_inplace<arrow::Buffer, std::allocator<void>, (__gnu_cxx::_Lock_policy)2> > > (__a=...) at C:/msys64/mingw64/include/c++/12.1.0/bits/allocated_ptr.h:98
#5  0x00007ff9dfb055a3 in std::__shared_count<(__gnu_cxx::_Lock_policy)2>::__shared_count<arrow::Buffer, std::allocator<void>, std::shared_ptr<arrow::Buffer> const&, long long const&, long long const&> (this=0x509bfe848,
    __p=@0x509bfe840: 0x0, __a=...)
    at C:/msys64/mingw64/include/c++/12.1.0/bits/shared_ptr_base.h:969
#6  0x00007ff9dfafc2e0 in std::__shared_ptr<arrow::Buffer, (__gnu_cxx::_Lock_policy)2>::__shared_ptr<std::allocator<void>, std::shared_ptr<arrow::Buffer> const&, long long const&, long long const&> (this=0x509bfe840, __tag=...)
    at C:/msys64/mingw64/include/c++/12.1.0/bits/shared_ptr_base.h:1712
#7  0x00007ff9dfaefb38 in std::shared_ptr<arrow::Buffer>::shared_ptr<std::allocator<void>, std::shared_ptr<arrow::Buffer> const&, long long const&, long long const&> (this=0x509bfe840, __tag=...)
    at C:/msys64/mingw64/include/c++/12.1.0/bits/shared_ptr.h:464
#8  0x00007ff9dfb2bc5a in std::make_shared<arrow::Buffer, std::shared_ptr<arrow::Buffer> const&, long long const&, long long const&> ()
    at C:/msys64/mingw64/include/c++/12.1.0/bits/shared_ptr.h:1010
#9  0x00007ff9dfa79401 in arrow::SliceBuffer (
    buffer=std::shared_ptr<arrow::Buffer> (use count 1, weak count 0) = {...}, offset=2, length=7) at C:/msys64/home/User/arrow/cpp/src/arrow/buffer.h:324
#10 0x00007ff9dfa7946a in arrow::flight::transport::grpc::ReadBytesZeroCopy (
    source_data=std::shared_ptr<arrow::Buffer> (use count 1, weak count 0) = {...}, input=0x509bfe960, out=0x1ace67a3020)
    at C:/msys64/home/User/arrow/cpp/src/arrow/flight/transport/grpc/serialization_internal.cc:83
#11 0x00007ff9dfa7ac8f in arrow::flight::transport::grpc::FlightDataDeserialize (buffer=0x509bfed58, out=0x1ace67a3008)
    at C:/msys64/home/User/arrow/cpp/src/arrow/flight/transport/grpc/serialization_internal.cc:366
#12 0x00007ff9dfa943fb in grpc::SerializationTraits<arrow::flight::protocol::FlightData, void>::Deserialize (buffer=0x509bfed58, msg=0x1ace67a3008)
    at C:/msys64/home/User/arrow/cpp/src/arrow/flight/transport/grpc/customize_grpc.h:102
#13 0x00007ff9dfa9a651 in grpc::internal::CallOpRecvMessage<arrow::flight::protocol::FlightData>::FinishOp (this=0x509bfed48, status=0x509bfecef)
    at C:/msys64/mingw64/include/grpcpp/impl/codegen/call_op_set.h:456
 

[asfimport]

Kouhei Sutou / @kou:
Issue resolved by pull request 13696
#13696