[C++] Seg. fault on make_shared<PoolBuffer>

[asfimport]

When creating a PoolBuffer, I get a Segmentation fault when I use make_shared. If I use the shared_ptr constructor of reset, it works fine. Here is an example:

#include <arrow/builder.h>

int main()
{
    arrow::MemoryPool* pool = arrow::default_memory_pool();

    arrow::Int64Builder builder(pool);
    builder.Append(1);

    // #1
    // std::shared_ptr<arrow::PoolBuffer> buffer(new arrow::PoolBuffer(pool));
    // #2
    // std::shared_ptr<arrow::PoolBuffer> buffer;
    // buffer.reset(new arrow::PoolBuffer(pool));
    // #3
    auto buffer = std::make_shared<arrow::PoolBuffer>(pool);
}

> g++-4.9 -std=c++11 -larrow foo.cpp && ./a.out
Segmentation fault (core dumped)

The example works fine with #1 or #2 options. It also works if the builder is commented out.

Environment: Debian jessie in a Docker container
libarrow-dev 0.8.0-2 (Ubuntu trusty)

Reporter: Rares Vernica / @rvernica
Assignee: Wes McKinney / @wesm

Original Issue Attachments:

• Dockerfile
• foo.cpp

_{Note: This issue was originally created as ARROW-2189. Please see the migration documentation for further details.}

[asfimport]

Phillip Cloud / @cpcloud:
Can you run gdb ./a.out and paste the stack trace here? I can't reproduce this.

[asfimport]

Phillip Cloud / @cpcloud:
Or, do you have a link to the Dockerfile or the image up in a docker registry?

[asfimport]

Rares Vernica / @rvernica:
Does this help?

> g++-4.9 -ggdb -std=c++11 -larrow foo.cpp     
> strace ./a.out 
strace: test_ptrace_setoptions_for_all: PTRACE_TRACEME doesn't work: Operation not permitted
strace: test_ptrace_setoptions_for_all: unexpected exit status 1
> gdb ./a.out 
GNU gdb (Debian 7.7.1+dfsg-5) 7.7.1
Copyright (C) 2014 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from ./a.out...done.
(gdb) run
Starting program: /a.out 
warning: Error disabling address space randomization: Operation not permitted
During startup program terminated with signal SIGSEGV, Segmentation fault.
(gdb) strace 
No default breakpoint address now.
(gdb) backtrace 
No stack.

Otherwise I can get you a Dockerfile.

I see a bunch of SELinux errors on the host, which is a Fedora 27, every time this crushes.

[asfimport]

Wes McKinney / @wesm:
Seems like this could be a mixed C++ standard library toolchain issue – @kou do you know what the policy is for using Trusty packages on Debian Jessie platform?

[asfimport]

Kouhei Sutou / @kou:
You can't use Ubuntu Trusty packages on Debian Jessie.

Can you try to use Debian Stretch or Ubuntu Trusty? They have packages for their platform.

[asfimport]

Rares Vernica / @rvernica:
It works fine with Debian Stretch or Ubuntu Trusty. It seems to be a package/toolchain mixing issues. Sorry for the trouble.

[asfimport]

Rares Vernica / @rvernica:
It works fine in Ubuntu Trusty if I use the default g++ compiler to compile this example.

On the other hand, I still get the Segmentation Fault if I use the g++-4.9 compiler from ppa:ubuntu-toolchain-r/test.

Is it because arrow is compiled with g++ and I compile this example with g++-4.9?

[asfimport]

Wes McKinney / @wesm:
Let's set up a Dockerfile so we can try to reproduce

[asfimport]

Rares Vernica / @rvernica:
See the attached Dokerfile and foo.cpp. You should be able to reproduce with:

> docker build .
...
Step 18/18 : RUN /a.out
---> Running in de9d8736cf20
Segmentation fault (core dumped)
The command '/bin/sh -c /a.out' returned a non-zero code: 139

[asfimport]

Rares Vernica / @rvernica:
@kou could this issue be caused by the differences in the rules file used to build Ubuntu Trusty packages vs. other Debian packages?

> diff debian/rules debian.ubuntu-trusty/rules
12,16c12
< 	dh $@ --with gir,autoreconf
< 
< override_dh_autoreconf:
< 	dh_autoreconf \
< 	  --sourcedirectory=c_glib
---
> 	dh $@ --with gir
25,30c21
< 	  -DARROW_PYTHON=ON \
< 	  -DARROW_BOOST_USE_SHARED=ON \
< 	  -DARROW_ORC=ON \
< 	  -DPythonInterp_FIND_VERSION=ON \
< 	  -DPythonInterp_FIND_VERSION_MAJOR=3 \
< 	  -DARROW_GPU=ON
---
> 	  -DARROW_ORC=ON
56a48,49
> 	mv debian/tmp/usr/lib/*/girepository-1.0/	\
> 	  debian/tmp/usr/lib/

 

[asfimport]

Rares Vernica / @rvernica:
I did some tests and it does not look like these differences help with the issue.

On the other hand, I compiled Arrow using g++-4.9 and the issue is gone.

[asfimport]

Kouhei Sutou / @kou:
Thanks. I confirmed it.

I also confirmed the followings:

  * Apache Arrow C++ compiled with g+-4.8 + -DCMAKE_BUILD_TYPE=Debug and foo.cpp compiled with g+-4.9 work well

  * Apache Arrow C++ compiled with g+-4.8 + -DCMAKE_BUILD_TYPE=Release and foo.cpp compiled with g+-4.9 don't work (crashed)

 Backtrace:

Program received signal SIGSEGV, Segmentation fault.
0x0000000000000000 in ?? ()
(gdb) bt full
#0  0x0000000000000000 in ?? ()
No symbol table info available.
#1  0x00007ffff7941490 in arrow::ArrayBuilder::Init (
    this=this@entry=0x7fffffffe640, capacity=capacity@entry=32)
    at /arrow/cpp/src/arrow/builder.cc:67
        _s = {state_ = 0x20}
        to_alloc = 4
        byte_capacity = <optimized out>
#2  0x00007ffff79584e0 in arrow::PrimitiveBuilder<arrow::Int64Type>::Init (
    this=0x7fffffffe640, capacity=32) at /arrow/cpp/src/arrow/builder.cc:233
        _s = {state_ = 0x20}
        nbytes = <optimized out>
#3  0x00007ffff7948b58 in arrow::PrimitiveBuilder<arrow::Int64Type>::Resize (
    this=0x7fffffffe640, capacity=32) at /arrow/cpp/src/arrow/builder.cc:253
        _s = {state_ = 0x1}
#4  0x00007ffff793e7ab in arrow::ArrayBuilder::Reserve (
    this=this@entry=0x7fffffffe640, elements=elements@entry=1)
    at /arrow/cpp/src/arrow/builder.cc:113
No locals.
#5  0x000000000040168f in Append (val=1, this=0x7fffffffe640)
    at /tmp/local-4.8/include/arrow/builder.h:276
        _s = {state_ = 0x616848}
#6  main () at foo.cpp:10
        pool = 
    0x7ffff7dd91a0 <arrow::default_memory_pool()::default_memory_pool_>
        builder = {<arrow::PrimitiveBuilder<arrow::Int64Type>> = {<arrow::ArrayBuilder> = {
              _vptr.ArrayBuilder = 0x4024f0 <vtable for arrow::NumericBuilder<arrow::Int64Type>+16>, type_ = std::shared_ptr (count 2, weak 0) 0x616848, 
              pool_ = 0x7ffff7dd91a0 <arrow::default_memory_pool()::default_memory_pool_>, null_bitmap_ = std::shared_ptr (count 1, weak 0) 0x616890, 
              null_count_ = 0, null_bitmap_data_ = 0x0, length_ = 0, 
              capacity_ = 0, children_ = std::vector of length 0, capacity 0}, 
            data_ = std::shared_ptr (empty) 0x0, 
            raw_data_ = 0x0}, <No data fields>}
        buffer = <optimized out>
(gdb) f 1
#1  0x00007ffff7941490 in arrow::ArrayBuilder::Init (
    this=this@entry=0x7fffffffe640, capacity=capacity@entry=32)
    at /arrow/cpp/src/arrow/builder.cc:67
67	  RETURN_NOT_OK(null_bitmap_->Resize(to_alloc));
(gdb) list
62	}
63	
64	Status ArrayBuilder::Init(int64_t capacity) {
65	  int64_t to_alloc = BitUtil::CeilByte(capacity) / 8;
66	  null_bitmap_ = std::make_shared<PoolBuffer>(pool_);
67	  RETURN_NOT_OK(null_bitmap_->Resize(to_alloc));
68	  // Buffers might allocate more then necessary to satisfy padding requirements
69	  const int64_t byte_capacity = null_bitmap_->capacity();
70	  capacity_ = capacity;
71	  null_bitmap_data_ = null_bitmap_->mutable_data();
(gdb) p to_alloc
$3 = 4

 I don't know why...

BTW, why do you want to use g++-4.9?

[asfimport]

Rares Vernica / @rvernica:
I'm linking Arrow to a "plugin" which I load in SciDB. SciDB is complied with g++-4.9.

I'm also trying to use Ubuntu Trusty packages on Debian Jessie (I know this is not advised) but the problem disappears if I compile Arrow with g++ 4.9.

[asfimport]

Wes McKinney / @wesm:
PoolBuffer is no longer a public API per ARROW-2893. Use the arrow::Allocate* APIs instead