[Python] Memory Issue passing table from python to c++ via cython

[asfimport]

I wanted to create a simple example of reading a table in Python and pass it to C+, but I'm doing something wrong or there is a memory issue. When the table gets to C+ and I print out column names it also prints out a lot of junk and what looks like pydocs. Let me know if you need any more info. Thanks!

demo.py

import numpy
from psy.automl import cyth
import pandas as pd
from absl import app

def main(argv):
  sup = pd.DataFrame({
  'int': [1, 2],
  'str': ['a', 'b']
  })
  table = pa.Table.from_pandas(sup)
  cyth.c_t(table)

cyth.pyx

import pandas as pd
import pyarrow as pa
from pyarrow.lib cimport *

cdef extern from "cyth.h" namespace "psy":
 void t(shared_ptr[CTable])

def c_t(obj):
# These print work
# for i in range(obj.num_columns):
# print(obj.column(i).name
  cdef shared_ptr[CTable] tbl = pyarrow_unwrap_table(obj)
  t(tbl)

cyth.h

#include <iostream>
#include <string>
#include "arrow/api.h"
#include "arrow/python/api.h"
#include "Python.h"

namespace psy {

void t(std::shared_ptr<arrow::Table> pytable) {

// This works
  std::cout << "NUM" << pytable->num_columns();

// This prints a lot of garbage
  for(int i = 0; i < pytable->num_columns(); i++) {
  std::cout << pytable->column(i)->name();
  }
 }
}

 

Environment: clang6
Reporter: Joseph Toth
Assignee: Antoine Pitrou / @pitrou

Related issues:

• [Python] Incorrect example for Cython usage (supercedes)

PRs and other links:

• GitHub Pull Request #2276

_{Note: This issue was originally created as ARROW-2787. Please see the migration documentation for further details.}

[asfimport]

Antoine Pitrou / @pitrou:
(fixed code snippet formatting)

[asfimport]

Antoine Pitrou / @pitrou:
I can reproduce with clang 6.0.1 (though not with gcc 4.9).

[asfimport]

Antoine Pitrou / @pitrou:
Ok, I see, this is because we build the Linux binaries with -D_GLIBCXX_USE_CXX11_ABI=0, so you must pass that flag when building your Cython extension as well. It kind of sucks that it crashes weirdly instead of failing to compile if you don't pass it, though.

[asfimport]

Antoine Pitrou / @pitrou:
Issue resolved by pull request 2276
#2276