Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

c++ use python to import reverb error #127

Open
jzstudent opened this issue Apr 6, 2023 · 0 comments
Open

c++ use python to import reverb error #127

jzstudent opened this issue Apr 6, 2023 · 0 comments

Comments

@jzstudent
Copy link

jzstudent commented Apr 6, 2023

python:

import traceback
import reverb
class RevervbUtil(object):
    # str,int,[str],int,int
    def init(self, learner_addr, learner_port, reverb_table_names,
             reverb_client_max_sequence_length, reverb_client_chunk_length):
        self.revervb_util = revervbutil(f'{learner_addr}:{learner_port}')
        self.reverb_table_names = reverb_table_names
        self.reverb_client_max_sequence_length = reverb_client_max_sequence_length
        self.reverb_client_chunk_length = reverb_client_chunk_length
    def send_datas(self, train_data):
        self.revervb_util.write_to_reverb_server_simple(self.reverb_table_names, train_data)

class revervbutil():
    def __init__(self, learner_addr) -> None:
        self.reverb_client = reverb.Client(learner_addr)
    def write_to_reverb_server_simple(self, reverb_table_names, train_data):
        try: 
            with self.reverb_client.writer(max_sequence_length=int(self.reverb_client_max_sequence_length), 
                                               chunk_length=int(self.reverb_client_chunk_length)) as writer:
                count = 0
                for sample in train_data:
                    for table in reverb_table_names:
                        writer.append(sample)
                        prioritezed = 1.0      
                        writer.create_item(table, 1, prioritezed)
                        writer.flush()
                    count += 1
        except Exception as e:
            print(f'learner_proxy send one data to reverb server error as {str(e)}, traceback.print_exc() is {traceback.format_exc()}')

c++:

#include "learner_proxy.h"

#include <python3.7m/Python.h>

using namespace std;

int main(int argc, char* argv[])
{
    //py::scoped_interpreter guard{}; 
    Py_Initialize();
    if(!Py_IsInitialized())
    {
        return -1;
    }

    PyRun_SimpleString("import sys");
        PyRun_SimpleString("sys.path.append('/data/projects/kaiwu-fwk/framework/server/cpp/src/aisrv/')");

    PyObject* pModule = PyImport_ImportModule("python_reverb_util");

    PyObject* pDict = PyModule_GetDict(pModule);
    PyObject* pClass = PyDict_GetItemString(pDict, "RevervbUtil");
    PyObject* pConstruct = PyInstanceMethod_New(pClass);
    PyObject* pInstanceReverb = PyObject_CallObject(pConstruct, NULL);
    
    return 0;
}

when I execute it, segment falut happens:

(gdb) bt
#0  0x00007fed93c8eb1a in tensorflow::kernel_factory::OpKernelRegistrar::InitInternal(tensorflow::KernelDef const*, absl::lts_2020_02_25::string_view, std::unique_ptr<tensorflow::kernel_factory::OpKernelFactory, std::default_delete<tensorflow::kernel_factory::OpKernelFactory> >) ()
   from /usr/local/python-3.7/lib/python3.7/site-packages/tensorflow/python/../libtensorflow_framework.so.2
#1  0x00007fed97835395 in tensorflow::kernel_factory::OpKernelRegistrar::OpKernelRegistrar(tensorflow::KernelDef const*, absl::lts_2020_02_25::string_view, tensorflow::OpKernel* (*)(tensorflow::OpKernelConstruction*)) ()
   from /usr/local/python-3.7/lib/python3.7/site-packages/tensorflow/python/_pywrap_tensorflow_internal.so
#2  0x00007fed935a0d4f in _GLOBAL__sub_I_dataset.cc ()
   from /usr/local/python-3.7/lib/python3.7/site-packages/tensorflow/python/../libtensorflow_framework.so.2

Butttt, I can run the python file in python environment.

if __name__ == '__main__':
    revervb_util = RevervbUtil('127.0.0.1:9999', 1, 1)
    train_data = []
    revervb_util.write_to_reverb_server_simple('a', train_data)

How can I fix it? :>

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant