Stream data easily- function to convert record batch to streamable format

[asfimport]

I am currently working on a project that  need to send the data in different processes. Instead of storing it in a file, I am using the IPC stream method, storing it into a buffer. When I try to send the data(bytes) through a socket, I loose the data in the client side. ( Which gives me the idea that when i leave the function(create buffer in the server side folder), I loose acces to that data. Can there be an example that explains this.

Also, can you implement a function that convert the record batch into the format that can be sent through the socket directly.

 

The code are in the attachment, one for the server side code and the other for the client side code.

Thank you

Environment: Linux
Reporter: Steven

Original Issue Attachments:

• code.zip

_{Note: This issue was originally created as ARROW-8719. Please see the migration documentation for further details.}

[asfimport]

Wes McKinney / @wesm:
Just looking at the client code

	int sock;
	sock = socket(AF_INET, SOCK_STREAM, 0);

	//setup an address
  struct sockaddr_in server_address;
	server_address.sin_family = AF_INET;
	server_address.sin_port = htons(9002);
	server_address.sin_addr.s_addr = INADDR_ANY;

	connect(sock, (struct sockaddr *) &server_address, sizeof(server_address));

	//send(sock, request, sizeof(request), 0);

	GArrowResizableBuffer *data = NULL;
	recv(sock, &data, garrow_buffer_get_size(GARROW_BUFFER(data)), 0);

	gsize data_size = sizeof(data);
	display_arrow_buffer(GARROW_BUFFER(data), data_size);

These lines won't do what you want:

GArrowResizableBuffer *data = NULL;
recv(sock, &data, garrow_buffer_get_size(GARROW_BUFFER(data)), 0);

You need to allocate memory before you use recv to write data into it. Consider using garrow_resizable_buffer_new (you will need to send a buffer size or something from the server so the client knows how much memory to allocate)

[asfimport]

Wes McKinney / @wesm:
I'm inclined to close this issue because we don't do user support on JIRA, can you please write to user@arrow.apache.org instead?