C stored function works different when called locally and remotely

[Gerold103]

Example. Run this from tarantool sources root folder:

box.cfg{listen = 3313}
box.schema.user.grant('guest', 'super')
build_path = '/Users/gerold/Work/Repositories/tarantool'
package.cpath = build_path..'/test/box/?.so;'..build_path..'/test/box/?.dylib;'..package.cpath

box.schema.func.create('function1.multireturn', {language = "C", exports = {'LUA'}})

c = require('net.box').connect(box.cfg.listen)
c:call('function1.multireturn')
---
- [[1], [1]]
...
box.func['function1.multireturn']:call()
---
- [1]
- [1]
...

Result of the function depends on from where it was called. When I call a Lua function, it returns the same result regardless of how I call it.
I don't know what a behaviour is correct - return an array in both cases, or multireturn. Seems like C functions always returned an array, because there was no other way to call them except via netbox, until box.func was introduced. I don't know. But behaviour should be at least the same.

[Korablev77]

It does not concern only C stored functions - Lua ones show the same bevaiour. Setup:

box.cfg{listen = 3301}
nb = require('net.box')
cn = nb:connect(3301)
box.execute('create table t(id int primary key)')
box.space.T:insert({1})
box.space.T:insert({2})
box.space.T:insert({3})
function foo() return 1, 2, 3 end
function select() return box.space.T:select() end
function replace() return box.space.T:replace({3}) end
box.schema.user.grant('guest', 'super')

Local calls:

tarantool> foo()
---
- 1
- 2
- 3
...

tarantool> select()
---
- - [1]
  - [2]
  - [3]
...

tarantool> replace()
---
- [3]
...

Remote calls:

tarantool> cn:call('select')
---
- [[1], [2], [3]]
...

tarantool> cn:call('replace')
---
- [3]
...

tarantool> cn:call('foo')
---
- 1
- 2
- 3
...
tarantool> cn.space.T:select()
---
- - [1]
  - [2]
  - [3]
...

As one can see, result for select() different for local and remote calls. However, there's special (legacy) call16, which emulates local behaviour:

tarantool> cn:call_16('select')
---
- - [1]
  - [2]
  - [3]
...

I have no idea why current call replaced call_16, but it looks like we shouldn't touch it...

[Gerold103]

No, I am not talking about calling Lua functions as variables. I am talking about working with registered functions using box API: box.func. If you will call your examples using box.func.foo/box.func.select/..., you will see it works like netbox call. But if the function is defined in C, it works different. This is inconsistency, because it should not depend on the language.

[NeraverinTarantool]

We will check this issue after #7000

[gmoshkin]

I had to investigate this behavior twice now, so I'll just keep a record of what I found here:

Implementation details

struct port

When the stored procedure is called if it has any return values those are stored
in a struct port helper struct which has different implementation for
different types of stored procedures.

LUA stored procedures have struct port_lua which stores a reference to a lua
state where the return values are stored on the stack.

FFI stored procedures have struct port_c which contains a linked list of
entries each of which stores a msgpack value.

And to extract values from that port a table of virtual methods is used with
different methods for different contexts.

box.schema.func.call

In case when the proc is called via box.schema.func.call (see lbox_func_call
in box/lua/call.c), the port.port_vtab->dump_lua is called, which pushes the
return values onto the lua stack of the caller. So in case of port_lua the
values are simply copied from one stack to another (see port_lua_dump_lua) and
in case of port_c the msgpack values from the linked list are decoded and
pushed to the stack one after another (see port_c_dump_lua).

IPROTO_CALL

When the proc is called via IPROTO_CALL (see tx_process_call), the result is
encoded as a IPROTO_RESPONSE, which consists of an IPROTO_HEADER (see
iproto_header_encode) and a body which is just a msgpack map with key
IRPOTO_DATA (0x30) and value -- a msgpack array. This msgpack array is
generated by calling port.port_vtab->dump_msgpack which encodes the elements
of the array and returns count the length of the array.

So in case of a LUA stored proc (see port_lua_dump -> encode_lua_call) the
return values are encoded to msgpack one after another and the resulting
count is the number of lua values returned.

But in case of an FFI stored proc (see port_c_dump_msgpack) the entries of
port_c are encoded into a msgpack array and the resulting count is
explicitly 1.

This effectively means that when calling an FFI stored proc the
result values are always explicitly wrapped into a msgpack array of length 1.