Plugin API

THIS PAGE is outdated, please consult https://gist.github.com/rtsisyk/aa95cf9ed9bbb538ff80

Tarantool Plugin API

In Tarantool, in addition to Lua stored procedures, it's possible to write C plugins. These plugins have full access to server core and add features not present in the stock version. For example, they make it possible to implement a custom client/server protocol or turn Tarantool into an HTTP server, or connect to another database.

The first two existing plugins are connectors to PostgreSQL and MySQL.

They make it possible to run queries against these databases from within stored procedures. For example, this is what a procedure pulling data from MySQL to a Tarantool space may look like:

local dbh = pg.connect('host', port, user, password, database)

local cards = dbh:select('SELECT * FROM cards WHERE time > ?', box.time())

for i, card in pairs(cards) do
	box.space.mydata:replace(card.id, card.name, card.title)
end

Let's consider the steps necessary to write your own plugin using an example. Imagine Tarantool is used as a statistics store, and it's necessary to publish various counters in a monitoring application once in a while. We must also assume that the protocol to work with this system is an external library (since otherwise it would be simpler to just open a box.socket instance and push a packet into it). Perhaps it's a proprietary system and the protocol is closed.

To sum up, let's assume we have

• an external library, libmonitor
• a library header file, monitor.h

To use the library in a C++ program, one typically writes a program like this:

#include <monitor.h>

struct monitor *mon = mon_connect("host", "auth");

mon_push(mon, value);

mon_close(mon);

Both methods, mon_connect and mon_push are "blocking", i.e. they block the current thread while they execute, so they can't be used directly in Tarantool Lua (if they could be, it would be more appropriate to bind them via LuaJIT FFI). Let's see how the library can be wrapped into a plugin, and its methods used in a non-blocking manner. For simplicity, exceptions and error handling are omitted.

1. Clone the source tree

git clone --recursive git@github.com:mailru/tarantool.git

2. Add plugin directory src/plugin/monitor

3. Add the plugin directory to the build list в src/plugin/CMakeLists.txt:

	add_subdirectory(monitor)

4. Add cmake rules to build the plugin:

(file src/plugin/monitor/CMakeLists.txt):

add_library(monitor SHARED monitor.cc)
install(TARGETS monitor LIBRARY DESTINATION ${PLUGIN_DIR})

The first rule instructs cmake to pack the new plugin into a shared library called libmonitor.so, and to build it using monitor.cc (its source code is given below). The second directive instructs cmake to install the new library into Tarantool plugin directory, from which it will be automatically loaded at server start.

5. Add the plugin source code.

At this point Tarantool does not define a strict plugin API, so the plugin is free to use any part of Tarantool core (as long as it doesn't break it) and any include in the include/ directory. To get loaded correctly, the plugin library needs to define the following:

• plugin name
• plugin version (an integer number)
• plugin load function (there is no unload function at this point since plugins are never unloaded)

A macro DECLARE_PLUGIN is defined in plugin.h to help define these parameters:

extern "C" {
        #include <lua.h>
        #include <lauxlib.h>
        #include <lualib.h>
}

#include <plugin.h>
#include <monitor.h>
#include <coeio.h>
#include <tarantool_ev.h>

static ssize_t
my_mon_connect(va_list ap)
{
	struct monitor **mon = va_arg(ap, typeof(mon));
	const char *host = va_arg(ap, typeof(host));
	const char *auth = va_arg(ap, typeof(auth));
	*mon = mon_connect(host, auth);
	return 0;
}

static ssize_t
my_mon_push(va_list ap)
{
	struct monitor **mon = va_arg(ap, typeof(mon));
	int value = va_arg(ap, typeof(value));
	mon_push(mon, value);
	return 0;
}


static int
lua_mon_send(struct lua_State *L)
{
	struct monitor *mon = *(struct monitor **)lua_touserdata(L, 1);
	int value = lua_tointeger(L, 2);

	coeio_custom(my_mon_push, TIMEOUT_INFINITY, mon, value);
	return 0;

}

static int
lua_mon_close(struct lua_State *L)
{
	struct monitor *mon = *(struct monitor **)lua_touserdata(L, 1);
	mon_close(mon);
}

static int
lua_mon_connect(struct lua_State *L)
{
	const char *host = lua_tostring(L, 1);
	const char *auth = lua_tostring(L, 2);

	struct monitor *mon = NULL;
	coeio_custom(my_mon_connect, TIMEOUT_INFINITY, &mon, host, auth);

	struct monitor **ptr = (struct monitor **)lua_newuserdata(sizeof(mon));
	ptr[0] = mon;

	lua_newtable(L);
	
	/* "destructor" */
	lua_pushstring(L, "__gc");
	lua_pushcfunction(L, lua_mon_close);
	lua_rawset(L, -3);

	/* send data */
	lua_pushstring(L, "__call");
	lua_pushcfunction(L, lua_mon_send);
	lua_rawset(L, -3);

	/* a metatable for established connection */
	lua_setmetatable(L, -2);

	return 1;
}

static void
init (struct lua_State *L)
{
	lua_pushcfunction(L, lua_mon_connect);
	lua_setfield(L, LUA_GLOBALSINDEX, "mon_connect");

}

DECLARE_PLUGIN("monitor", 1, init, NULL);

To sum up:

1. lua*.h headers allow the plugin to access the global Lua interpreter state used in Tarantool (they are described in Lua and LuaJIT manuals).
2. plugin.h is necessary to DECLARE_PLUGIN
3. monitor.h is a header file of the library used to connect to the remote system
4. coeio.h - a header of Tarantool core API, making it possible to use blocking calls from the transaction processor event loop
5. tarantool_ev.h - defines TIMEOUT_INFINITY

The plugin is activated when init function is called, this happens on server start. This function must initialize the plugin internal state. It is given a reference to the instance of the global Lua interpreter as a parameter. This function registers a binding to the C calls the plugin defines in Lua. These bindings will be later used in stored procedures.

	local mon = mon_connect(host, auth) -- connect to the monitor
	mon(123)	-- send a value to the monitor 
	mon(234)	-- send another value

In this example, whenever mon_connect is called in Lua, the plugin function lua_mon_connect is invoked. This function calls coeio_custom, which, in turns, calls mon_connectin a separate thread and yields until connect is finished. The results ofmon_connect()` are converted to Lua object: connection and status. The connection object has two metamethods:

• __gc - destructor, calls mon_close
• __call - calls mon_push

6. Compile the plugin and the plugin host:

	cmake .
	make

Library libmonitor.so will appear in src/plugin/monitor.

7. Notes and comments

• at start, Tarantool looks for plugins in the server library path (usually /usr{/local}/lib/tarantool), as well as in the directory defined in TARANTOOL_PLUGIN_DIR environment library.
• show plugin lists all loaded plugins.