Plugin Architecture

[novabyte]

Implement a collection of plugin hooks which are executed at various stages in the socket pipeline which can be used to trigger behaviour in custom plugins. These plugins must be written in Go and will be loaded at server start with their configurations. This can be used to reduce the number of SDKs required in game clients for analytics, Ads, dynamic IAP prices, etc.

[novabyte]

I've put together an initial outline on what the plugin API will look like:

https://github.com/heroiclabs/nakama-plugins/blob/master/example/example.go

It'll be a unidirectional events API which can be used to stream information into other services like analytics and backoffice support tools. The plugin system will work as follows:

1. A plugin worker pool will be created at server start.
2. Each plugin will be read from the datadir and loaded.
3. The plugins will be "scanned" so only events they're registered to listen on will be sent.
4. All plugins will have their Start function called. Plugins are named and will be loaded by name lexicographically for deterministic execution.
5. Events emitted by the socket connections for each client will be collected on a queue.
6. The queue will be read by the plugin worker pool.
7. Each plugin registered for the events (which will be batched) will receive them and can operate on the info.

The design above allows the game server to be configured to limit the resource utilisation produced by connected clients. An important element of the design is also that when too many messages are pushed onto the queue and the queue capacity is reached it will be purged. This ensures the server can never be OOMed by too many game client connections or slow plugins.

The "queue" mentioned above will actually be implemented as a circular buffer for performance. The consequence of the design (which balances server availability, resource limits, and performance) is that some events will never reach the plugins if the worker pool cannot keep up with the producers (game clients).

Plugins will be written in Go only.

[jompu]

What is the state of plugin architecture?

I am interested to use this project for our back-end needs. I would not like to fork the project to be able to add features. Plugin architecture could also help you to build/scale community.

[novabyte]

Hi @jompu. It depends on what you need to develop as custom features. Do you specifically need analytics or custom events to be exported out of the server or do you want to develop custom gameplay code?

The plugins architecture was added as a proof of concept to be built on top of Go plugins. This was added in the Go 1.8 release but has been left unfinished by the Go team at Google:

The plugin support is currently incomplete, only supports Linux, and has known bugs. Please report any issues.

If you want to add custom game logic and interact with 3rd party HTTP APIs have a look at our code runtime. It includes Lua as an embedded language to develop authoritative code. It's how most developers and studios are extending the server with game-specific logic.

[jompu]

Thanks @novabyte!

One reason why I'm thinking this is, that I'm trying to figure out how easily the community can add features and how easily they can share those. The plugin system would greatly help on that, by unifying the process of developing and publishing plugins by community.

One thing I would need to add is to send automatic e-mail when new account is created and support for resetting password by e-mail. Although in most cases I can use HTTP APIs by third party, some of them have already SDK for Go.

Lua is good choice for server side game play code. And thanks for your great effort to develop an open source gaming back end, I hope you will be successful.

[kerma]

I'm trying to get the plugin system working. I know it's WIP, but I'm following the setup from: https://github.com/heroiclabs/nakama/blob/master/sample_go_module/README.md

I'm building the minimal example with Docker using the nakama-pluginbuilder:2.3.1 container. Nakama container version is 2.3.1 as well. However this is what I see when server tries to load the plugin:

{"level":"info","ts":"2019-01-16T12:52:36.468Z",
 "msg":"Initialising runtime","path":"/server/data/modules"}
{"level":"fatal","ts":"2019-01-16T12:52:36.568Z",
 "msg":"Error reading InitModule function in Go module",
"name":"playwin",
"stacktrace":"github.com/heroiclabs/nakama/server.NewRuntimeProviderGo\n\t/go/src/github.com/heroiclabs/nakama/server/runtime_go.go:1821\ngithub.com/heroiclabs/nakama/server.NewRuntime\n\t/go/src/github.com/heroiclabs/nakama/server/runtime.go:392\nmain.main\n\tmain.go:109\nruntime.main\n\t/usr/local/go/src/runtime/proc.go:201"} 

Seems that it's function signature validation issue:

nakama/server/runtime_go.go

Line 1821 in 1174be7

startupLogger.Fatal("Error reading InitModule function in Go module", zap.String("name", name))

..but I've checked this multiple times now, pretty sure it's ok. Any ideas how to debug this futher?

[zyro]

@kerma This issue is not related to runtime Go plugins, it's for a separate but similarly named feature. Hop on to our community channel and we'll be happy to help - it's probably a function signature definition issue.

[mosdeo]

I'm trying to get the plugin system working. I know it's WIP, but I'm following the setup from: https://github.com/heroiclabs/nakama/blob/master/sample_go_module/README.md

I'm building the minimal example with Docker using the nakama-pluginbuilder:2.3.1 container. Nakama container version is 2.3.1 as well. However this is what I see when server tries to load the plugin:

{"level":"info","ts":"2019-01-16T12:52:36.468Z",
 "msg":"Initialising runtime","path":"/server/data/modules"}
{"level":"fatal","ts":"2019-01-16T12:52:36.568Z",
 "msg":"Error reading InitModule function in Go module",
"name":"playwin",
"stacktrace":"github.com/heroiclabs/nakama/server.NewRuntimeProviderGo\n\t/go/src/github.com/heroiclabs/nakama/server/runtime_go.go:1821\ngithub.com/heroiclabs/nakama/server.NewRuntime\n\t/go/src/github.com/heroiclabs/nakama/server/runtime.go:392\nmain.main\n\tmain.go:109\nruntime.main\n\t/usr/local/go/src/runtime/proc.go:201"} 

Seems that it's function signature validation issue:

nakama/server/runtime_go.go

Line 1821 in 1174be7

startupLogger.Fatal("Error reading InitModule function in Go module", zap.String("name", name))

..but I've checked this multiple times now, pretty sure it's ok. Any ideas how to debug this futher?

I also encountered this problem, I've checked that all day but can't solve.

[novabyte]

@mosdeo Is there a particular reason you've used the 2.3.1 release of Nakama? I would recommend you use the latest release of Nakama which is 2.9.0.

In the 2.7.0 release we had to change how the runtime package for Nakama is imported. It became part of a separate repo (i.e. nakama-common) so that Go's new(er) dependency management solution with modfiles could be supported.

This will be why the InitModule signature appears wrong because the Go types are different as they come from different dependencies. If you continue to have trouble please open a question on the forums.

[novabyte]

This was completed in an earlier release of Nakama and is documented now as "Events":

https://heroiclabs.com/docs/advanced-events/