KrakenD Plugin System - error while loading package file

[labsvisual]

Describe what are you trying to do
We decided to go ahead with KrakenD while evaluating the plethora of choices available for an API gateway. More specifically, the plugin system and its accompanying example of implementing the gRPC gateway as a HTTP-client plugin was what sealed the deal. (Since that is exactly our use-case.)

I followed the exact steps as mentioned in the tutorial and compiled the plugin library using the golang:latest image; copied the library over to the plugins folder and started KrakenD.

A couple of log lines were more peculiar upon initial inspection:

[KRAKEND] 2020/07/22 - 16:41:47.955 ▶ WARNIN loading plugins: plugin loader found 1 error(s):
opening plugin 0 (/opt/krakend/plugins/plugin.so): plugin.Open("/opt/krakend/plugins/plugin"): plugin was built with a different version of package internal/cpu

After some Googling, it was understood that this is a known problem with Go's plugin system since it's very picky about runtime and system library versions.

Your configuration file
The configuration file is the same as picked from the example:

{
    "version": 2,
    "name": "gRPC Gateway - v1",
    "port": 8080,
    "cache_ttl": "3600s",
    "timeout": "3s",
    "plugin": {
        "pattern":".so",
        "folder": "/opt/krakend/plugins/"
    },
    "extra_config": {
        "github_com/devopsfaith/krakend-gologging": {
            "level":  "DEBUG",
            "prefix": "[KRAKEND]",
            "syslog": false,
            "stdout": true
        }
    },
    "endpoints": [
        {
            "endpoint": "/user",
            "backend": [
                {
                    "host": [ "http://ignore.this" ],
                    "url_pattern": "/v1/user",
                    "querystring_params": [ "user_id" ],
                    "extra_config": {
                        "github.com/devopsfaith/krakend/transport/http/client/executor": {
                            "name": "internal-grpc-gateway",
                            "endpoints": [ "localhost:50051", "localhost:50052" ]
                        }
                    }
                }
            ]
        }
    ]
}

Commands used
The Kraken base image was used (devopsfaith/krakend:latest) and the plugin files were copied to the configured location (in this case /opt/krakend/plugins/plugin.so).

The CMD parameter was modified to:

CMD [ "--config", "/opt/krakend/config.json", "run", "--port", "8080" ]

What all have I tried
Over the last hour or so, I have tried every possible thing which may result in successful runtime precondition validation. To list out all the methods in their detail:

• I started with adding the -trimpath flag as mentioned by this comment; sadly, this didn't work.
• I copied the exact linker flags from the current Makefile but it gave the same error;
• I used go1.12 as mentioned in the go.mod file but that gave me: plugin.Open("/opt/krakend/plugins/plugin.so"): plugin was built with a different version of package errors;
• thought it might be an issue caused by transitive and indirect dependencies so I pinned the entire module set in Kraken's go.mod and added them as _ imports but still that was of no avail.

I am sure I am doing something wrong but can't pinpoint it; any help would be amazing.

[kpacha]

in order to know the exact deps used at a given version, you can check the go.sum and Makefile from the related tag.

krakend 1.1.1 was built with go1.14.0 (https://github.com/devopsfaith/krakend-ce/blob/v1.1.1/Makefile#L21), so building your plugins with that version should fix the problem.

on the other hand, notice the plugin interface evolved from the PoC described at that blog post and the final release (https://godoc.org/github.com/devopsfaith/krakend/transport/http/client/plugin). In fact, the only thing you should change is the name of your symbol: from GRPCRegisterer to ClientRegisterer

[labsvisual]

Built it with go1.14.0, updated a few packages and got the following:

[KRAKEND] 2020/07/22 - 19:02:44.310 ▶ WARNIN loading plugins: plugin loader found 1 error(s):
opening plugin 0 (/opt/krakend/plugins/plugin.so): plugin.Open("/opt/krakend/plugins/plugin"): plugin was built with a different version of package google.golang.org/genproto/googleapis/api/annotations

Since genproto is a transitive dependency for Kraken, it's a little tough to pinpoint the exact version in use. Is there a recommended way to resolve this issue because manually cherry-picking versions like this is very time consuming and prone to human errors.

(I understand that this has nothing to do with Kraken directly, but just wanted to know how do you do it while you're testing/developing Kraken or writing your own plugins?)

[kpacha]

as said, the go.sum file contains all the versions used in a given build: https://github.com/devopsfaith/krakend-ce/blob/v1.1.1/go.sum#L1025

[labsvisual]

Tried that out; it's too much work and leads to a very brittle build process. Thanks for the help, regardless! 😄

[alombarte]

@labsvisual this is why we have it automated on KrakenD Studio through our gRPC plugin generator (available to KrakenD Enterprise users). If you want an evaluation or more information get in touch

[abhishekkundalia]

try building the plugin on the same machine type. This helped me along with same go version.

[dpalmasan]

@labsvisual Not sure if you managed to solve this, I had a similar issue, and I solved it by compiling krakend binary and using that binary.