Is the documentation for making non-go plugins outdated?

[SSSOCPaulCote]

I'm using this library in a project and I tried making a python plugin following the instructions in the docs folder and I keep getting this error:

2022-09-08T04:54:52.820Z [TRACE] plugin.stdio: waiting for stdio data
2022-09-08T04:54:52.827Z [DEBUG] plugin.stdio: received EOF, stopping recv loop: err="rpc error: code = Unimplemented desc = Method not found!"

I then cloned this repo and tried running the python plugin example and get the same error. While looking at the source code, I noticed that golang plugins register many gRPC services that are not present for the python equivalent:
grpc_server.go

// ServerProtocol impl.
func (s *GRPCServer) Init() error {
	// Create our server
	var opts []grpc.ServerOption
	if s.TLS != nil {
		opts = append(opts, grpc.Creds(credentials.NewTLS(s.TLS)))
	}
	s.server = s.Server(opts)

	// Register the health service
	healthCheck := health.NewServer()
	healthCheck.SetServingStatus(
		GRPCServiceName, grpc_health_v1.HealthCheckResponse_SERVING)
	grpc_health_v1.RegisterHealthServer(s.server, healthCheck)

	// Register the reflection service
	reflection.Register(s.server) // reflection not in python

	// Register the broker service
	brokerServer := newGRPCBrokerServer()
	plugin.RegisterGRPCBrokerServer(s.server, brokerServer) // broker not in python
	s.broker = newGRPCBroker(brokerServer, s.TLS)
	go s.broker.Run()

	// Register the controller
	controllerServer := &grpcControllerServer{server: s} 
	plugin.RegisterGRPCControllerServer(s.server, controllerServer) // controller not in python

	// Register the stdio service
	s.stdioServer = newGRPCStdioServer(s.logger, s.Stdout, s.Stderr)
	plugin.RegisterGRPCStdioServer(s.server, s.stdioServer) // stdio not in python

The gRPC client at the least seems to interact with Stdio and Broker services:
grpc_client.go

// newGRPCClient creates a new GRPCClient. The Client argument is expected
// to be successfully started already with a lock held.
func newGRPCClient(doneCtx context.Context, c *Client) (*GRPCClient, error) {
	conn, err := dialGRPCConn(c.config.TLSConfig, c.dialer, c.config.GRPCDialOptions...)
	if err != nil {
		return nil, err
	}

	// Start the broker.
	brokerGRPCClient := newGRPCBrokerClient(conn)
	broker := newGRPCBroker(brokerGRPCClient, c.config.TLSConfig)
	go broker.Run()
	go brokerGRPCClient.StartStream()

	// Start the stdio client
	stdioClient, err := newGRPCStdioClient(doneCtx, c.logger.Named("stdio"), conn)
	if err != nil {
		return nil, err
	}
	go stdioClient.Run(c.config.SyncStdout, c.config.SyncStderr)

	cl := &GRPCClient{
		Conn:       conn,
		Plugins:    c.config.Plugins,
		doneCtx:    doneCtx,
		broker:     broker,
		controller: plugin.NewGRPCControllerClient(conn),
	}

	return cl, nil
}

What I'm asking is, should there be gRPC Broker and Stdio implementations in python for python plugins to work?

[fairclothjm]

Hi, @SSSOCPaulCote

should there be gRPC Broker and Stdio implementations in python for python plugins to work

No, this is not required, please see the PR which Adds a Connection Broker for GRPC plugins. These are listed as optional for plugins implemented in other languages.

I was able to run the python plugin in the examples directory successfully.

Notice the log lines you posted are TRACE and DEBUG:

2022-09-08T04:54:52.820Z [TRACE] plugin.stdio: waiting for stdio data
2022-09-08T04:54:52.827Z [DEBUG] plugin.stdio: received EOF, stopping recv loop: err="rpc error: code = Unimplemented desc = Method not found!"

These are not reported by go-plugin as errors.

Can you try the following commands and see if you are able to put and get with the python example?

# This builds the main CLI
$ go build -o kv

# This tells the KV binary to use the "kv-go-grpc" binary
$ export KV_PLUGIN="python3 plugin-python/plugin.py"

# Read and write
$ ./kv put hello world

$ ./kv get hello
world

[SSSOCPaulCote]

@fairclothjm You're right, the issue is actually in my Python plugin, I'm somehow not implementing the interface on my project. I'll have to figure that one out. Could you point me in the right direction to how your stub kv_pb2.py and kv_pb2_grpc.py files are generated? I use a method from this repo which has you clone a googleapis repo and use that as a proto path $ python -m grpc_tools.protoc --proto_path=googleapis:. --python_out=. --grpc_python_out=. test.proto

EDIT: I figured it out, I had different invoke paths between my Go and Python stubs. Also, my VM is acting weird and using different versions of the plugin file I made every time I run the example. Doing this on main OS has consistent results