External gRPC components

[jjcollinge]

/area runtime

Describe the proposal

NOTE: I haven't had much time to give this problem a lot of thought or to put together this proposal so please excuse any mistakes or oversights but I wanted to get something out there so we can start to move this area forward.

Prototype: https://github.com/jjcollinge/dapr-external-state

What?

This proposal outlines an approach to enable Dapr to integrate with "external" (not compiled into daprd) state store providers using a gRPC API. There is a bigger objective to allow all components to be external/remote and/or written in other lanugages but this proposal only focuses on the state store building block. This proposal should also not be seen as the ONLY way to use external state stores (components) as we will likely want to support WASM/WASI and possibly other interfaces too at a later date.

Why?

Allowing Dapr to use state stores that have not been contributed back to the upstream project (out of tree) or have been written in a language other than Go is really important for adoption as some people may have private state stores they wish to use or might not be comfortable writing in Go.
This is a frequently requested feature and is related to the following issues:

• Enable gRPC based components #3513
• [Proposal] Implement WASM for state/pubsub #3496
• [Proposal]: Dapr as a capability delivery vehicle #1311
• Proposal: Provide a "Plugin" Architecture to Load components-contrib Components #739
• Offer multiple flavors of Dapr sidecar #3168

How?

gRPC

As mentioned above, this proposal suggests we use gRPC to integrate with external state store providers initially. For details on why gRPC please refer to this thread (TLDR; Go doesn't have dynamic libraries and WASM/WASI is too restricted currently for full plugins). gRPC can be run over tcp or unix domain sockets so we should look to support both for performance sensitive workloads.

In order to provide an external gRPC state store, Dapr needs to define an interface using protobuf that the external store can implement.

In this case, I've simply mapped the existing Store interface that is used to implement state stores over to protobuf (likely needs refinement).

store.proto

// ------------------------------------------------------------
// Copyright (c) Microsoft Corporation and Dapr Contributors.
// Licensed under the MIT License.
// ------------------------------------------------------------

syntax = "proto3";

package components.proto.state.v1;

import "google/protobuf/empty.proto";
import "common.proto";

// TODO: other language packages...
option go_package = "github.com/dapr/components-contrib/state/proto/v1;state";

// Store service provides a gRPC interface for state store components.
service Store {
  rpc Init(MetadataRequest) returns (google.protobuf.Empty) {}
  rpc Features(google.protobuf.Empty) returns (FeaturesResponse) {}
  rpc Delete(DeleteRequest) returns (google.protobuf.Empty) {}
  rpc Get(GetRequest) returns (GetResponse) {}
  rpc Set(SetRequest) returns (google.protobuf.Empty) {}
  rpc Ping(google.protobuf.Empty) returns (google.protobuf.Empty) {}
  rpc BulkDelete(BulkDeleteRequest) returns (google.protobuf.Empty) {}
  rpc BulkGet(BulkGetRequest) returns (BulkGetResponse) {}
  rpc BulkSet(BulkSetRequest) returns (google.protobuf.Empty) {}
}

message MetadataRequest {
  map<string, string> properties = 1;
}

message FeaturesResponse {
  repeated string feature = 1;
}

message GetRequest {
  string key = 1;
  map<string, string> metadata = 2;
  dapr.proto.common.v1.StateOptions.StateConsistency consistency = 3;
}

message GetResponse {
  bytes data = 1;
  dapr.proto.common.v1.Etag etag = 2;
  map<string, string> metadata = 3;
}

message DeleteRequest {
  string key = 1;
  dapr.proto.common.v1.Etag etag = 2;
  map<string, string> metadata = 3;
  dapr.proto.common.v1.StateOptions options = 4;
}

message SetRequest {
  string key = 1;
  bytes value = 2;
  dapr.proto.common.v1.Etag etag = 3;
  map<string, string> metadata = 4;
  dapr.proto.common.v1.StateOptions options = 5;
}

message BulkDeleteRequest {
  repeated DeleteRequest items = 1;
}

message BulkGetRequest {
  repeated GetRequest items = 1;
}

message BulkStateItem {
  string key = 1;
  bytes data = 2;
  dapr.proto.common.v1.Etag etag = 3;
  string error = 4;
  map<string, string> metadata = 5;
}

message BulkGetResponse {
  repeated BulkStateItem items = 1;
  bool got = 2;
}

message BulkSetRequest {
  repeated SetRequest items = 1;
}

This proto definition should be hosted in the dapr/dapr repo along with the generated client/server stubs and be strongly versioned as per the other Dapr protos.

Dapr

Dapr needs a way to use the client stubs to invoke a remote service that satisfies the gRPC API. In order to do this, we can define a new state store components called external (or something gRPC specific like grpc) in dapr/components-contrib. This state store component implements the Dapr Store interface but is simply an adaptor to invoke a remote gRPC API using the generate Go client. This adaptor will need to map the incoming Dapr state store types to the protobuf types and then invoke the remote method and map back the response.

So how do we use this external state store provider and how does Dapr know how to talk to the external gRPC API? Well the store interface has a Init(metadata state.Metadata) error which is typically used to parse metadata and initialize the state store connection. The metadata struct is just a property bag so for our external state store we can just grab the connection details from here. How will the remote state store be configured? the gRPC API could be pre-configured independently - but I think given that we want to mirror the approach of existing state stores as much as possible - we can simply invoke the Init method remotely and pass the native metadata from the Dapr component across (excluding any additional external properties).

This would allow us to have a component definition like:

apiVersion: dapr.io/v1alpha1
kind: Component
metadata:
  name: statestore
  namespace: statestore
spec:
  type: state.external # Use the external state store type.
  version: v1 # Defines the version of the proto the external state store implements.
  metadata:
  - name: externalAddress # This field is required and will be stripped before sending to the external state store.
    value: localhost:9191
  - name: redisHost # These properties stay the same as the concrete state store.
    value: localhost:6379

Where the state store type is external and we provide this additional property externalAddress to inform Dapr where the remote instance is. The rest of the metadata needs to conform to the concrete state store's schema - in this case Redis - as the remote state store will handle this. We will likely be sending "secrets" across to the external service so we'll need to ensure it is trusted and the transport encrypted (see: #security).

With this, the external state store can open a connection to the remote gRPC server and "proxy" the operations. The Store interface currently doesn't have a Close method so there is currently no way to gracefully close the gRPC connection with the external state store.

External State Store

We can implement the gRPC server API in whatever language we want to (as long as it is supports gRPC). The gRPC server can then handle requests however it needs to - including invoking a custom or proprietary state store.

Given that Dapr defines the gRPC interface, I believe Dapr can provide standard server implementations for the most common languages which uses a similar factory pattern to daprd and components contrib. These implementations will map the gRPC data back into a Dapr model and then invoke a Store interface defined in the native language. This way, to add a new state store component in a different language, a contributor can just implement the Store interface (as you would in Go) and plug it into the appropriate server implementation. We could then also support "built-in" state store components contributed in other languages in these servers whilst also supporting people who have private or separate state stores.

Security

As with all Dapr comms - we'll want to ensure that we are leveraging mTLS between Dapr and the external state store where possible. This is where Dapr providing standardized servers might mean we can hook into Sentry to issue the gRPC server trusted certificates.

Performance

Obviously there's going to be a performance impact by having to invoke an external state store service rather than an in memory implementation. Therefore, users must be informed about the trade off they are making when using an external state store. We can try to minimize the performance impact as much as possible by allowing users to use unix domain sockets as well as tcp. Longer term, I expect the maturity of WASI to become a more performant alternative but that'll take time.

Prototype

I have built a prototype based on this proposal which is available here that implemented the external state store in Go and ruses the existing state stores remotely. Beware this was hacked together very quickly so has sharp corners - but hopefully it's useful to demonstrate the proposal.

This is just an initial draft of an approach to see whether this seems viable - I'm sure there are alternatives that are worth discussing so please feel free to give candid feedback before I progress this any further.

Future

Although I have scoped this issue to just the state store build block - I believe this pattern could be used to support other build block APIs. Depending on the progress of WASM (interface types, WASI, etc.) which might provide a better solution, we may get to a point where we have language proxies for the entire Dapr API.

[pkedy]

we may get to a point where we have language proxies for the entire Dapr API.

For communicating with polyglot components via external processes, I envisioned it working as a language proxy as well. Instead of hosting one component, like in your fantastic experiment app, it would be able to proxy to multiple components of each building block type. One challenge is that metadata.name is not accessible from the component's scope. name would be helpful to pass in the gRPC request so that the "language proxy" can route it to the correct component instance. Security/TLS also should be considered so there is a strong trust between Dapr and the language proxy.

At that point, I could see each implementation of the language proxy doing the following:

• Maintain a thread-safe map of all initialized components
• If supported by the language, dynamically load component code from .so/.dlls/.wasm
• Expose a "metadata" service to list the component types it hosts
• Close/reset a specific component or all components
• ...Basically a simple component lifecycle model

Details aside, this is great and I hope to see something like this move forward.

[dapr-bot]

This issue has been automatically marked as stale because it has not had activity in the last 30 days. It will be closed in the next 7 days unless it is tagged (pinned, good first issue, help wanted or triaged/resolved) or other activity occurs. Thank you for your contributions.

[dapr-bot]

This issue has been automatically closed because it has not had activity in the last 37 days. If this issue is still valid, please ping a maintainer and ask them to label it as pinned, good first issue, help wanted or triaged/resolved. Thank you for your contributions.

[patrickhuber]

@artursouza after our conversation last week, I have created a sample implementation of this as well.

@jjcollinge I like your implementation. It uses grpc services that are running externally correct? The big difference with this implementation is the use of go-plugin which is a wrapper around grpc and handles some of the component lifecycle. There are also numerous other features that address security, upgrade, logging and more. I didn't see this proposal at first, but I plan on using your proto file. With a quick glance the only difference I see is the use of the google Empty message.

You can see my implementation here https://github.com/patrickhuber/components-contrib/tree/master/state/plugin

I have yet to test in full dapr, but a local communication between a unit test component and a compiled plugin worked without issue.

The plugin client is setup to filter out metadata for the plugin and non-prefixed keys to the plugin. In theory, this should make it very easy to wrap existing components. With some conventions and a registry of some kind, the basedir could probably be removed. At least plugin name, version and runner (dotnet | java | node | docker | python | exec | ...). Here is a proposed sample file:

apiVersion: dapr.io/v1alpha1
kind: Component
metadata:
  name: statestore
spec:
  type: state.plugin
  version: v1
  metadata:
  - name: plugin.basedir
    value: ./plugins
  - name: plugin.version
    value: 0.0.1
  - name: plugin.name
    value: go-memory
  - name: plugin.runner
    value: exec
  - name: message
    value: hello

A reference go memory plugin is in the fixtures directory https://github.com/patrickhuber/components-contrib/tree/master/state/plugin/fixtures/state/go-memory/0.0.1 . The plugin must be built before running the unit test. The pathing and naming is roughly based on how hashicorp plugins are stored.

I still need to test this with a fork of core dapr to make sure it works properly in the larger scope.

[jjcollinge]

@patrickhuber great work 👍. I meant to look into trying go-plugin as an alternative so you've saved me the effort! If it manages the plugin lifecycle and security then this is a big win. I'll dig into your sample later this week.

[patrickhuber]

Wonderful! I'm going to continue with some of the other plugin samples in other languages. I think python will be next because there is already a sample in the go-plugin repo.

…

On Sun, Jan 9, 2022, 12:35 PM Joni Collinge ***@***.***> wrote: @patrickhuber <https://github.com/patrickhuber> great work 👍. I meant to look into trying go-plugin as an alternative so you've saved me the effort! If it manages the plugin lifecycle and security then this is a big win. I'll dig into your sample later this week. — Reply to this email directly, view it on GitHub <#3787 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AABMQ7TAGU3ZP4C7YOP7KQLUVHBN5ANCNFSM5GLWI3YQ> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>. You are receiving this because you were mentioned.Message ID: ***@***.***>

[patrickhuber]

I finished up the node and python examples. https://github.com/patrickhuber/components-contrib/tree/master/state/plugin/fixtures/state/

[yaron2]

I finished up the node and python examples. https://github.com/patrickhuber/components-contrib/tree/master/state/plugin/fixtures/state/

Hey Patrick, this will be discussed in tomorrow's community call. It'd be great if you could participate.

[patrickhuber]

Ok. I put it on my calendar, I'll try my best to be there.

…

On Mon, Jan 10, 2022, 8:01 PM Yaron Schneider ***@***.***> wrote: I finished up the node and python examples. https://github.com/patrickhuber/components-contrib/tree/master/state/plugin/fixtures/state/ Hey Patrick, this will be discussed in tomorrow's community call. It's be great if you could participate. — Reply to this email directly, view it on GitHub <#3787 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AABMQ7WWZY27ZECACOH24I3UVN6OVANCNFSM5GLWI3YQ> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>. You are receiving this because you were mentioned.Message ID: ***@***.***>

[jjcollinge]

I won't be able to make the community call unfortunately but I think one thing we should try to address is what @pkedy mentioned. Basically, is it better to host a language proxy that can load multiple plugins to support multiple Dapr components and APIs (Indirect plugins in the diagram below). Or like in @patrickhuber sample, we just load plugins on demand for specific components (direct plugins in the diagram below). I personally think that if go-plugin is providing component lifecycle that the simpler plugin protocol is to use the direct plugins and not try to offer a more complex language proxy (but do offer an SDK to make it easy to implement) but I'm open to debate.

[patrickhuber]

There is also an opportunity to scope plugins in more of a provider model. So one plugin could provide multiple different services. I drew up a visual representation. The language aspects would be exposed via an sdk. The developer of a plugin would only need to consume the SDK, implement the service contracts and register the service in a key map. The model could apply to other component types as well.