[Proposal] (Updated) gRPC Components (aka "Pluggable components")

[johnewart]

To reset the conversation a bit in preparation for 1.9, this is an updated proposal for gRPC-based components. This proposal and its underlying implementation does not deviate much from the previous proposal and work from @patrickhuber but adds a bit more clarity around the design and proposed packaging of components.

Dapr Pluggable Components

Proposal

To build a general-purpose mechanism for implementing Dapr components outside of the daprd binary itself. The goal of this is to both improve the overall health of the Dapr sidecar as well as increase flexibility when building new component implementations.

Terminology

Building block -- an abstract, reusable, description of a set of operations including their expected behavior, inputs and outputs. For example, a "state store" building block can store state, fetch state, delete state or update state.

Component an implementation of a Dapr building block that provides the desired functionality. For example, a Redis state store would implement the "state store" building block operations by using Redis as a storage engine.

Design goals

• Create a mechanism for building Dapr components in a way that does not require modifying the dapr source code and recompiling the binaries.
• Minimize the performance penalty incurred by such a system.
• Allow components to be developed in languages other than Go (specifically, those supported by gRPC)
• Increase flexibility of the dapr sidecar
• Improve long-term health of the dapr project

Non-goals

• Support components running on machines separate from where the dapr sidecar or daemon is being run (i.e., via remote TCP/IP sockets).
• Supporting programming languages that do not have existing protobuf compilers.

Background

Today, building a new component requires that the code for the component be built into the Dapr binary itself. This mechanism presents a few hurdles for Dapr (though this also applies to any other generally extensible system) that make development of new features and long-term maintenance of the platform itself challenging. In addition to project health-related challenges, it increases the friction for adoption of Dapr when customers need to integrate with systems that are not otherwise open-source. As a result, Dapr will benefit from a framework that enables the development of components that are not tightly coupled to the Dapr source code in both maintainability and adoption.

Challenges related to project health

As Dapr grows, and along with it the number of components that exist, there arFe several technical challenges that the project will face including development, deployment and security-related problems. These include (but are not necessarily limited to):

• New components increase the size of the dependency-graph, leading to a higher likelihood of conflicts or forced-upgrades.

• Larger application footprint as all modules will be built into the binary distribution whether they are used or not.

• Increased surface area for security vulnerabilities due to the inclusion of unused code and dependencies.

• All components are required to be built using Go (which is not a problem per-se but can make adding new components challenging if the technology being used does not already have a Go SDK or implementation).

• Bugs in components can cause instability across the platform, rendering the system inoperable.

Challenges related to adoption of Dapr

As much as we may want all things to be open source the truth is that users cannot always open source the things that they build. This means that, if a customer needs to extend Dapr for integration with their own internal systems, they currently face a few challenges that makes it harder for them to adopt and maintain Dapr internally. For example:

• Adopters must maintain an internal fork of Dapr (or apply patches to Dapr) to build and manage their own custom extensions that are not open source.

• As mentioned before, all extensions to Dapr must be written in Go, which may or may not be feasible in any given organization for a variety of reasons; this means that a team may not be able to adopt Dapr.

• Adding support for features that require CGO or other exotic implementations present additional challenges and changes to the build process / toolchain

Proposed Solution

This document proposes a system by which components can be plugged in to Dapr without being compiled into the core binary. To achieve this, the Dapr platform will implement each building block / component type in the dapr binary as a stub and then communicate with the implementation (component) using inter-process communication (IPC). In this proposal, we will look at using gRPC as the transport mechanism (see below for alternatives considered). Components can then be built external to the dapr binary and communicate with the runtime using gRPC as the underlying transport to communicate with the Dapr runtime. Plugins can then implement the APIs for these building blocks, much the same way they do now, but communicate with Dapr using gRPC as opposed to operating in-process with the dapr binary. This presents several benefits, such as the potential to have a much smaller core codebase for dapr, smaller binaries, and a more flexible system for building components in any language that supports gRPC (which includes most commonly used languages as of today). However, it is also to be noted that this model is not without its drawbacks, including increased latency (see bottom of proposal for a brief initial analysis), a requirement to build a system to package and deploy these extensions, and challenges around compatibility guarantees as the components change at different rates.

Rather than compile all the component implementations into the Dapr sidecar binary, this solution proposes that the Dapr sidecar instead implement each component as two parts:

• A gRPC stub (component client) that has no concrete functionality.

• A gRPC daemon (component service) that is responsible for implementing the actual component logic (i.e a Redis state store, a RabbitMQ pub/sub bus).

In addition to these, there is a "pluggable component" registry in the Dapr runtime that integrates with the existing component registry but performs the following additional behavior:

• Specify a UNIX domain socket that both the client and server can connect to.

• Register an instance of the component in the Dapr runtime, which is a gRPC client implementing the underlying building block’s API (i.e., a "state store”)

• Create a new instance of the pluggable component (container or process)

• Wait for the external component to be ready and listening on the UNIX domain socket.

• Signal that the component is ready (or block on dependencies).

• Communicate with the service to provide the component to the Dapr runtime.

The gRPC daemon (service) external to the Dapr runtime performs the following:

• Initialize the component implementation (connect to Redis, RabbitMQ, etc.)

• Listen for gRPC connections on the specified UNIX domain socket.

• Serve the component API to the Dapr runtime and perform the underlying operations.

In this model, the components themselves are processes that run alongside the Dapr runtime; the Dapr runtime is responsible for communicating with these components, not implementing the functionality. As such, the Dapr runtime behaves more like an abstraction layer than a concrete implementation, leaving the implementation details to the component processes.

Side-effects of this model include reduced dependencies in the runtime, which has several benefits on its own (including reduced surface area for security vulnerabilities, as well as decreasing the chance of dependency conflicts), and isolation of the component processes, which means that if a single component has a bug the runtime can degrade gracefully rather than crash entirely.

Packaging / Runtime

One of the challenges of implementing this approach is the other side of one of its benefits – operating components as separate processes means that we need a way to deploy, execute, discover, and manage those components. As Dapr has two primary modes of operation, standalone and cluster-mode, there are two different scenarios under which this model needs to function. We propose that this be solved using containers as the default deployment model, though not necessarily the required deployment model. In order to connect components with the runtime, they need to have access to a shared UNIX domain socket on the local host that they can use to communicate with one another, and so it is possible to run components without using containers if they are able to access these shared sockets.

In this implementation it would be possible to have daprd spawn and manage processes for pluggable components (as the only requirement is being able to communicate with the component via gRPC) but this is not currently part of this proposal for a number of reasons:

• Dependency management inside a single container / host, including runtimes, for all external components is challenging
• Based on feedback, users would like to be able to run "official" dapr release artifacts (containers, binaries, etc.) with minimal addition to those artifacts and would prefer external components to be packaged and managed independently
• This model implificitly brings on the requirement that Dapr re-implement lifecycle management for external components (which Kubernetes / Docker already support)
• Distribution of pluggable components is much simpler in this model as they can be vended as a container, complete with runtime, and be more easily audited and managed independently.

Kubernetes Mode

When run alongside an application in a Kubernetes cluster, components would be deployed as additional sidecars within a pod; being that all the containers in a pod execute on the same host, it is simple enough for a deployment to map a shared volume of some sort, ephemeral or other, between the runtime and the components being used. For example, a pod that is deploying the runtime along with a handful of components may end up looking like the following diagram:

Standalone Mode

When running Dapr in standalone mode, containers can still be leveraged as a tool for deployment and execution of components (assuming that a container runtime is available and able to interact with the Dapr runtime), this guarantees a consistent runtime environment for each component. If a container runtime is not available, it is entirely possible to deploy components as standalone services running on the host but then it is up to the operator to ensure that a compatible runtime (and any dependencies) are available on the host in order for the component(s) to operate.

Alternatives Considered

There are many ways to achieve inter-process communication (IPC) as it is prevalent in modern software (for example Google Chrome uses IPC to transfer data between internal processes). As a result, there are a lot of different approaches available to us beyond using gRPC. In fact, there are several reasons why gRPC is not (technically) the best choice, including: large library overhead, use of non-standard HTTP libraries, and only supporting HTTP/2 as the communication protocol. In order to be comprehensive, we looked at several alternatives before deciding on this approach including:

gRPC + protobuf

Pros:

• Reuse existing protobuf mechanisms
• Auto-generates client / server stubs
• Used already by dapr runtime and SDKs for API communication (making it easy to write components using existing SDK infrastructure)
• Relatively minimal latency (~50us measured)

Cons:

• Latency is higher than in-process
• Only HTTP/2 supported
• Non-standard HTTP library (in Go)

ZeroMQ + protobuf

Pros:

• Lightweight
• Multiple languages
• Easy to implement
• Uses existing protobuf mechanism

Cons:

• Requires a new library / way of doing things
• Depending on Go library, may require C bindings / FFI
• No auto-implemented client/service components

Twirp (gRPC-alike framework from Twitch)

Pros:

• HTTP/1.1 as well as HTTP/2
• Multiple serializers
• Uses standard HTTP library in Go

Cons:

• Not in use by runtime currently (new model)

Custom-written IPC library

Pros:

• None, per-se, other than raw performance

Cons:

• You are likely to be eaten by a Grue... 😅

Go binary plugins (using Go's built-in plugin library)

Pros:

• Low overhead, likely the same, or very similar, latency as built-ins
• No requirement to build a gRPC service + implementation

Cons:

• Only works on FreeBSD, macOS and Linux (no Windows support)
• Still requires that all components be written in Go

Future work

There are several follow-on items to be evaluated, including (but not necessarily limited to):

• Potentially leveraging CNAB (Cloud-Native Application Bundles) as a packaging tool
• Cross-host component discovery and hosting over TCP or other network protocol
• Post-launch attachment / dynamic registration of components

Notes on latency

This is by no means a complete analysis but rather a quick sanity check of the performance overhead introduced by a gRPC plugin. In this case, I have written a Redis-based .NET state store and compared its performance with the built-in Redis state store by using the .NET "bank transaction" API service to exercise each of them. Each was run with the same user profiles / behavior over time using Gatling (gatling.io), and the results are plotted here. Before launch we will do strenuous load testing of the various building blocks to determine where users might encounter issues with regards to load, throughput, and overhead.

[yaron2]

Original issue: #3787.

[yaron2]

Thanks for the proposal @johnewart. To be clear, are you proposing that in self-hosted mode it's daprd's responsibility to execute the component process, or the developer's?

[yaron2]

Also, @johnewart can you please provide an end to end user experience for this, including the CRD?

[johnewart]

Yes, in standalone mode (as it stands now), daprd starts the containers -- though it's possible to avoid that by using a 1-to-1 mapping of PluggableComponent to Component and specify the path to the domain socket manually. This would be, as we discussed out-of-band, for the case where daprd is run in standalone mode but not deployed with elevated privileges and so would not be able to run containers on its own. In these cases one would not be required to use containers if they preferred to deploy them alongside on the same host and manage them independently. I will post an example and the proposed CRD as well shortly.

[yaron2]

Yes, in standalone mode (as it stands now), daprd starts the containers -- though it's possible to avoid that by using a 1-to-1 mapping of PluggableComponent to Component and specify the path to the domain socket manually. This would be, as we discussed out-of-band, for the case where daprd is run in standalone mode but not deployed with elevated privileges and so would not be able to run containers on its own. In these cases one would not be required to use containers if they preferred to deploy them alongside on the same host and manage them independently. I will post an example and the proposed CRD as well shortly.

Thanks. To be clear, I'd like to avoid daprd starting containers or processes, optional or not. This should be done via developers / infrastructure teams, as Dapr is not a container orchestrator and from a security perspective this will require Dapr to run with elevated privileges.

To elaborate further:

Having daprd start containers (as an opt-in feature or not) creates a larger attack surface, not only for its ability to pull and execute remote code into the environment, but it also opens up existing features to attack by running with elevated privileges.

In addition, process / container orchestration is done today with industry leading tools such as Kubernetes, Docker Compose, Chef, Puppet, Ansible and a myriad of other tools. Developers and operators use these tools to deploy Dapr sidecars along side their apps and any other sidecars / containers / processes they have. Dapr's focus is on providing developers with APIs to write distributed systems, not container orchestration capabilities, and as such enabling Dapr to run containers can not only divert Dapr's focus, but also cause a never ending "feature chasing" race with said tools.

[daixiang0]

Yes, in standalone mode (as it stands now), daprd starts the containers -- though it's possible to avoid that by using a 1-to-1 mapping of PluggableComponent to Component and specify the path to the domain socket manually. This would be, as we discussed out-of-band, for the case where daprd is run in standalone mode but not deployed with elevated privileges and so would not be able to run containers on its own. In these cases one would not be required to use containers if they preferred to deploy them alongside on the same host and manage them independently. I will post an example and the proposed CRD as well shortly.

Thanks. To be clear, I'd like to avoid daprd starting containers or processes, optional or not. This should be done via developers / infrastructure teams, as Dapr is not a container orchestrator and from a security perspective this will require Dapr to run with elevated privileges.

To elaborate further:

Having daprd start containers (as an opt-in feature or not) creates a larger attack surface, not only for its ability to pull and execute remote code into the environment, but it also opens up existing features to attack by running with elevated privileges.

In addition, process / container orchestration is done today with industry leading tools such as Kubernetes, Docker Compose, Chef, Puppet, Ansible and a myriad of other tools. Developers and operators use these tools to deploy Dapr sidecars along side their apps and any other sidecars / containers / processes they have. Dapr's focus is on providing developers with APIs to write distributed systems, not container orchestration capabilities, and as such enabling Dapr to run containers can not only divert Dapr's focus, but also cause a never ending "feature chasing" race with said tools.

I agree that daprd should not become a process / container orchestration, which is not the goal / foucs of Dapr.

I can understand the basics of this idea refer to https://docs.dapr.io/operations/hosting/self-hosted/self-hosted-overview. Based on the current architecture of self-host, introducing docker-compose to cover the gPRC components is a choice (maybe include Redis and Zipkin too). We can provide docker-compose config file and teach users how to config all gRPC components, but not run / start new process / container in daprd.

[johnewart]

Looks like my previous comment wasn't saved (probably my fault, sorry) -- here's a proposed CRD for PluggableComponent (and a Component that uses it). It would be feasible to put the pluggable bits inside the Component definition but even if it's there I think that it's important to have a type-safe structure for specifying the pluggable definition because it allows for various ways of describing it that are easily discoverable - i.e a socket path, a container image, a path to a binary, etc. and makes it very clear that it's not a built-in thing. Thoughts?

A spec that has a container defined:

apiVersion: dapr.io/v1alpha1
kind: PluggableComponent
metadata:
  name: memstore
spec:
  type: state
  version: v1
  container:
    image: dapr-memstore
    version: latest
---
apiVersion: dapr.io/v1alpha1
kind: Component
metadata:
  name: pluggablestate
spec:
  type: state.memstore
  version: v1

A spec where the path to the socket is specified:

apiVersion: dapr.io/v1alpha1
kind: PluggableComponent
metadata:
  name: dotnetredis
spec:
  type: state
  version: v1
  socket: /tmp/dotnetredis.sock
---
apiVersion: dapr.io/v1alpha1
kind: Component
metadata:
  name: userstore
spec:
   type: state.dotnetredis
   version: v1
 

Additionally, after conversations about it, I agree that daprd should not be responsible for container management -- instead of having daprd orchestrate containers what if we look at adding that functionality to the CLI instead (to make developer lives simpler when bootstrapping)?

[yaron2]

Additionally, after conversations about it, I agree that daprd should not be responsible for container management -- instead of having daprd orchestrate containers what if we look at adding that functionality to the CLI instead (to make developer lives simpler when bootstrapping)?

That conversation should happen on the CLI repository. Personally, I think it's feasible to allow tooling to help with this, but ultimately I think for local dev, > 90% of users will use industry known tools like Docker compose, and would very much like to have documentation in place that meets users where they are. In any case, I do not think we should allow the container section at all, see reasoning below.

Overall the PluggableComponent CRD and its relationship with the Component CRD looks good.

I would remove this section entirely:

  container:
    image: dapr-memstore
    version: latest

Since this will not be supported in self-hosted mode by daprd, the container section creates an inconsistency between self-hosted and Kubernetes modes. Putting the container section in self-hosted mode for the CLI or other tooling only creates a tight coupling between daprd and the CLI which we should not allow.

For Kubernetes, the pluggable component image will be part of the Pod. Kubernetes operators and users expect to define containers that hold their bits via a PodSpec. In that case, it's redundant to put the container section in the PluggableComponent CRD as the definition of it will be expected live in the Deployment/StatefulSet/Job/Other object that governs the Pod spec.

I am assuming, of course, that the container section shown above is the Kubernetes Container/v1 spec, as users will definitely need to control things like arguments, command, security contexts, probes, termination etc.
And since this is the Kubernetes v1 pod spec, this brings us to another problem that we might have when tooling (such as the CLI) tries to implement this spec - it will most likely want to honor the spec for the sake of consistency and portability.

Other than that, LGTM.

[johnewart]

Sounds good, I think we can figure out a way to make it easier for developers to bootstrap their local dapr environment without having to create any coupling between dapr and daprd. Since our current expectation is that people will self-manage their pluggable components we can stick with just having the socket implementation (but that does allow for future mechanisms where it makes sense).

[yaron2]

Sounds good, I think we can figure out a way to make it easier for developers to bootstrap their local dapr environment without having to create any coupling between dapr and daprd. Since our current expectation is that people will self-manage their pluggable components we can stick with just having the socket implementation (but that does allow for future mechanisms where it makes sense).

Sounds good, thanks for driving this.

[tmacam]

I did a PoC Java implementation of a pluggable State Store component in https://github.com/tmacam/DaprPluggableComponent-Java

[mcandeia]

As aligned with @johnewart, we decided to not have the socketPath attr inside pluggable component CRD for two reasons:

1. The socket would be different per component instance, not per pluggable component, hence, the relationship between pluggable component and the component instance would be 1..N, meaning N sockets could be potentially created.
2. The socket name will be defined in runtime based on the name of the pluggable component, its type, its version and the name of the component that's being used.

For the given example

apiVersion: dapr.io/v1alpha1
kind: PluggableComponent
metadata:
  name: dotnetredis
spec:
  type: state
  version: v1
---
apiVersion: dapr.io/v1alpha1
kind: Component
metadata:
  name: userstore
spec:
   type: state.dotnetredis
   version: v1

the socket path would be /var/run/dapr/state.dotnetredis-v1-userstore.sock,

notice that /var/run/dapr could be an ENV var $DAPR_PLUGGABLE_COMPONENTS_SOCKETS

thoughts @yaron2 @daixiang0 ?