diff --git a/daprdocs/content/en/concepts/overview.md b/daprdocs/content/en/concepts/overview.md index 1d19e3aad09..4a6263e7f4a 100644 --- a/daprdocs/content/en/concepts/overview.md +++ b/daprdocs/content/en/concepts/overview.md @@ -36,7 +36,7 @@ Each of these building block APIs is independent, meaning that you can use one, | Building Block | Description | |----------------|-------------| | [**Service-to-service invocation**]({{< ref "service-invocation-overview.md" >}}) | Resilient service-to-service invocation enables method calls, including retries, on remote services, wherever they are located in the supported hosting environment. -| [**State management**]({{< ref "state-management-overview.md" >}}) | With state management for storing and querying key/value pairs, long-running, highly available, stateful services can be easily written alongside stateless services in your application. The state store is pluggable and examples include AWS DynamoDB, Azure CosmosDB, Azure SQL Server, GCP Firebase, PostgreSQL or Redis, among others. +| [**State management**]({{< ref "state-management-overview.md" >}}) | With state management for storing and querying key/value pairs, long-running, highly available, stateful services can be easily written alongside stateless services in your application. The state store is pluggable and examples include AWS DynamoDB, Azure Cosmos DB, Azure SQL Server, GCP Firebase, PostgreSQL or Redis, among others. | [**Publish and subscribe**]({{< ref "pubsub-overview.md" >}}) | Publishing events and subscribing to topics between services enables event-driven architectures to simplify horizontal scalability and make them resilient to failure. Dapr provides at-least-once message delivery guarantee, message TTL, consumer groups and other advance features. | [**Resource bindings**]({{< ref "bindings-overview.md" >}}) | Resource bindings with triggers builds further on event-driven architectures for scale and resiliency by receiving and sending events to and from any external source such as databases, queues, file systems, etc. | [**Actors**]({{< ref "actors-overview.md" >}}) | A pattern for stateful and stateless objects that makes concurrency simple, with method and state encapsulation. Dapr provides many capabilities in its actor runtime, including concurrency, state, and life-cycle management for actor activation/deactivation, and timers and reminders to wake up actors. diff --git a/daprdocs/content/en/concepts/service-mesh.md b/daprdocs/content/en/concepts/service-mesh.md index 96613a31918..0e3c4cf9ef3 100644 --- a/daprdocs/content/en/concepts/service-mesh.md +++ b/daprdocs/content/en/concepts/service-mesh.md @@ -34,7 +34,6 @@ Dapr does work with service meshes. In the case where both are deployed together Watch these recordings from the Dapr community calls showing presentations on running Dapr together with different service meshes: - General overview and a demo of [Dapr and Linkerd](https://youtu.be/xxU68ewRmz8?t=142) - Demo of running [Dapr and Istio](https://youtu.be/ngIDOQApx8g?t=335) -- Learn more about [running Dapr with Open Service Mesh (OSM)]({{}}). ## When to use Dapr or a service mesh or both Should you be using Dapr, a service mesh, or both? The answer depends on your requirements. If, for example, you are looking to use Dapr for one or more building blocks such as state management or pub/sub, and you are considering using a service mesh just for network security or observability, you may find that Dapr is a good fit and that a service mesh is not required. diff --git a/daprdocs/content/en/concepts/terminology.md b/daprdocs/content/en/concepts/terminology.md index a9bd4f54ecf..80d5c89d7a2 100644 --- a/daprdocs/content/en/concepts/terminology.md +++ b/daprdocs/content/en/concepts/terminology.md @@ -16,6 +16,7 @@ This page details all of the common terms you may come across in the Dapr docs. | Configuration | A YAML file declaring all of the settings for Dapr sidecars or the Dapr control plane. This is where you can configure control plane mTLS settings, or the tracing and middleware settings for an application instance. | [Dapr configuration]({{< ref configuration-concept.md >}}) | Dapr | Distributed Application Runtime. | [Dapr overview]({{< ref overview.md >}}) | Dapr control plane | A collection of services that are part of a Dapr installation on a hosting platform such as a Kubernetes cluster. This allows Dapr-enabled applications to run on the platform and handles Dapr capabilities such as actor placement, Dapr sidecar injection, or certificate issuance/rollover. | [Self-hosted overview]({{< ref self-hosted-overview >}})
[Kubernetes overview]({{< ref kubernetes-overview >}}) +| HTTPEndpoint | HTTPEndpoint is a Dapr resource use to identify non-Dapr endpoints to invoke via the service invocation API. | [Service invocation API]({{< ref service_invocation_api.md >}}) | Self-hosted | Windows/macOS/Linux machine(s) where you can run your applications with Dapr. Dapr provides the capability to run on machines in "self-hosted" mode. | [Self-hosted mode]({{< ref self-hosted-overview.md >}}) | Service | A running application or binary. This can refer to your application or to a Dapr application. | Sidecar | A program that runs alongside your application as a separate process or container. | [Sidecar pattern](https://docs.microsoft.com/azure/architecture/patterns/sidecar) diff --git a/daprdocs/content/en/contributing/docs-contrib/maintainer-guide.md b/daprdocs/content/en/contributing/docs-contrib/maintainer-guide.md index 8810bc62609..15891b3e06f 100644 --- a/daprdocs/content/en/contributing/docs-contrib/maintainer-guide.md +++ b/daprdocs/content/en/contributing/docs-contrib/maintainer-guide.md @@ -260,7 +260,7 @@ You need Microsoft employee access to create a new Azure Static Web App. - One saying your request was received. - One saying the request was completed. 1. Back in the Azure Portal, click **Add**. You may need to click a couple times to account for DNS delay. -1. An SSL is now generated for you and the DNS record is saved. This may take 2-3 minutes. +1. A TLS certificate is now generated for you and the DNS record is saved. This may take 2-3 minutes. 1. Navigate to `https://v1-2.docs.dapr.io` and verify a blank website loads correctly. ### Configure future website branch diff --git a/daprdocs/content/en/developing-applications/building-blocks/actors/actors-overview.md b/daprdocs/content/en/developing-applications/building-blocks/actors/actors-overview.md index bda74c2a512..eed8749691e 100644 --- a/daprdocs/content/en/developing-applications/building-blocks/actors/actors-overview.md +++ b/daprdocs/content/en/developing-applications/building-blocks/actors/actors-overview.md @@ -86,6 +86,13 @@ The Dapr actor runtime provides a simple turn-based access model for accessing a - [Learn more about actor reentrancy]({{< ref "actor-reentrancy.md" >}}) - [Learn more about the turn-based access model]({{< ref "actors-features-concepts.md#turn-based-access" >}}) +### State + +Transactional state stores can be used to store actor state. To specify which state store to use for actors, specify value of property `actorStateStore` as `true` in the state store component's metadata section. Actors state is stored with a specific scheme in transactional state stores, allowing for consistent querying. Only a single state store component can be used as the state store for all actors. Read the [state API reference]({{< ref state_api.md >}}) and the [actors API reference]({{< ref actors_api.md >}}) to learn more about state stores for actors. + +#### Time to Live (TTL) on state +You should always set the TTL metadata field (`ttlInSeconds`), or the equivalent API call in your chosen SDK when saving actor state to ensure that state eventually removed. Read [actors overview]({{< ref actors-overview.md >}}) for more information. + ### Actor timers and reminders Actors can schedule periodic work on themselves by registering either timers or reminders. @@ -105,4 +112,4 @@ This distinction allows users to trade off between light-weight but stateless ti ## Related links - [Actors API reference]({{< ref actors_api.md >}}) -- Refer to the [Dapr SDK documentation and examples]({{< ref "developing-applications/sdks/#sdk-languages" >}}). \ No newline at end of file +- Refer to the [Dapr SDK documentation and examples]({{< ref "developing-applications/sdks/#sdk-languages" >}}). diff --git a/daprdocs/content/en/developing-applications/building-blocks/actors/howto-actors-partitioning.md b/daprdocs/content/en/developing-applications/building-blocks/actors/howto-actors-partitioning.md index 6356bdb9080..ad3473d9093 100644 --- a/daprdocs/content/en/developing-applications/building-blocks/actors/howto-actors-partitioning.md +++ b/daprdocs/content/en/developing-applications/building-blocks/actors/howto-actors-partitioning.md @@ -8,7 +8,7 @@ aliases: - "/developing-applications/building-blocks/actors/actors-background" --- -[Actor reminders]({{< ref "howto-actors-partitioning.md#actor-reminders" >}}) are persisted and continue to be triggered after sidecar restarts. Applications with multiple reminders registered can experience the following issues: +[Actor reminders]({{< ref "actors-timers-reminders.md#actor-reminders" >}}) are persisted and continue to be triggered after sidecar restarts. Applications with multiple reminders registered can experience the following issues: - Low throughput on reminders registration and de-registration - Limited number of reminders registered based on the single record size limit on the state store diff --git a/daprdocs/content/en/developing-applications/building-blocks/bindings/bindings-overview.md b/daprdocs/content/en/developing-applications/building-blocks/bindings/bindings-overview.md index ccc1007d24b..e98b5440edd 100644 --- a/daprdocs/content/en/developing-applications/building-blocks/bindings/bindings-overview.md +++ b/daprdocs/content/en/developing-applications/building-blocks/bindings/bindings-overview.md @@ -24,6 +24,10 @@ For example, with bindings, your microservice can respond to incoming Twilio/SMS Bindings are developed independently of Dapr runtime. You can [view and contribute to the bindings](https://github.com/dapr/components-contrib/tree/master/bindings). {{% /alert %}} +{{% alert title="Note" color="primary" %}} +If you are using the HTTP Binding, then it is preferable to use [service invocation]({{< ref service_invocation_api.md >}}) instead. Read [How-To: Invoke Non-Dapr Endpoints using HTTP]({{< ref "howto-invoke-non-dapr-endpoints.md" >}}) for more information. +{{% /alert %}} + ## Input bindings With input bindings, you can trigger your application when an event from an external resource occurs. An optional payload and metadata may be sent with the request. diff --git a/daprdocs/content/en/developing-applications/building-blocks/bindings/howto-bindings.md b/daprdocs/content/en/developing-applications/building-blocks/bindings/howto-bindings.md index a73fc863054..7a26103546f 100644 --- a/daprdocs/content/en/developing-applications/building-blocks/bindings/howto-bindings.md +++ b/daprdocs/content/en/developing-applications/building-blocks/bindings/howto-bindings.md @@ -6,6 +6,7 @@ description: "Invoke external systems with output bindings" weight: 300 --- + With output bindings, you can invoke external resources. An optional payload and metadata can be sent with the invocation request. Diagram showing bindings of example service diff --git a/daprdocs/content/en/developing-applications/building-blocks/observability/app-health.md b/daprdocs/content/en/developing-applications/building-blocks/observability/app-health.md index c137c340f01..4834bb5432b 100644 --- a/daprdocs/content/en/developing-applications/building-blocks/observability/app-health.md +++ b/daprdocs/content/en/developing-applications/building-blocks/observability/app-health.md @@ -26,9 +26,9 @@ App health checks are disabled by default. ### App health checks vs platform-level health checks -App health checks in Dapr are meant to be complementary to, and not replace, any platform-level health checks, like [liveness probes](https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/) when running on Kubernetes. +App health checks in Dapr are meant to be complementary to, and not replace, any platform-level health checks, like [liveness probes](https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/) when running on Kubernetes. -Platform-level health checks (or liveness probes) generally ensure that the application is running, and cause the platform to restart the application in case of failures. For Kubernetes, a failing App Health check won't remove a pod from service discovery. This remains the responsibility of the Kubernetes liveness probe, _not_ Dapr. +Platform-level health checks (or liveness probes) generally ensure that the application is running, and cause the platform to restart the application in case of failures. Unlike platform-level health checks, Dapr's app health checks focus on pausing work to an application that is currently unable to accept it, but is expected to be able to resume accepting work *eventually*. Goals include: - Not bringing more load to an application that is already overloaded. @@ -36,11 +36,9 @@ Unlike platform-level health checks, Dapr's app health checks focus on pausing w In this regard, Dapr's app health checks are "softer", waiting for an application to be able to process work, rather than terminating the running process in a "hard" way. -## Configuring app health checks +> Note that for Kubernetes, a failing App Health check won't remove a pod from service discovery: this remains the responsibility of the Kubernetes liveness probe, _not_ Dapr. -{{% alert title="Note" color="primary" %}} -App health checks are currently a **preview feature** and require the `AppHealthCheck` feature flag to be enabled. Refer to the documentation for [enabling preview features]({{}}). -{{% /alert %}} +## Configuring app health checks App health checks are disabled by default, but can be enabled with either: @@ -54,21 +52,21 @@ The full list of options are listed in this table: | CLI flags | Kubernetes deployment annotation | Description | Default value | | ----------------------------- | ----------------------------------- | ----------- | ------------- | | `--enable-app-health-check` | `dapr.io/enable-app-health-check` | Boolean that enables the health checks | Disabled | -| `--app-health-check-path` | `dapr.io/app-health-check-path` | Path that Dapr invokes for health probes when the app channel is HTTP (this value is ignored if the app channel is using gRPC) | `/health` | +| `--app-health-check-path` | `dapr.io/app-health-check-path` | Path that Dapr invokes for health probes when the app channel is HTTP (this value is ignored if the app channel is using gRPC) | `/healthz` | | `--app-health-probe-interval` | `dapr.io/app-health-probe-interval` | Number of *seconds* between each health probe | `5` | | `--app-health-probe-timeout` | `dapr.io/app-health-probe-timeout` | Timeout in *milliseconds* for health probe requests | `500` | | `--app-health-threshold` | `dapr.io/app-health-threshold` | Max number of consecutive failures before the app is considered unhealthy | `3` | > See the [full Dapr arguments and annotations reference]({{}}) for all options and how to enable them. -Additionally, app health checks are impacted by the protocol used for the app channel, which is configured with the `--app-protocol` flag (self-hosted) or the `dapr.io/app-protocol` annotation (Kubernetes); supported values are `http` (default) or `grpc`. +Additionally, app health checks are impacted by the protocol used for the app channel, which is configured with the `--app-protocol` flag (self-hosted) or the `dapr.io/app-protocol` annotation (Kubernetes); supported values are `http` (default), `grpc`, `https`, `grpcs`, and `h2c` (HTTP/2 Cleartext). ### Health check paths -When using HTTP for `app-protocol`, Dapr performs health probes by making an HTTP call to the path specified in `app-health-check-path`, which is `/health` by default. +When using HTTP (including `http`, `https`, and `h2c`) for `app-protocol`, Dapr performs health probes by making an HTTP call to the path specified in `app-health-check-path`, which is `/health` by default. For your app to be considered healthy, the response must have an HTTP status code in the 200-299 range. Any other status code is considered a failure. Dapr is only concerned with the status code of the response, and ignores any response header or body. -When using gRPC for the app channel, Dapr invokes the method `/dapr.proto.runtime.v1.AppCallbackHealthCheck/HealthCheck` in your application. Most likely, you will use a Dapr SDK to implement the handler for this method. +When using gRPC for the app channel (`app-protocol` set to `grpc` or `grpcs`), Dapr invokes the method `/dapr.proto.runtime.v1.AppCallbackHealthCheck/HealthCheck` in your application. Most likely, you will use a Dapr SDK to implement the handler for this method. While responding to a health probe request, your app *may* decide to perform additional internal health checks to determine if it's ready to process work from the Dapr runtime. However, this is not required; it's a choice that depends on your application's needs. @@ -88,8 +86,6 @@ Thresholds only apply to failures. A single successful response is enough for Da ## Example -Because app health checks are currently a preview feature, make sure to enable the `AppHealthCheck` feature flag. Refer to the documentation for [enabling preview features]({{}}) before following the examples below. - {{< tabs "Self-Hosted (CLI)" Kubernetes >}} {{% codetab %}} diff --git a/daprdocs/content/en/developing-applications/building-blocks/service-invocation/howto-invoke-discover-services.md b/daprdocs/content/en/developing-applications/building-blocks/service-invocation/howto-invoke-discover-services.md index ba3740f0a41..6e70fe765dd 100644 --- a/daprdocs/content/en/developing-applications/building-blocks/service-invocation/howto-invoke-discover-services.md +++ b/daprdocs/content/en/developing-applications/building-blocks/service-invocation/howto-invoke-discover-services.md @@ -19,7 +19,7 @@ This article demonstrates how to deploy services each with an unique application Dapr allows you to assign a global, unique ID for your app. This ID encapsulates the state for your application, regardless of the number of instances it may have. -{{< tabs Dotnet Java Python Go Javascript Kubernetes>}} +{{< tabs Dotnet Java Python Go JavaScript Kubernetes>}} {{% codetab %}} @@ -31,13 +31,13 @@ dapr run --app-id orderprocessing --app-port 6001 --dapr-http-port 3601 --dapr-g ``` -If your app uses an SSL connection, you can tell Dapr to invoke your app over an insecure SSL connection: +If your app uses a TLS, you can tell Dapr to invoke your app over a TLS connection by setting `--app-protocol https`: ```bash -dapr run --app-id checkout --app-port 6002 --dapr-http-port 3602 --dapr-grpc-port 60002 --app-ssl dotnet run +dapr run --app-id checkout --app-port 6002 --dapr-http-port 3602 --dapr-grpc-port 60002 --app-protocol https dotnet run -dapr run --app-id orderprocessing --app-port 6001 --dapr-http-port 3601 --dapr-grpc-port 60001 --app-ssl dotnet run +dapr run --app-id orderprocessing --app-port 6001 --dapr-http-port 3601 --dapr-grpc-port 60001 --app-protocol https dotnet run ``` @@ -53,13 +53,13 @@ dapr run --app-id orderprocessing --app-port 6001 --dapr-http-port 3601 --dapr-g ``` -If your app uses an SSL connection, you can tell Dapr to invoke your app over an insecure SSL connection: +If your app uses a TLS, you can tell Dapr to invoke your app over a TLS connection by setting `--app-protocol https`: ```bash -dapr run --app-id checkout --app-port 6002 --dapr-http-port 3602 --dapr-grpc-port 60002 --app-ssl mvn spring-boot:run +dapr run --app-id checkout --app-port 6002 --dapr-http-port 3602 --dapr-grpc-port 60002 --app-protocol https mvn spring-boot:run -dapr run --app-id orderprocessing --app-port 6001 --dapr-http-port 3601 --dapr-grpc-port 60001 --app-ssl mvn spring-boot:run +dapr run --app-id orderprocessing --app-port 6001 --dapr-http-port 3601 --dapr-grpc-port 60001 --app-protocol https mvn spring-boot:run ``` @@ -75,13 +75,13 @@ dapr run --app-id orderprocessing --app-port 6001 --dapr-http-port 3601 --dapr-g ``` -If your app uses an SSL connection, you can tell Dapr to invoke your app over an insecure SSL connection: +If your app uses a TLS, you can tell Dapr to invoke your app over a TLS connection by setting `--app-protocol https`: ```bash -dapr run --app-id checkout --app-port 6002 --dapr-http-port 3602 --dapr-grpc-port 60002 --app-ssl -- python3 CheckoutService.py +dapr run --app-id checkout --app-port 6002 --dapr-http-port 3602 --dapr-grpc-port 60002 --app-protocol https -- python3 CheckoutService.py -dapr run --app-id orderprocessing --app-port 6001 --dapr-http-port 3601 --dapr-grpc-port 60001 --app-ssl -- python3 OrderProcessingService.py +dapr run --app-id orderprocessing --app-port 6001 --dapr-http-port 3601 --dapr-grpc-port 60001 --app-protocol https -- python3 OrderProcessingService.py ``` @@ -97,13 +97,13 @@ dapr run --app-id orderprocessing --app-port 6001 --dapr-http-port 3601 --dapr-g ``` -If your app uses an SSL connection, you can tell Dapr to invoke your app over an insecure SSL connection: +If your app uses a TLS, you can tell Dapr to invoke your app over a TLS connection by setting `--app-protocol https`: ```bash -dapr run --app-id checkout --app-port 6002 --dapr-http-port 3602 --dapr-grpc-port 60002 --app-ssl go run CheckoutService.go +dapr run --app-id checkout --app-port 6002 --dapr-http-port 3602 --dapr-grpc-port 60002 --app-protocol https go run CheckoutService.go -dapr run --app-id orderprocessing --app-port 6001 --dapr-http-port 3601 --dapr-grpc-port 60001 --app-ssl go run OrderProcessingService.go +dapr run --app-id orderprocessing --app-port 6001 --dapr-http-port 3601 --dapr-grpc-port 60001 --app-protocol https go run OrderProcessingService.go ``` @@ -119,13 +119,13 @@ dapr run --app-id orderprocessing --app-port 6001 --dapr-http-port 3601 --dapr-g ``` -If your app uses an SSL connection, you can tell Dapr to invoke your app over an insecure SSL connection: +If your app uses a TLS, you can tell Dapr to invoke your app over a TLS connection by setting `--app-protocol https`: ```bash -dapr run --app-id checkout --app-port 6002 --dapr-http-port 3602 --dapr-grpc-port 60002 --app-ssl npm start +dapr run --app-id checkout --app-port 6002 --dapr-http-port 3602 --dapr-grpc-port 60002 --app-protocol https npm start -dapr run --app-id orderprocessing --app-port 6001 --dapr-http-port 3601 --dapr-grpc-port 60001 --app-ssl npm start +dapr run --app-id orderprocessing --app-port 6001 --dapr-http-port 3601 --dapr-grpc-port 60001 --app-protocol https npm start ``` @@ -161,7 +161,7 @@ spec: ... ``` -*If your app uses an SSL connection, you can tell Dapr to invoke your app over an insecure SSL connection with the `app-ssl: "true"` annotation (full list [here]({{< ref arguments-annotations-overview.md >}}))* +If your app uses a TLS connection, you can tell Dapr to invoke your app over TLS with the `app-protocol: "https"` annotation (full list [here]({{< ref arguments-annotations-overview.md >}})). Note that Dapr does not validate TLS certificates presented by the app. {{% /codetab %}} diff --git a/daprdocs/content/en/developing-applications/building-blocks/service-invocation/howto-invoke-non-dapr-endpoints.md b/daprdocs/content/en/developing-applications/building-blocks/service-invocation/howto-invoke-non-dapr-endpoints.md new file mode 100644 index 00000000000..8196bd0b1e5 --- /dev/null +++ b/daprdocs/content/en/developing-applications/building-blocks/service-invocation/howto-invoke-non-dapr-endpoints.md @@ -0,0 +1,92 @@ +--- +type: docs +title: "How-To: Invoke Non-Dapr Endpoints using HTTP" +linkTitle: "How-To: Invoke Non-Dapr Endpoints" +description: "Call Non-Dapr endpoints from Dapr applications using service invocation" +weight: 2000 +--- + +This article demonstrates how to call a non-Dapr endpoint using Dapr over HTTP. + +Using Dapr's service invocation API, you can communicate with endpoints that either use or do not use Dapr. Using Dapr to call endpoints that do not use Dapr not only provides a consistent API, but also the following [Dapr service invocation]({{< ref service-invocation-overview.md >}}) benefits: + +- Ability to apply resiliency policies +- Call observability with tracing & metrics +- Security access control through scoping +- Ability to utilize middleware pipeline components +- Service discovery +- Authentication through the use of headers + +## HTTP service invocation to external services or non-Dapr endpoints +Sometimes you need to call a non-Dapr HTTP endpoint. For example: +- You may choose to only use Dapr in part of your overall application, including brownfield development +- You may not have access to the code to migrate an existing application to use Dapr +- You need to call an external HTTP service. + +By defining an `HTTPEndpoint` resource, you declaratively define a way to interact with a non-Dapr endpoint. You then use the service invocation URL to invoke non-Dapr endpoints. Alternatively, you can place a non-Dapr Fully Qualified Domain Name (FQDN) endpoint URL directly into the service invocation URL. + +### Order of precedence between HttpEndpoint, FQDN URL, and appId +When using service invocation, the Dapr runtime follows a precedence order: + +1. Is this a named `HTTPEndpoint` resource? +2. Is this an FQDN URL with an`http://` or `https://` prefix? +3. Is this an `appID`? + +## Service invocation and non-Dapr HTTP endpoint +The diagram below is an overview of how Dapr's service invocation works when invoking non-Dapr endpoints. + +Diagram showing the steps of service invocation to non-Dapr endpoints + +1. Service A makes an HTTP call targeting Service B, a non-Dapr endpoint. The call goes to the local Dapr sidecar. +2. Dapr discovers Service B's location using the `HTTPEndpoint` or FQDN URL. +3. Dapr forwards the message to Service B. +4. Service B runs its business logic code. +5. Service B sends a response to Service A's Dapr sidecar. +6. Service A receives the response. + +## Using an HTTPEndpoint resource or FQDN URL for non-Dapr endpoints +There are two ways to invoke a non-Dapr endpoint when communicating either to Dapr applications or non-Dapr applications. A Dapr application can invoke a non-Dapr endpoint by providing one of the following: + +- A named `HTTPEndpoint` resource, including defining an `HTTPEndpoint` resource type. See the [HTTPEndpoint reference]({{< ref httpendpoints-reference.md >}}) guide for an example. + + ```sh + localhost:3500/v1.0/invoke//method/ + ``` + + For example, with an `HTTPEndpoint` resource called "palpatine" and a method called "Order66", this would be: + ```sh + curl http://localhost:3500/v1.0/invoke/palpatine/method/order66 + ``` + +- A FQDN URL to the non-Dapr endpoint. + + ```sh + localhost:3500/v1.0/invoke//method/ + ``` + + For example, with an FQDN resource called `https://darthsidious.starwars`, this would be: + ```sh + curl http://localhost:3500/v1.0/invoke/https://darthsidious.starwars/method/order66 + ``` + +### Using appId when calling Dapr enabled applications +AppIDs are always used to call Dapr applications with the `appID` and `my-method. Read the [How-To: Invoke services using HTTP]({{< ref howto-invoke-discover-services.md >}}) guide for more information. For example: + +```sh +localhost:3500/v1.0/invoke//method/ +``` +```sh +curl http://localhost:3602/v1.0/invoke/orderprocessor/method/checkout +``` + +## Related Links + +- [HTTPEndpoint reference]({{< ref httpendpoints-reference.md >}}) +- [Service invocation overview]({{< ref service-invocation-overview.md >}}) +- [Service invocation API specification]({{< ref service_invocation_api.md >}}) + +## Community call demo +Watch this [video](https://youtu.be/BEXJgLsO4hA?t=364) on how to use service invocation to call non-Dapr endpoints. +
+ +
diff --git a/daprdocs/content/en/developing-applications/building-blocks/service-invocation/howto-invoke-services-grpc.md b/daprdocs/content/en/developing-applications/building-blocks/service-invocation/howto-invoke-services-grpc.md index 6373ed1d480..fc953f6034d 100644 --- a/daprdocs/content/en/developing-applications/building-blocks/service-invocation/howto-invoke-services-grpc.md +++ b/daprdocs/content/en/developing-applications/building-blocks/service-invocation/howto-invoke-services-grpc.md @@ -220,10 +220,11 @@ spec: dapr.io/app-port: "50051" ... ``` -*If your app uses an SSL connection, you can tell Dapr to invoke your app over an insecure SSL connection with the `app-ssl: "true"` annotation (full list [here]({{< ref arguments-annotations-overview.md >}}))* The `dapr.io/app-protocol: "grpc"` annotation tells Dapr to invoke the app using gRPC. +If your app uses a TLS connection, you can tell Dapr to invoke your app over TLS with the `app-protocol: "grpcs"` annotation (full list [here]({{< ref arguments-annotations-overview.md >}})). Note that Dapr does not validate TLS certificates presented by the app. + ### Namespaces When running on [namespace supported platforms]({{< ref "service_invocation_api.md#namespace-supported-platforms" >}}), you include the namespace of the target app in the app ID: `myApp.production` diff --git a/daprdocs/content/en/developing-applications/building-blocks/service-invocation/service-invocation-overview.md b/daprdocs/content/en/developing-applications/building-blocks/service-invocation/service-invocation-overview.md index 0c4d3bc41a3..4d2ce7a2c43 100644 --- a/daprdocs/content/en/developing-applications/building-blocks/service-invocation/service-invocation-overview.md +++ b/daprdocs/content/en/developing-applications/building-blocks/service-invocation/service-invocation-overview.md @@ -25,7 +25,7 @@ Dapr uses a sidecar architecture. To invoke an application using Dapr: - Each application communicates with its own instance of Dapr. - The Dapr instances discover and communicate with each other. -The diagram below is an overview of how Dapr's service invocation works. +The diagram below is an overview of how Dapr's service invocation works between two Dapr-ized applications. Diagram showing the steps of service invocation @@ -38,8 +38,10 @@ The diagram below is an overview of how Dapr's service invocation works. 6. Dapr forwards the response to Service A's Dapr sidecar. 7. Service A receives the response. +You can also call non-Dapr HTTP endpoints using the service invocation API. For example, you may only use Dapr in part of an overall application, may not have access to the code to migrate an existing application to use Dapr, or simply need to call an external HTTP service. Read ["How-To: Invoke Non-Dapr Endpoints using HTTP"]({{< ref howto-invoke-non-dapr-endpoints.md >}}) for more information. + ## Features -Service invocation provides several features to make it easy for you to call methods between applications. +Service invocation provides several features to make it easy for you to call methods between applications or to call external HTTP endpoints. ### HTTP and gRPC service invocation - **HTTP**: If you're already using HTTP protocols in your application, using the Dapr HTTP header might be the easiest way to get started. You don't need to change your existing endpoint URLs; just add the `dapr-app-id` header and you're ready to go. For more information, see [Invoke Services using HTTP]({{< ref howto-invoke-discover-services.md >}}). diff --git a/daprdocs/content/en/developing-applications/building-blocks/state-management/state-management-overview.md b/daprdocs/content/en/developing-applications/building-blocks/state-management/state-management-overview.md index f9f5813f0ea..a7dacc36194 100644 --- a/daprdocs/content/en/developing-applications/building-blocks/state-management/state-management-overview.md +++ b/daprdocs/content/en/developing-applications/building-blocks/state-management/state-management-overview.md @@ -93,6 +93,9 @@ You can group write, update, and delete operations into a request, which are the Transactional state stores can be used to store actor state. To specify which state store to use for actors, specify value of property `actorStateStore` as `true` in the state store component's metadata section. Actors state is stored with a specific scheme in transactional state stores, allowing for consistent querying. Only a single state store component can be used as the state store for all actors. Read the [state API reference]({{< ref state_api.md >}}) and the [actors API reference]({{< ref actors_api.md >}}) to learn more about state stores for actors. +#### Time to Live (TTL) on actor state +You should always set the TTL metadata field (`ttlInSeconds`), or the equivalent API call in your chosen SDK when saving actor state to ensure that state eventually removed. Read [actors overview]({{< ref actors-overview.md >}}) for more information. + ### State encryption Dapr supports automatic client encryption of application state with support for key rotations. This is supported on all Dapr state stores. For more info, read the [How-To: Encrypt application state]({{< ref howto-encrypt-state.md >}}) topic. @@ -178,4 +181,4 @@ Want to skip the quickstarts? Not a problem. You can try out the state managemen - [How-To: Build a stateful service]({{< ref howto-stateful-service.md >}}) - Review the list of [state store components]({{< ref supported-state-stores.md >}}) - Read the [state management API reference]({{< ref state_api.md >}}) -- Read the [actors API reference]({{< ref actors_api.md >}}) \ No newline at end of file +- Read the [actors API reference]({{< ref actors_api.md >}}) diff --git a/daprdocs/content/en/developing-applications/building-blocks/state-management/state-store-ttl.md b/daprdocs/content/en/developing-applications/building-blocks/state-management/state-store-ttl.md index 49e00d79f4d..4717a1889b2 100644 --- a/daprdocs/content/en/developing-applications/building-blocks/state-management/state-store-ttl.md +++ b/daprdocs/content/en/developing-applications/building-blocks/state-management/state-store-ttl.md @@ -28,7 +28,7 @@ Refer to the TTL column in the [state store components guide]({{< ref supported- You can set state TTL in the metadata as part of the state store set request: -{{< tabs ".NET" Python Go "HTTP API (Bash)" "HTTP API (PowerShell)">}} +{{< tabs Python ".NET" Go "HTTP API (Bash)" "HTTP API (PowerShell)">}} {{% codetab %}} diff --git a/daprdocs/content/en/developing-applications/building-blocks/workflow/howto-author-workflow.md b/daprdocs/content/en/developing-applications/building-blocks/workflow/howto-author-workflow.md index b9f7d260750..b6191d23de3 100644 --- a/daprdocs/content/en/developing-applications/building-blocks/workflow/howto-author-workflow.md +++ b/daprdocs/content/en/developing-applications/building-blocks/workflow/howto-author-workflow.md @@ -28,19 +28,88 @@ The Dapr sidecar doesn’t load any workflow definitions. Rather, the sidecar si ## Write the workflow activities -Define the workflow activities you'd like your workflow to perform. Activities are a class definition and can take inputs and outputs. Activities also participate in dependency injection, like binding to a Dapr client. +[Workflow activities]({{< ref "workflow-features-concepts.md#workflow-activites" >}}) are the basic unit of work in a workflow and are the tasks that get orchestrated in the business process. {{< tabs ".NET" >}} {{% codetab %}} -Continuing the ASP.NET order processing example, the `OrderProcessingWorkflow` class is derived from a base class called `Workflow` with input and output parameter types. +Define the workflow activities you'd like your workflow to perform. Activities are a class definition and can take inputs and outputs. Activities also participate in dependency injection, like binding to a Dapr client. -It also includes a `RunAsync` method that does the heavy lifting of the workflow and calls the workflow activities. The activities called in the example are: +The activities called in the example below are: - `NotifyActivity`: Receive notification of a new order. - `ReserveInventoryActivity`: Check for sufficient inventory to meet the new order. - `ProcessPaymentActivity`: Process payment for the order. Includes `NotifyActivity` to send notification of successful order. +### NotifyActivity + +```csharp +public class NotifyActivity : WorkflowActivity +{ + //... + + public NotifyActivity(ILoggerFactory loggerFactory) + { + this.logger = loggerFactory.CreateLogger(); + } + + //... +} +``` + +[See the full `NotifyActivity.cs` workflow activity example.](https://github.com/dapr/dotnet-sdk/blob/master/examples/Workflow/WorkflowConsoleApp/Activities/NotifyActivity.cs) + +### ReserveInventoryActivity + +```csharp +public class ReserveInventoryActivity : WorkflowActivity +{ + //... + + public ReserveInventoryActivity(ILoggerFactory loggerFactory, DaprClient client) + { + this.logger = loggerFactory.CreateLogger(); + this.client = client; + } + + //... + +} +``` +[See the full `ReserveInventoryActivity.cs` workflow activity example.](https://github.com/dapr/dotnet-sdk/blob/master/examples/Workflow/WorkflowConsoleApp/Activities/ReserveInventoryActivity.cs) + +### ProcessPaymentActivity + +```csharp +public class ProcessPaymentActivity : WorkflowActivity +{ + //... + public ProcessPaymentActivity(ILoggerFactory loggerFactory) + { + this.logger = loggerFactory.CreateLogger(); + } + + //... + +} +``` + +[See the full `ProcessPaymentActivity.cs` workflow activity example.](https://github.com/dapr/dotnet-sdk/blob/master/examples/Workflow/WorkflowConsoleApp/Activities/ProcessPaymentActivity.cs) + +{{% /codetab %}} + +{{< /tabs >}} + +## Write the workflow + +Next, register and call the activites in a workflow. + +{{< tabs ".NET" >}} + +{{% codetab %}} + +The `OrderProcessingWorkflow` class is derived from a base class called `Workflow` with input and output parameter types. It also includes a `RunAsync` method that does the heavy lifting of the workflow and calls the workflow activities. + ```csharp class OrderProcessingWorkflow : Workflow { @@ -73,19 +142,21 @@ It also includes a `RunAsync` method that does the heavy lifting of the workflow } ``` +[See the full workflow example in `OrderProcessingWorkflow.cs`.](https://github.com/dapr/dotnet-sdk/blob/master/examples/Workflow/WorkflowConsoleApp/Workflows/OrderProcessingWorkflow.cs) + {{% /codetab %}} {{< /tabs >}} -## Write the workflow +## Write the application -Compose the workflow activities into a workflow. +Finally, compose the application using the workflow. {{< tabs ".NET" >}} {{% codetab %}} -[In the following example](https://github.com/dapr/dotnet-sdk/blob/master/examples/Workflow/WorkflowConsoleApp/Program.cs), for a basic ASP.NET order processing application using the .NET SDK, your project code would include: +[In the following `Program.cs` example](https://github.com/dapr/dotnet-sdk/blob/master/examples/Workflow/WorkflowConsoleApp/Program.cs), for a basic ASP.NET order processing application using the .NET SDK, your project code would include: - A NuGet package called `Dapr.Workflow` to receive the .NET SDK capabilities - A builder with an extension method called `AddDaprWorkflow` diff --git a/daprdocs/content/en/developing-applications/building-blocks/workflow/howto-manage-workflow.md b/daprdocs/content/en/developing-applications/building-blocks/workflow/howto-manage-workflow.md index 34aa106034e..7b2af68b477 100644 --- a/daprdocs/content/en/developing-applications/building-blocks/workflow/howto-manage-workflow.md +++ b/daprdocs/content/en/developing-applications/building-blocks/workflow/howto-manage-workflow.md @@ -21,16 +21,27 @@ string workflowComponent = "dapr"; string workflowName = "OrderProcessingWorkflow"; OrderPayload input = new OrderPayload("Paperclips", 99.95); Dictionary workflowOptions; // This is an optional parameter -CancellationToken cts = CancellationToken.None; -// Start the workflow. This returns back a "WorkflowReference" which contains the instanceID for the particular workflow instance. -WorkflowReference startResponse = await daprClient.StartWorkflowAsync(orderId, workflowComponent, workflowName, input, workflowOptions, cts); +// Start the workflow. This returns back a "StartWorkflowResponse" which contains the instance ID for the particular workflow instance. +StartWorkflowResponse startResponse = await daprClient.StartWorkflowAsync(orderId, workflowComponent, workflowName, input, workflowOptions); -// Get information on the workflow. This response will contain information such as the status of the workflow, when it started, and more! +// Get information on the workflow. This response contains information such as the status of the workflow, when it started, and more! GetWorkflowResponse getResponse = await daprClient.GetWorkflowAsync(orderId, workflowComponent, workflowName); // Terminate the workflow -await daprClient.TerminateWorkflowAsync(instanceId, workflowComponent); +await daprClient.TerminateWorkflowAsync(orderId, workflowComponent); + +// Raise an event (an incoming purchase order) that your workflow will wait for. This returns the item waiting to be purchased. +await daprClient.RaiseWorkflowEventAsync(orderId, workflowComponent, workflowName, input); + +// Pause +await daprClient.PauseWorkflowAsync(orderId, workflowComponent); + +// Resume +await daprClient.ResumeWorkflowAsync(orderId, workflowComponent); + +// Purge +await daprClient.PurgeWorkflowAsync(orderId, workflowComponent); ``` {{% /codetab %}} @@ -44,7 +55,7 @@ Manage your workflow using HTTP calls. The example below plugs in the properties To start your workflow with an ID `12345678`, run: -```bash +```http POST http://localhost:3500/v1.0-alpha1/workflows/dapr/OrderProcessingWorkflow/start?instanceID=12345678 ``` @@ -54,15 +65,49 @@ Note that workflow instance IDs can only contain alphanumeric characters, unders To terminate your workflow with an ID `12345678`, run: -```bash +```http POST http://localhost:3500/v1.0-alpha1/workflows/dapr/12345678/terminate ``` +### Raise an event + +For workflow components that support subscribing to external events, such as the Dapr Workflow engine, you can use the following "raise event" API to deliver a named event to a specific workflow instance. + +```http +POST http://localhost:3500/v1.0-alpha1/workflows///raiseEvent/ +``` + +> An `eventName` can be any function. + +### Pause or resume a workflow + +To plan for down-time, wait for inputs, and more, you can pause and then resume a workflow. To pause a workflow with an ID `12345678` until triggered to resume, run: + +```http +POST http://localhost:3500/v1.0-alpha1/workflows/dapr/12345678/pause +``` + +To resume a workflow with an ID `12345678`, run: + +```http +POST http://localhost:3500/v1.0-alpha1/workflows/dapr/12345678/resume +``` + +### Purge a workflow + +The purge API can be used to permanently delete workflow metadata from the underlying state store, including any stored inputs, outputs, and workflow history records. This is often useful for implementing data retention policies and for freeing resources. + +Only workflow instances in the COMPLETED, FAILED, or TERMINATED state can be purged. If the workflow is in any other state, calling purge returns an error. + +```http +POST http://localhost:3500/v1.0-alpha1/workflows/dapr/12345678/purge +``` + ### Get information about a workflow To fetch workflow information (outputs and inputs) with an ID `12345678`, run: -```bash +```http GET http://localhost:3500/v1.0-alpha1/workflows/dapr/12345678 ``` diff --git a/daprdocs/content/en/developing-applications/building-blocks/workflow/workflow-features-concepts.md b/daprdocs/content/en/developing-applications/building-blocks/workflow/workflow-features-concepts.md index 1ae7c3dd81b..45649967692 100644 --- a/daprdocs/content/en/developing-applications/building-blocks/workflow/workflow-features-concepts.md +++ b/daprdocs/content/en/developing-applications/building-blocks/workflow/workflow-features-concepts.md @@ -105,6 +105,36 @@ Dapr Workflows allow you to schedule reminder-like durable delays for any time r Some APIs in the workflow authoring SDK may internally schedule durable timers to implement internal timeout behavior. {{% /alert %}} +## Retry policies + +Workflows support durable retry policies for activities and child workflows. Workflow retry policies are separate and distinct from [Dapr resiliency policies]({{< ref "resiliency-overview.md" >}}) in the following ways. + +- Workflow retry policies are configured by the workflow author in code, whereas Dapr Resiliency policies are configured by the application operator in YAML. +- Workflow retry policies are durable and maintain their state across application restarts, whereas Dapr Resiliency policies are not durable and must be re-applied after application restarts. +- Workflow retry policies are triggered by unhandled errors/exceptions in activities and child workflows, whereas Dapr Resiliency policies are triggered by operation timeouts and connectivity faults. + +Retries are internally implemented using durable timers. This means that workflows can be safely unloaded from memory while waiting for a retry to fire, conserving system resources. This also means that delays between retries can be arbitrarily long, including minutes, hours, or even days. + +{{% alert title="Note" color="primary" %}} +The actions performed by a retry policy are saved into a workflow's history. Care must be taken not to change the behavior of a retry policy after a workflow has already been executed. Otherwise, the workflow may behave unexpectedly when replayed. See the notes on [updating workflow code]({{< ref "#updating-workflow-code" >}}) for more information. +{{% /alert %}} + +It's possible to use both workflow retry policies and Dapr Resiliency policies together. For example, if a workflow activity uses a Dapr client to invoke a service, the Dapr client uses the configured resiliency policy. See [Quickstart: Service-to-service resiliency]({{< ref "#resiliency-serviceinvo-quickstart" >}}) for more information with an example. However, if the activity itself fails for any reason, including exhausting the retries on the resiliency policy, then the workflow's resiliency policy kicks in. + +{{% alert title="Note" color="primary" %}} +Using workflow retry policies and resiliency policies together can result in unexpected behavior. For example, if a workflow activity exhausts its configured retry policy, the workflow engine will still retry the activity according to the workflow retry policy. This can result in the activity being retried more times than expected. +{{% /alert %}} + +Because workflow retry policies are configured in code, the exact developer experience may vary depending on the version of the workflow SDK. In general, workflow retry policies can be configured with the following parameters. + +| Parameter | Description | +| --- | --- | +| **Maximum number of attempts** | The maximum number of times to execute the activity or child workflow. | +| **First retry interval** | The amount of time to wait before the first retry. | +| **Backoff coefficient** | The amount of time to wait before each subsequent retry. | +| **Maximum retry interval** | The maximum amount of time to wait before each subsequent retry. | +| **Retry timeout** | The overall timeout for retries, regardless of any configured max number of attempts. | + ## External events Sometimes workflows will need to wait for events that are raised by external systems. For example, an approval workflow may require a human to explicitly approve an order request within an order processing workflow if the total cost exceeds some threshold. Another example is a trivia game orchestration workflow that pauses while waiting for all participants to submit their answers to trivia questions. These mid-execution inputs are referred to as _external events_. diff --git a/daprdocs/content/en/developing-applications/building-blocks/workflow/workflow-patterns.md b/daprdocs/content/en/developing-applications/building-blocks/workflow/workflow-patterns.md index e82217874f2..5140042789f 100644 --- a/daprdocs/content/en/developing-applications/building-blocks/workflow/workflow-patterns.md +++ b/daprdocs/content/en/developing-applications/building-blocks/workflow/workflow-patterns.md @@ -31,25 +31,27 @@ Dapr Workflow solves these complexities by allowing you to implement the task ch ```csharp // Expotential backoff retry policy that survives long outages -var retryPolicy = TaskOptions.FromRetryPolicy(new RetryPolicy( - maxNumberOfAttempts: 10, - firstRetryInterval: TimeSpan.FromMinutes(1), - backoffCoefficient: 2.0, - maxRetryInterval: TimeSpan.FromHours(1))); +var retryOptions = new WorkflowTaskOptions +{ + RetryPolicy = new WorkflowRetryPolicy( + firstRetryInterval: TimeSpan.FromMinutes(1), + backoffCoefficient: 2.0, + maxRetryInterval: TimeSpan.FromHours(1), + maxNumberOfAttempts: 10), +}; -// Task failures are surfaced as ordinary .NET exceptions try { - var result1 = await context.CallActivityAsync("Step1", wfInput, retryPolicy); - var result2 = await context.CallActivityAsync("Step2", result1, retryPolicy); - var result3 = await context.CallActivityAsync("Step3", result2, retryPolicy); - var result4 = await context.CallActivityAsync("Step4", result3, retryPolicy); + var result1 = await context.CallActivityAsync("Step1", wfInput, retryOptions); + var result2 = await context.CallActivityAsync("Step2", result1, retryOptions); + var result3 = await context.CallActivityAsync("Step3", result2, retryOptions); + var result4 = await context.CallActivityAsync("Step4", result3, retryOptions); return string.Join(", ", result4); } -catch (TaskFailedException) +catch (TaskFailedException) // Task failures are surfaced as TaskFailedException { // Retries expired - apply custom compensation logic - await context.CallActivityAsync("MyCompensation", options: retryPolicy); + await context.CallActivityAsync("MyCompensation", options: retryOptions); throw; } ``` @@ -143,35 +145,33 @@ The Dapr workflow HTTP API supports the asynchronous request-reply pattern out-o The following `curl` commands illustrate how the workflow APIs support this pattern. ```bash -curl -X POST http://localhost:3500/v1.0-alpha1/workflows/dapr/OrderProcessingWorkflow/12345678/start -d '{"input":{"Name":"Paperclips","Quantity":1,"TotalCost":9.95}}' +curl -X POST http://localhost:3500/v1.0-alpha1/workflows/dapr/OrderProcessingWorkflow/start?instanceID=12345678 -d '{"Name":"Paperclips","Quantity":1,"TotalCost":9.95}' ``` The previous command will result in the following response JSON: ```json -{"instance_id":"12345678"} +{"instanceID":"12345678"} ``` The HTTP client can then construct the status query URL using the workflow instance ID and poll it repeatedly until it sees the "COMPLETE", "FAILURE", or "TERMINATED" status in the payload. ```bash -curl http://localhost:3500/v1.0-alpha1/workflows/dapr/OrderProcessingWorkflow/12345678 +curl http://localhost:3500/v1.0-alpha1/workflows/dapr/12345678 ``` The following is an example of what an in-progress workflow status might look like. ```json { - "WFInfo": { - "instance_id": "12345678" - }, - "start_time": "2023-02-05T00:32:05Z", - "metadata": { + "instanceID": "12345678", + "workflowName": "OrderProcessingWorkflow", + "createdAt": "2023-05-03T23:22:11.143069826Z", + "lastUpdatedAt": "2023-05-03T23:22:22.460025267Z", + "runtimeStatus": "RUNNING", + "properties": { "dapr.workflow.custom_status": "", - "dapr.workflow.input": "{\"Name\":\"Paperclips\",\"Quantity\":1,\"TotalCost\":9.95}", - "dapr.workflow.last_updated": "2023-02-05T00:32:18Z", - "dapr.workflow.name": "OrderProcessingWorkflow", - "dapr.workflow.runtime_status": "RUNNING" + "dapr.workflow.input": "{\"Name\":\"Paperclips\",\"Quantity\":1,\"TotalCost\":9.95}" } } ``` @@ -182,17 +182,15 @@ If the workflow has completed, the status might look as follows. ```json { - "WFInfo": { - "instance_id": "12345678" - }, - "start_time": "2023-02-05T00:32:05Z", - "metadata": { + "instanceID": "12345678", + "workflowName": "OrderProcessingWorkflow", + "createdAt": "2023-05-03T23:30:11.381146313Z", + "lastUpdatedAt": "2023-05-03T23:30:52.923870615Z", + "runtimeStatus": "COMPLETED", + "properties": { "dapr.workflow.custom_status": "", "dapr.workflow.input": "{\"Name\":\"Paperclips\",\"Quantity\":1,\"TotalCost\":9.95}", - "dapr.workflow.last_updated": "2023-02-05T00:32:23Z", - "dapr.workflow.name": "OrderProcessingWorkflow", - "dapr.workflow.output": "{\"Processed\":true}", - "dapr.workflow.runtime_status": "COMPLETED" + "dapr.workflow.output": "{\"Processed\":true}" } } ``` diff --git a/daprdocs/content/en/developing-applications/integrations/Azure/_index.md b/daprdocs/content/en/developing-applications/integrations/Azure/_index.md index 229226cc6a8..895ae03a34a 100644 --- a/daprdocs/content/en/developing-applications/integrations/Azure/_index.md +++ b/daprdocs/content/en/developing-applications/integrations/Azure/_index.md @@ -2,6 +2,6 @@ type: docs title: "Integrations with Azure" linkTitle: "Azure" -weight: 1500 +weight: 1000 description: "Dapr integrations with Azure services" --- \ No newline at end of file diff --git a/daprdocs/content/en/developing-applications/integrations/Azure/authenticating-azure.md b/daprdocs/content/en/developing-applications/integrations/Azure/authenticating-azure.md deleted file mode 100644 index 93e0e374c08..00000000000 --- a/daprdocs/content/en/developing-applications/integrations/Azure/authenticating-azure.md +++ /dev/null @@ -1,401 +0,0 @@ ---- -type: docs -title: "Authenticating to Azure" -linkTitle: "Authenticating to Azure" -description: "How to authenticate Azure components using Azure AD and/or Managed Identities" -aliases: - - "/operations/components/setup-secret-store/supported-secret-stores/azure-keyvault-managed-identity/" - - "/reference/components-reference/supported-secret-stores/azure-keyvault-managed-identity/" -weight: 1000 ---- - -## Common Azure authentication layer - -Certain Azure components for Dapr offer support for the *common Azure authentication layer*, which enables applications to access data stored in Azure resources by authenticating with Azure AD. Thanks to this, administrators can leverage all the benefits of fine-tuned permissions with RBAC (Role-Based Access Control), and applications running on certain Azure services such as Azure VMs, Azure Kubernetes Service, or many Azure platform services can leverage [Managed Service Identities (MSI)](https://docs.microsoft.com/azure/active-directory/managed-identities-azure-resources/overview). - -Some Azure components offer alternative authentication methods, such as systems based on "master keys" or "shared keys". Whenever possible, it is recommended that you authenticate your Dapr components using Azure AD for increased security and ease of management, as well as for the ability to leverage MSI if your app is running on supported Azure services. - -> Currently, only a subset of Azure components for Dapr offer support for this authentication method. Over time, support will be expanded to all other Azure components for Dapr. You can track the progress of the work, component-by-component, on [this issue](https://github.com/dapr/components-contrib/issues/1103). - -### About authentication with Azure AD - -Azure AD is Azure's identity and access management (IAM) solution, which is used to authenticate and authorize users and services. - -Azure AD is built on top of open standards such OAuth 2.0, which allows services (applications) to obtain access tokens to make requests to Azure services, including Azure Storage, Azure Key Vault, Cosmos DB, etc. In the Azure terminology, an application is also called a "Service Principal". - -Many of the services listed above also support authentication using other systems, such as "master keys" or "shared keys". Although those are always valid methods to authenticate your application (and Dapr continues to support them, as explained in each component's reference page), using Azure AD when possible offers various benefits, including: - -- The ability to leverage Managed Service Identities, which allow your application to authenticate with Azure AD, and obtain an access token to make requests to Azure services, without the need to use any credential. When your application is running on a supported Azure service (including, but not limited to, Azure VMs, Azure Kubernetes Service, Azure Web Apps, etc), an identity for your application can be assigned at the infrastructure level. - This way, your code does not have to deal with credentials of any kind, removing the challenge of safely managing credentials, allowing greater separation of concerns between development and operations teams and reducing the number of people with access to credentials, and lastly simplifying operational aspects–especially when multiple environments are used. -- Using RBAC (Role-Based Access Control) with supported services (such as Azure Storage and Cosmos DB), permissions given to an application can be fine-tuned, for example allowing restricting access to a subset of data or making it read-only. -- Better auditing for access. -- Ability to authenticate using certificates (optional). - -## Credentials metadata fields - -To authenticate with Azure AD, you will need to add the following credentials as values in the metadata for your Dapr component (read the next section for how to create them). There are multiple options depending on the way you have chosen to pass the credentials to your Dapr service. - -**Authenticating using client credentials:** - -| Field | Required | Details | Example | -|---------------------|----------|--------------------------------------|----------------------------------------------| -| `azureTenantId` | Y | ID of the Azure AD tenant | `"cd4b2887-304c-47e1-b4d5-65447fdd542b"` | -| `azureClientId` | Y | Client ID (application ID) | `"c7dd251f-811f-4ba2-a905-acd4d3f8f08b"` | -| `azureClientSecret` | Y | Client secret (application password) | `"Ecy3XG7zVZK3/vl/a2NSB+a1zXLa8RnMum/IgD0E"` | - -When running on Kubernetes, you can also use references to Kubernetes secrets for any or all of the values above. - -**Authenticating using a PFX certificate:** - -| Field | Required | Details | Example | -|--------|--------|--------|--------| -| `azureTenantId` | Y | ID of the Azure AD tenant | `"cd4b2887-304c-47e1-b4d5-65447fdd542b"` | -| `azureClientId` | Y | Client ID (application ID) | `"c7dd251f-811f-4ba2-a905-acd4d3f8f08b"` | -| `azureCertificate` | One of `azureCertificate` and `azureCertificateFile` | Certificate and private key (in PFX/PKCS#12 format) | `"-----BEGIN PRIVATE KEY-----\n MIIEvgI... \n -----END PRIVATE KEY----- \n -----BEGIN CERTIFICATE----- \n MIICoTC... \n -----END CERTIFICATE-----` | -| `azureCertificateFile` | One of `azureCertificate` and `azureCertificateFile` | Path to the PFX/PKCS#12 file containing the certificate and private key | `"/path/to/file.pem"` | -| `azureCertificatePassword` | N | Password for the certificate if encrypted | `"password"` | - -When running on Kubernetes, you can also use references to Kubernetes secrets for any or all of the values above. - -**Authenticating with Managed Service Identities (MSI):** - -| Field | Required | Details | Example | -|-----------------|----------|----------------------------|------------------------------------------| -| `azureClientId` | N | Client ID (application ID) | `"c7dd251f-811f-4ba2-a905-acd4d3f8f08b"` | - -Using MSI you're not required to specify any value, although you may optionally pass `azureClientId` if needed. - -### Aliases - -For backwards-compatibility reasons, the following values in the metadata are supported as aliases, although their use is discouraged. - -| Metadata key | Aliases (supported but deprecated) | -|----------------------------|------------------------------------| -| `azureTenantId` | `spnTenantId`, `tenantId` | -| `azureClientId` | `spnClientId`, `clientId` | -| `azureClientSecret` | `spnClientSecret`, `clientSecret` | -| `azureCertificate` | `spnCertificate` | -| `azureCertificateFile` | `spnCertificateFile` | -| `azureCertificatePassword` | `spnCertificatePassword` | - -## Generating a new Azure AD application and Service Principal - -To start, create a new Azure AD application, which will also be used as Service Principal. - -Prerequisites: - -- Azure Subscription -- [Azure CLI](https://docs.microsoft.com/cli/azure/install-azure-cli) -- [jq](https://stedolan.github.io/jq/download/) -- OpenSSL (included by default on all Linux and macOS systems, as well as on WSL) -- The scripts below are optimized for a bash or zsh shell - -> If you haven't already, start by logging in to Azure using the Azure CLI: -> -> ```sh -> # Log in Azure -> az login -> # Set your default subscription -> az account set -s [your subscription id] -> ``` - -### Creating an Azure AD application - -First, create the Azure AD application with: - -```sh -# Friendly name for the application / Service Principal -APP_NAME="dapr-application" - -# Create the app -APP_ID=$(az ad app create --display-name "${APP_NAME}" | jq -r .appId) -``` - -{{< tabs "Client secret" "Certificate">}} - -{{% codetab %}} - -To create a **client secret**, then run this command. This will generate a random password based on the base64 charset and 40-characters long. Additionally, it will make the password valid for 2 years, before it will need to be rotated: - -```sh -az ad app credential reset \ - --id "${APP_ID}" \ - --years 2 -``` - -The output of the command above will be similar to this: - -```json -{ - "appId": "c7dd251f-811f-4ba2-a905-acd4d3f8f08b", - "password": "Ecy3XG7zVZK3/vl/a2NSB+a1zXLa8RnMum/IgD0E", - "tenant": "cd4b2887-304c-47e1-b4d5-65447fdd542b" -} -``` - -Take note of the values above, which you'll need to use in your Dapr components' metadata, to allow Dapr to authenticate with Azure: - -- `appId` is the value for `azureClientId` -- `password` is the value for `azureClientSecret` (this was randomly-generated) -- `tenant` is the value for `azureTenantId` - -{{% /codetab %}} - -{{% codetab %}} -If you'd rather use a **PFX (PKCS#12) certificate**, run this command which will create a self-signed certificate: - -```sh -az ad app credential reset \ - --id "${APP_ID}" \ - --create-cert -``` - -> Note: self-signed certificates are recommended for development only. For production, you should use certificates signed by a CA and imported with the `--cert` flag. - -The output of the command above should look like: - -```json -{ - "appId": "c7dd251f-811f-4ba2-a905-acd4d3f8f08b", - "fileWithCertAndPrivateKey": "/Users/alessandro/tmpgtdgibk4.pem", - "password": null, - "tenant": "cd4b2887-304c-47e1-b4d5-65447fdd542b" -} -``` - -Take note of the values above, which you'll need to use in your Dapr components' metadata: - -- `appId` is the value for `azureClientId` -- `tenant` is the value for `azureTenantId` -- The self-signed PFX certificate and private key are written in the file at the path specified in `fileWithCertAndPrivateKey`. - Use the contents of that file as `azureCertificate` (or write it to a file on the server and use `azureCertificateFile`) - -> While the generated file has the `.pem` extension, it contains a certificate and private key encoded as PFX (PKCS#12). - -{{% /codetab %}} - -{{< /tabs >}} - -### Creating a Service Principal - -Once you have created an Azure AD application, create a Service Principal for that application, which will allow us to grant it access to Azure resources. Run: - -```sh -SERVICE_PRINCIPAL_ID=$(az ad sp create \ - --id "${APP_ID}" \ - | jq -r .id) -echo "Service Principal ID: ${SERVICE_PRINCIPAL_ID}" -``` - -The output will be similar to: - -```text -Service Principal ID: 1d0ccf05-5427-4b5e-8eb4-005ac5f9f163 -``` - -Note that the value above is the ID of the **Service Principal** which is different from the ID of application in Azure AD (client ID)! The former is defined within an Azure tenant and is used to grant access to Azure resources to an application. The client ID instead is used by your application to authenticate. To sum things up: - -- You'll use the client ID in Dapr manifests to configure authentication with Azure services -- You'll use the Service Principal ID to grant permissions to an application to access Azure resources - -Keep in mind that the Service Principal that was just created does not have access to any Azure resource by default. Access will need to be granted to each resource as needed, as documented in the docs for the components. - -> Note: this step is different from the [official documentation](https://docs.microsoft.com/cli/azure/create-an-azure-service-principal-azure-cli) as the short-hand commands included there create a Service Principal that has broad read-write access to all Azure resources in your subscription. -> Not only doing that would grant our Service Principal more access than you are likely going to desire, but this also applies only to the Azure management plane (Azure Resource Manager, or ARM), which is irrelevant for Dapr anyways (all Azure components are designed to interact with the data plane of various services, and not ARM). - -### Example usage in a Dapr component - -In this example, you will set up an Azure Key Vault secret store component that uses Azure AD to authenticate. - -{{< tabs "Self-Hosted" "Kubernetes">}} - -{{% codetab %}} - -To use a **client secret**, create a file called `azurekeyvault.yaml` in the components directory, filling in with the details from the above setup process: - -```yaml -apiVersion: dapr.io/v1alpha1 -kind: Component -metadata: - name: azurekeyvault - namespace: default -spec: - type: secretstores.azure.keyvault - version: v1 - metadata: - - name: vaultName - value: "[your_keyvault_name]" - - name: azureTenantId - value: "[your_tenant_id]" - - name: azureClientId - value: "[your_client_id]" - - name: azureClientSecret - value : "[your_client_secret]" -``` - -If you want to use a **certificate** saved on the local disk, instead, use: - -```yaml -apiVersion: dapr.io/v1alpha1 -kind: Component -metadata: - name: azurekeyvault - namespace: default -spec: - type: secretstores.azure.keyvault - version: v1 - metadata: - - name: vaultName - value: "[your_keyvault_name]" - - name: azureTenantId - value: "[your_tenant_id]" - - name: azureClientId - value: "[your_client_id]" - - name: azureCertificateFile - value : "[pfx_certificate_file_fully_qualified_local_path]" -``` -{{% /codetab %}} - -{{% codetab %}} -In Kubernetes, you store the client secret or the certificate into the Kubernetes Secret Store and then refer to those in the YAML file. - -To use a **client secret**: - -1. Create a Kubernetes secret using the following command: - - ```bash - kubectl create secret generic [your_k8s_secret_name] --from-literal=[your_k8s_secret_key]=[your_client_secret] - ``` - - - `[your_client_secret]` is the application's client secret as generated above - - `[your_k8s_secret_name]` is secret name in the Kubernetes secret store - - `[your_k8s_secret_key]` is secret key in the Kubernetes secret store - -2. Create an `azurekeyvault.yaml` component file. - - The component yaml refers to the Kubernetes secretstore using `auth` property and `secretKeyRef` refers to the client secret stored in the Kubernetes secret store. - - ```yaml - apiVersion: dapr.io/v1alpha1 - kind: Component - metadata: - name: azurekeyvault - namespace: default - spec: - type: secretstores.azure.keyvault - version: v1 - metadata: - - name: vaultName - value: "[your_keyvault_name]" - - name: azureTenantId - value: "[your_tenant_id]" - - name: azureClientId - value: "[your_client_id]" - - name: azureClientSecret - secretKeyRef: - name: "[your_k8s_secret_name]" - key: "[your_k8s_secret_key]" - auth: - secretStore: kubernetes - ``` - -3. Apply the `azurekeyvault.yaml` component: - - ```bash - kubectl apply -f azurekeyvault.yaml - ``` - -To use a **certificate**: - -1. Create a Kubernetes secret using the following command: - - ```bash - kubectl create secret generic [your_k8s_secret_name] --from-file=[your_k8s_secret_key]=[pfx_certificate_file_fully_qualified_local_path] - ``` - - - `[pfx_certificate_file_fully_qualified_local_path]` is the path to the PFX file you obtained earlier - - `[your_k8s_secret_name]` is secret name in the Kubernetes secret store - - `[your_k8s_secret_key]` is secret key in the Kubernetes secret store - -2. Create an `azurekeyvault.yaml` component file. - - The component yaml refers to the Kubernetes secretstore using `auth` property and `secretKeyRef` refers to the certificate stored in the Kubernetes secret store. - - ```yaml - apiVersion: dapr.io/v1alpha1 - kind: Component - metadata: - name: azurekeyvault - namespace: default - spec: - type: secretstores.azure.keyvault - version: v1 - metadata: - - name: vaultName - value: "[your_keyvault_name]" - - name: azureTenantId - value: "[your_tenant_id]" - - name: azureClientId - value: "[your_client_id]" - - name: azureCertificate - secretKeyRef: - name: "[your_k8s_secret_name]" - key: "[your_k8s_secret_key]" - auth: - secretStore: kubernetes - ``` - -3. Apply the `azurekeyvault.yaml` component: - - ```bash - kubectl apply -f azurekeyvault.yaml - ``` - -{{% /codetab %}} - -{{< /tabs >}} - -## Using Managed Service Identities - -Using MSI, authentication happens automatically by virtue of your application running on top of an Azure service that has an assigned identity. For example, when you create an Azure VM or an Azure Kubernetes Service cluster and choose to enable a managed identity for that, an Azure AD application is created for you and automatically assigned to the service. Your Dapr services can then leverage that identity to authenticate with Azure AD, transparently and without you having to specify any credential. - -To get started with managed identities, first you need to assign an identity to a new or existing Azure resource. The instructions depend on the service use. Below are links to the official documentation: - -- [Azure Kubernetes Service (AKS)](https://docs.microsoft.com/azure/aks/use-managed-identity) -- [Azure App Service](https://docs.microsoft.com/azure/app-service/overview-managed-identity) (including Azure Web Apps and Azure Functions) -- [Azure Virtual Machines (VM)](https://docs.microsoft.com/azure/active-directory/managed-identities-azure-resources/qs-configure-cli-windows-vm) -- [Azure Virtual Machines Scale Sets (VMSS)](https://docs.microsoft.com/azure/active-directory/managed-identities-azure-resources/qs-configure-cli-windows-vmss) -- [Azure Container Instance (ACI)](https://docs.microsoft.com/azure/container-instances/container-instances-managed-identity) - -Other Azure application services may offer support for MSI; please check the documentation for those services to understand how to configure them. - -After assigning a managed identity to your Azure resource, you will have credentials such as: - -```json -{ - "principalId": "", - "tenantId": "", - "type": "SystemAssigned", - "userAssignedIdentities": null -} -``` - -From the list above, take note of **`principalId`** which is the ID of the Service Principal that was created. You'll need that to grant access to Azure resources to your Service Principal. - -## Support for other Azure environments - -By default, Dapr components are configured to interact with Azure resources in the "public cloud". If your application is deployed to another cloud, such as Azure China, Azure Government, or Azure Germany, you can enable that for supported components by setting the `azureEnvironment` metadata property to one of the supported values: - -- Azure public cloud (default): `"AZUREPUBLICCLOUD"` -- Azure China: `"AZURECHINACLOUD"` -- Azure Government: `"AZUREUSGOVERNMENTCLOUD"` -- Azure Germany: `"AZUREGERMANCLOUD"` - -## References - -- [Azure AD app credential: Azure CLI reference](https://docs.microsoft.com/cli/azure/ad/app/credential) -- [Azure Managed Service Identity (MSI) overview](https://docs.microsoft.com/azure/active-directory/managed-identities-azure-resources/overview) -- [Secrets building block]({{< ref secrets >}}) -- [How-To: Retrieve a secret]({{< ref "howto-secrets.md" >}}) -- [How-To: Reference secrets in Dapr components]({{< ref component-secrets.md >}}) -- [Secrets API reference]({{< ref secrets_api.md >}}) diff --git a/daprdocs/content/en/developing-applications/integrations/Azure/azure-api-management.md b/daprdocs/content/en/developing-applications/integrations/Azure/azure-api-management.md index 5eb42a7cdad..a04c46ea003 100644 --- a/daprdocs/content/en/developing-applications/integrations/Azure/azure-api-management.md +++ b/daprdocs/content/en/developing-applications/integrations/Azure/azure-api-management.md @@ -6,6 +6,11 @@ description: "Publish APIs for Dapr services and components through Azure API Ma weight: 2000 --- -Azure API Management (APIM) is a way to create consistent and modern API gateways for back-end services, including those built with Dapr. Dapr support can be enabled in self-hosted API Management gateways to allow them to forward requests to Dapr services, send messages to Dapr Pub/Sub topics, or trigger Dapr output bindings. For more information, read the guide on [API Management Dapr Integration policies](https://docs.microsoft.com/azure/api-management/api-management-dapr-policies) and try out the [Dapr & Azure API Management Integration Demo](https://github.com/dapr/samples/tree/master/dapr-apim-integration). +[Azure API Management](https://learn.microsoft.com/azure/api-management/api-management-key-concepts) is a way to create consistent and modern API gateways for back-end services, including those built with Dapr. You can enable Dapr support in self-hosted API Management gateways to allow them to: +- Forward requests to Dapr services +- Send messages to Dapr Pub/Sub topics +- Trigger Dapr output bindings -{{< button text="Learn more" link="https://docs.microsoft.com/azure/api-management/api-management-dapr-policies" >}} +Try out the [Dapr & Azure API Management Integration sample](https://github.com/dapr/samples/tree/master/dapr-apim-integration). + +{{< button text="Learn more about Dapr integration policies" link="https://docs.microsoft.com/azure/api-management/api-management-dapr-policies" >}} diff --git a/daprdocs/content/en/developing-applications/integrations/Azure/azure-authentication/_index.md b/daprdocs/content/en/developing-applications/integrations/Azure/azure-authentication/_index.md new file mode 100644 index 00000000000..a94e4d7230b --- /dev/null +++ b/daprdocs/content/en/developing-applications/integrations/Azure/azure-authentication/_index.md @@ -0,0 +1,7 @@ +--- +type: docs +title: "Authenticate to Azure" +linkTitle: "Authenticate to Azure" +weight: 1600 +description: "Learn about authenticating Azure components using Azure Active Directory or Managed Service Identities" +--- \ No newline at end of file diff --git a/daprdocs/content/en/developing-applications/integrations/Azure/azure-authentication/authenticating-azure.md b/daprdocs/content/en/developing-applications/integrations/Azure/azure-authentication/authenticating-azure.md new file mode 100644 index 00000000000..65cc1a24883 --- /dev/null +++ b/daprdocs/content/en/developing-applications/integrations/Azure/azure-authentication/authenticating-azure.md @@ -0,0 +1,273 @@ +--- +type: docs +title: "Authenticating to Azure" +linkTitle: "Overview" +description: "How to authenticate Azure components using Azure AD and/or Managed Identities" +aliases: + - "/operations/components/setup-secret-store/supported-secret-stores/azure-keyvault-managed-identity/" + - "/reference/components-reference/supported-secret-stores/azure-keyvault-managed-identity/" +weight: 10000 +--- + +Certain Azure components for Dapr offer support for the *common Azure authentication layer*, which enables applications to access data stored in Azure resources by authenticating with Azure Active Directory (Azure AD). Thanks to this: +- Administrators can leverage all the benefits of fine-tuned permissions with Role-Based Access Control (RBAC). +- Applications running on Azure services such as Azure Container Apps, Azure Kubernetes Service, Azure VMs, or any other Azure platform services can leverage [Managed Service Identities (MSI)](https://docs.microsoft.com/azure/active-directory/managed-identities-azure-resources/overview). + + +## About authentication with Azure AD + +Azure AD is Azure's identity and access management (IAM) solution, which is used to authenticate and authorize users and services. + +Azure AD is built on top of open standards such OAuth 2.0, which allows services (applications) to obtain access tokens to make requests to Azure services, including Azure Storage, Azure Key Vault, Cosmos DB, etc. + +> In Azure terminology, an application is also called a "Service Principal". + +Some Azure components offer alternative authentication methods, such as systems based on "master keys" or "shared keys". Although both master keys and shared keys are valid and supported by Dapr, you should authenticate your Dapr components using Azure AD. Using Azure AD offers benefits like the following. + +### Managed Service Identities + +With Managed Service Identities (MSI), your application can authenticate with Azure AD and obtain an access token to make requests to Azure services. When your application is running on a supported Azure service, an identity for your application can be assigned at the infrastructure level. + +Once using MSI, your code doesn't have to deal with credentials, which: +- Removes the challenge of managing credentials safely +- Allows greater separation of concerns between development and operations teams +- Reduces the number of people with access to credentials +- Simplifies operational aspects–especially when multiple environments are used + +### Role-based Access Control + +When using Role-Based Access Control (RBAC) with supported services, permissions given to an application can be fine-tuned. For example, you can restrict access to a subset of data or make it read-only. + +### Auditing + +Using Azure AD provides an improved auditing experience for access. + +### (Optional) Authenticate using certificates + +While Azure AD allows you to use MSI or RBAC, you still have the option to authenticate using certificates. + +## Support for other Azure environments + +By default, Dapr components are configured to interact with Azure resources in the "public cloud". If your application is deployed to another cloud, such as Azure China, Azure Government, or Azure Germany, you can enable that for supported components by setting the `azureEnvironment` metadata property to one of the supported values: + +- Azure public cloud (default): `"AZUREPUBLICCLOUD"` +- Azure China: `"AZURECHINACLOUD"` +- Azure Government: `"AZUREUSGOVERNMENTCLOUD"` +- Azure Germany: `"AZUREGERMANCLOUD"` + +## Credentials metadata fields + +To authenticate with Azure AD, you will need to add the following credentials as values in the metadata for your [Dapr component]({{< ref "#example-usage-in-a-dapr-component" >}}). + +### Metadata options + +Depending on how you've passed credentials to your Dapr services, you have multiple metadata options. + +#### Authenticating using client credentials + +| Field | Required | Details | Example | +|---------------------|----------|--------------------------------------|----------------------------------------------| +| `azureTenantId` | Y | ID of the Azure AD tenant | `"cd4b2887-304c-47e1-b4d5-65447fdd542b"` | +| `azureClientId` | Y | Client ID (application ID) | `"c7dd251f-811f-4ba2-a905-acd4d3f8f08b"` | +| `azureClientSecret` | Y | Client secret (application password) | `"Ecy3XG7zVZK3/vl/a2NSB+a1zXLa8RnMum/IgD0E"` | + +When running on Kubernetes, you can also use references to Kubernetes secrets for any or all of the values above. + +#### Authenticating using a PFX certificate + +| Field | Required | Details | Example | +|--------|--------|--------|--------| +| `azureTenantId` | Y | ID of the Azure AD tenant | `"cd4b2887-304c-47e1-b4d5-65447fdd542b"` | +| `azureClientId` | Y | Client ID (application ID) | `"c7dd251f-811f-4ba2-a905-acd4d3f8f08b"` | +| `azureCertificate` | One of `azureCertificate` and `azureCertificateFile` | Certificate and private key (in PFX/PKCS#12 format) | `"-----BEGIN PRIVATE KEY-----\n MIIEvgI... \n -----END PRIVATE KEY----- \n -----BEGIN CERTIFICATE----- \n MIICoTC... \n -----END CERTIFICATE-----` | +| `azureCertificateFile` | One of `azureCertificate` and `azureCertificateFile` | Path to the PFX/PKCS#12 file containing the certificate and private key | `"/path/to/file.pem"` | +| `azureCertificatePassword` | N | Password for the certificate if encrypted | `"password"` | + +When running on Kubernetes, you can also use references to Kubernetes secrets for any or all of the values above. + +#### Authenticating with Managed Service Identities (MSI) + +| Field | Required | Details | Example | +|-----------------|----------|----------------------------|------------------------------------------| +| `azureClientId` | N | Client ID (application ID) | `"c7dd251f-811f-4ba2-a905-acd4d3f8f08b"` | + +Using MSI, you're not required to specify any value, although you may pass `azureClientId` if needed. + +### Aliases + +For backwards-compatibility reasons, the following values in the metadata are supported as aliases. Their use is discouraged. + +| Metadata key | Aliases (supported but deprecated) | +|----------------------------|------------------------------------| +| `azureTenantId` | `spnTenantId`, `tenantId` | +| `azureClientId` | `spnClientId`, `clientId` | +| `azureClientSecret` | `spnClientSecret`, `clientSecret` | +| `azureCertificate` | `spnCertificate` | +| `azureCertificateFile` | `spnCertificateFile` | +| `azureCertificatePassword` | `spnCertificatePassword` | + + +### Example usage in a Dapr component + +In this example, you will set up an Azure Key Vault secret store component that uses Azure AD to authenticate. + +{{< tabs "Self-Hosted" "Kubernetes">}} + +{{% codetab %}} + +To use a **client secret**, create a file called `azurekeyvault.yaml` in the components directory, filling in with the details from the above setup process: + +```yaml +apiVersion: dapr.io/v1alpha1 +kind: Component +metadata: + name: azurekeyvault + namespace: default +spec: + type: secretstores.azure.keyvault + version: v1 + metadata: + - name: vaultName + value: "[your_keyvault_name]" + - name: azureTenantId + value: "[your_tenant_id]" + - name: azureClientId + value: "[your_client_id]" + - name: azureClientSecret + value : "[your_client_secret]" +``` + +If you want to use a **certificate** saved on the local disk, instead, use: + +```yaml +apiVersion: dapr.io/v1alpha1 +kind: Component +metadata: + name: azurekeyvault + namespace: default +spec: + type: secretstores.azure.keyvault + version: v1 + metadata: + - name: vaultName + value: "[your_keyvault_name]" + - name: azureTenantId + value: "[your_tenant_id]" + - name: azureClientId + value: "[your_client_id]" + - name: azureCertificateFile + value : "[pfx_certificate_file_fully_qualified_local_path]" +``` +{{% /codetab %}} + +{{% codetab %}} +In Kubernetes, you store the client secret or the certificate into the Kubernetes Secret Store and then refer to those in the YAML file. + +To use a **client secret**: + +1. Create a Kubernetes secret using the following command: + + ```bash + kubectl create secret generic [your_k8s_secret_name] --from-literal=[your_k8s_secret_key]=[your_client_secret] + ``` + + - `[your_client_secret]` is the application's client secret as generated above + - `[your_k8s_secret_name]` is secret name in the Kubernetes secret store + - `[your_k8s_secret_key]` is secret key in the Kubernetes secret store + +1. Create an `azurekeyvault.yaml` component file. + + The component yaml refers to the Kubernetes secretstore using `auth` property and `secretKeyRef` refers to the client secret stored in the Kubernetes secret store. + + ```yaml + apiVersion: dapr.io/v1alpha1 + kind: Component + metadata: + name: azurekeyvault + namespace: default + spec: + type: secretstores.azure.keyvault + version: v1 + metadata: + - name: vaultName + value: "[your_keyvault_name]" + - name: azureTenantId + value: "[your_tenant_id]" + - name: azureClientId + value: "[your_client_id]" + - name: azureClientSecret + secretKeyRef: + name: "[your_k8s_secret_name]" + key: "[your_k8s_secret_key]" + auth: + secretStore: kubernetes + ``` + +1. Apply the `azurekeyvault.yaml` component: + + ```bash + kubectl apply -f azurekeyvault.yaml + ``` + +To use a **certificate**: + +1. Create a Kubernetes secret using the following command: + + ```bash + kubectl create secret generic [your_k8s_secret_name] --from-file=[your_k8s_secret_key]=[pfx_certificate_file_fully_qualified_local_path] + ``` + + - `[pfx_certificate_file_fully_qualified_local_path]` is the path to the PFX file you obtained earlier + - `[your_k8s_secret_name]` is secret name in the Kubernetes secret store + - `[your_k8s_secret_key]` is secret key in the Kubernetes secret store + +1. Create an `azurekeyvault.yaml` component file. + + The component yaml refers to the Kubernetes secretstore using `auth` property and `secretKeyRef` refers to the certificate stored in the Kubernetes secret store. + + ```yaml + apiVersion: dapr.io/v1alpha1 + kind: Component + metadata: + name: azurekeyvault + namespace: default + spec: + type: secretstores.azure.keyvault + version: v1 + metadata: + - name: vaultName + value: "[your_keyvault_name]" + - name: azureTenantId + value: "[your_tenant_id]" + - name: azureClientId + value: "[your_client_id]" + - name: azureCertificate + secretKeyRef: + name: "[your_k8s_secret_name]" + key: "[your_k8s_secret_key]" + auth: + secretStore: kubernetes + ``` + +1. Apply the `azurekeyvault.yaml` component: + + ```bash + kubectl apply -f azurekeyvault.yaml + ``` + +{{% /codetab %}} + +{{< /tabs >}} + +## Next steps + +{{< button text="Generate a new Azure AD application and Service Principal >>" page="howto-aad.md" >}} + +## References + +- [Azure AD app credential: Azure CLI reference](https://docs.microsoft.com/cli/azure/ad/app/credential) +- [Azure Managed Service Identity (MSI) overview](https://docs.microsoft.com/azure/active-directory/managed-identities-azure-resources/overview) +- [Secrets building block]({{< ref secrets >}}) +- [How-To: Retrieve a secret]({{< ref "howto-secrets.md" >}}) +- [How-To: Reference secrets in Dapr components]({{< ref component-secrets.md >}}) +- [Secrets API reference]({{< ref secrets_api.md >}}) diff --git a/daprdocs/content/en/developing-applications/integrations/Azure/azure-authentication/howto-aad.md b/daprdocs/content/en/developing-applications/integrations/Azure/azure-authentication/howto-aad.md new file mode 100644 index 00000000000..f76cf2fc957 --- /dev/null +++ b/daprdocs/content/en/developing-applications/integrations/Azure/azure-authentication/howto-aad.md @@ -0,0 +1,147 @@ +--- +type: docs +title: "How to: Generate a new Azure AD application and Service Principal" +linkTitle: "How to: Generate Azure AD and Service Principal" +weight: 30000 +description: "Learn how to generate an Azure Active Directory and use it as a Service Principal" +--- + +## Prerequisites + +- [An Azure subscription](https://azure.microsoft.com/free/) +- [Azure CLI](https://docs.microsoft.com/cli/azure/install-azure-cli) +- [jq](https://stedolan.github.io/jq/download/) +- OpenSSL (included by default on all Linux and macOS systems, as well as on WSL) +- Make sure you're using a bash or zsh shell + +## Log into Azure using the Azure CLI + +In a new terminal, run the following command: + +```sh +az login +az account set -s [your subscription id] +``` + +### Create an Azure AD application + +Create the Azure AD application with: + +```sh +# Friendly name for the application / Service Principal +APP_NAME="dapr-application" + +# Create the app +APP_ID=$(az ad app create --display-name "${APP_NAME}" | jq -r .appId) +``` + +Select how you'd prefer to pass credentials. + +{{< tabs "Client secret" "PFX certificate">}} + +{{% codetab %}} + +To create a **client secret**, run the following command. + +```sh +az ad app credential reset \ + --id "${APP_ID}" \ + --years 2 +``` + +This generates a random, 40-characters long password based on the `base64` charset. This password will be valid for 2 years, before you need to rotate it. + +Save the output values returned; you'll need them for Dapr to authenticate with Azure. The expected output: + +```json +{ + "appId": "", + "password": "", + "tenant": "" +} +``` + +When adding the returned values to your Dapr component's metadata: +- `appId` is the value for `azureClientId` +- `password` is the value for `azureClientSecret` (this was randomly-generated) +- `tenant` is the value for `azureTenantId` + +{{% /codetab %}} + +{{% codetab %}} +For a **PFX (PKCS#12) certificate**, run the following command to create a self-signed certificate: + +```sh +az ad app credential reset \ + --id "${APP_ID}" \ + --create-cert +``` + +> **Note:** Self-signed certificates are recommended for development only. For production, you should use certificates signed by a CA and imported with the `--cert` flag. + +The output of the command above should look like: + +Save the output values returned; you'll need them for Dapr to authenticate with Azure. The expected output: + +```json +{ + "appId": "", + "fileWithCertAndPrivateKey": "", + "password": null, + "tenant": "" +} +``` + +When adding the returned values to your Dapr component's metadata: +- `appId` is the value for `azureClientId` +- `tenant` is the value for `azureTenantId` +- `fileWithCertAndPrivateKey` indicates the location of the self-signed PFX certificate and private key. Use the contents of that file as `azureCertificate` (or write it to a file on the server and use `azureCertificateFile`) + +> **Note:** While the generated file has the `.pem` extension, it contains a certificate and private key encoded as _PFX (PKCS#12)_. + +{{% /codetab %}} + +{{< /tabs >}} + +### Create a Service Principal + +Once you have created an Azure AD application, create a Service Principal for that application. With this Service Principal, you can grant it access to Azure resources. + +To create the Service Principal, run the following command: + +```sh +SERVICE_PRINCIPAL_ID=$(az ad sp create \ + --id "${APP_ID}" \ + | jq -r .id) +echo "Service Principal ID: ${SERVICE_PRINCIPAL_ID}" +``` + +Expected output: + +```text +Service Principal ID: 1d0ccf05-5427-4b5e-8eb4-005ac5f9f163 +``` + +The returned value above is the **Service Principal ID**, which is different from the Azure AD application ID (client ID). + +**The Service Principal ID** is: +- Defined within an Azure tenant +- Used to grant access to Azure resources to an application + +You'll use the Service Principal ID to grant permissions to an application to access Azure resources. + +Meanwhile, **the client ID** is used by your application to authenticate. You'll use the client ID in Dapr manifests to configure authentication with Azure services. + +Keep in mind that the Service Principal that was just created does not have access to any Azure resource by default. Access will need to be granted to each resource as needed, as documented in the docs for the components. + +{{% alert title="Note" color="primary" %}} +This step is different from the [official Azure documentation](https://docs.microsoft.com/cli/azure/create-an-azure-service-principal-azure-cli). The short-hand commands included in the official documentation creates a Service Principal that has broad `read-write` access to all Azure resources in your subscription, which: + +- Grants your Service Principal more access than you likely desire. +- Applies _only_ to the Azure management plane (Azure Resource Manager, or ARM), which is irrelevant for Dapr components, which are designed to interact with the data plane of various services. + +{{% /alert %}} + +## Next steps + +{{< button text="Use MSI >>" page="howto-msi.md" >}} diff --git a/daprdocs/content/en/developing-applications/integrations/Azure/azure-authentication/howto-msi.md b/daprdocs/content/en/developing-applications/integrations/Azure/azure-authentication/howto-msi.md new file mode 100644 index 00000000000..bebfa1b784c --- /dev/null +++ b/daprdocs/content/en/developing-applications/integrations/Azure/azure-authentication/howto-msi.md @@ -0,0 +1,38 @@ +--- +type: docs +title: "How to: Use Managed Service Identities" +linkTitle: "How to: Use MSI" +weight: 40000 +description: "Learn how to use Managed Service Identities" +--- + +Using MSI, authentication happens automatically by virtue of your application running on top of an Azure service that has an assigned identity. + +For example, let's say you enable a managed service identity for an Azure VM, Azure Container App, or an Azure Kubernetes Service cluster. When you do, an Azure AD application is created for you and automatically assigned to the service. Your Dapr services can then leverage that identity to authenticate with Azure AD, transparently and without you having to specify any credential. + +To get started with managed identities, you need to assign an identity to a new or existing Azure resource. The instructions depend on the service use. Check the following official documentation for the most appropriate instructions: + +- [Azure Kubernetes Service (AKS)](https://docs.microsoft.com/azure/aks/use-managed-identity) +- [Azure Container Apps (ACA)](https://learn.microsoft.com/azure/container-apps/dapr-overview?tabs=bicep1%2Cyaml#using-managed-identity) +- [Azure App Service](https://docs.microsoft.com/azure/app-service/overview-managed-identity) (including Azure Web Apps and Azure Functions) +- [Azure Virtual Machines (VM)](https://docs.microsoft.com/azure/active-directory/managed-identities-azure-resources/qs-configure-cli-windows-vm) +- [Azure Virtual Machines Scale Sets (VMSS)](https://docs.microsoft.com/azure/active-directory/managed-identities-azure-resources/qs-configure-cli-windows-vmss) +- [Azure Container Instance (ACI)](https://docs.microsoft.com/azure/container-instances/container-instances-managed-identity) + + +After assigning a managed identity to your Azure resource, you will have credentials such as: + +```json +{ + "principalId": "", + "tenantId": "", + "type": "SystemAssigned", + "userAssignedIdentities": null +} +``` + +From the returned values, take note of **`principalId`**, which is the Service Principal ID that was created. You'll use that to grant access to Azure resources to your Service Principal. + +## Next steps + +{{< button text="Refer to Azure component specs >>" page="components-reference" >}} diff --git a/daprdocs/content/en/developing-applications/integrations/Azure/azure-functions.md b/daprdocs/content/en/developing-applications/integrations/Azure/azure-functions.md index 20d73152192..7fe9dd013ad 100644 --- a/daprdocs/content/en/developing-applications/integrations/Azure/azure-functions.md +++ b/daprdocs/content/en/developing-applications/integrations/Azure/azure-functions.md @@ -3,8 +3,16 @@ type: docs title: "Dapr extension for Azure Functions runtime" linkTitle: "Azure Functions extension" description: "Access Dapr capabilities from your Azure Functions runtime application" -weight: 4000 +weight: 3000 --- -Dapr integrates with the Azure Functions runtime via an extension that lets a function seamlessly interact with Dapr. Azure Functions provides an event-driven programming model and Dapr provides cloud-native building blocks. With this extension, you can bring both together for serverless and event-driven apps. For more information read -[Azure Functions extension for Dapr](https://cloudblogs.microsoft.com/opensource/2020/07/01/announcing-azure-functions-extension-for-dapr/) and visit the [Azure Functions extension](https://github.com/dapr/azure-functions-extension) repo to try out the samples. \ No newline at end of file +{{% alert title="Note" color="primary" %}} +The Dapr Functions extension is currently in preview. +{{% /alert %}} + + +Dapr integrates with the [Azure Functions runtime](https://learn.microsoft.com/azure/azure-functions/functions-overview) via an extension that lets a function seamlessly interact with Dapr. Azure Functions provides an event-driven programming model and Dapr provides cloud-native building blocks. The extension combines the two for serverless and event-driven apps. + +Try out the [Dapr Functions extension](https://github.com/dapr/azure-functions-extension) samples. + +{{< button text="Learn more about the Dapr Function extension in preview" link="https://cloudblogs.microsoft.com/opensource/2020/07/01/announcing-azure-functions-extension-for-dapr/" >}} diff --git a/daprdocs/content/en/developing-applications/integrations/Azure/azure-kubernetes-service-extension.md b/daprdocs/content/en/developing-applications/integrations/Azure/azure-kubernetes-service-extension.md index a8dab479e66..aae063bffdd 100644 --- a/daprdocs/content/en/developing-applications/integrations/Azure/azure-kubernetes-service-extension.md +++ b/daprdocs/content/en/developing-applications/integrations/Azure/azure-kubernetes-service-extension.md @@ -6,104 +6,12 @@ description: "Provision Dapr on your Azure Kubernetes Service (AKS) cluster with weight: 4000 --- -# Prerequisites -- Azure subscription -- [Azure CLI](https://docs.microsoft.com/cli/azure/install-azure-cli-windows?tabs=azure-cli) and the ***aks-preview*** extension. -- [Azure Kubernetes Service (AKS) cluster](https://docs.microsoft.com/azure/aks/tutorial-kubernetes-deploy-cluster?tabs=azure-cli) - -## Install Dapr using the AKS Dapr extension -The recommended approach for installing Dapr on AKS is to use the AKS Dapr extension. The extension offers support for all native Dapr configuration capabilities through command-line arguments via the Azure CLI and offers the option of opting into automatic minor version upgrades of the Dapr runtime. +The recommended approach for installing Dapr on AKS is to use the AKS Dapr extension. The extension offers: +- Support for all native Dapr configuration capabilities through command-line arguments via the Azure CLI +- The option of opting into automatic minor version upgrades of the Dapr runtime {{% alert title="Note" color="warning" %}} -If you install Dapr through the AKS extension, our recommendation is to continue using the extension for future management of Dapr instead of the Dapr CLI. Combining the two tools can cause conflicts and result in undesired behavior. +If you install Dapr through the AKS extension, best practice is to continue using the extension for future management of Dapr _instead of the Dapr CLI_. Combining the two tools can cause conflicts and result in undesired behavior. {{% /alert %}} -### How the extension works -The Dapr extension works by provisioning the Dapr control plane on your AKS cluster through the Azure CLI. The dapr control plane consists of: - -- **dapr-operator**: Manages component updates and Kubernetes services endpoints for Dapr (state stores, pub/subs, etc.) -- **dapr-sidecar-injector**: Injects Dapr into annotated deployment pods and adds the environment variables `DAPR_HTTP_PORT` and `DAPR_GRPC_PORT`. This enables user-defined applications to communicate with Dapr without the need to hard-code Dapr port values. -- **dapr-placement**: Used for actors only. Creates mapping tables that map actor instances to pods -- **dapr-sentry**: Manages mTLS between services and acts as a certificate authority. For more information read the security overview. - -### Extension Prerequisites -In order to use the AKS Dapr extension, you must first enable the `AKS-ExtensionManager` and `AKS-Dapr` feature flags on your Azure subscription. - -The below command will register the `AKS-ExtensionManager` and `AKS-Dapr` feature flags on your Azure subscription: - -```bash -az feature register --namespace "Microsoft.ContainerService" --name "AKS-ExtensionManager" -az feature register --namespace "Microsoft.ContainerService" --name "AKS-Dapr" -``` - -After a few minutes, check the status to show `Registered`. Confirm the registration status by using the az feature list command: - -```bash -az feature list -o table --query "[?contains(name, 'Microsoft.ContainerService/AKS-ExtensionManager')].{Name:name,State:properties.state}" -az feature list -o table --query "[?contains(name, 'Microsoft.ContainerService/AKS-Dapr')].{Name:name,State:properties.state}" -``` - -Next, refresh the registration of the `Microsoft.KubernetesConfiguration` and `Microsoft.ContainerService` resource providers by using the az provider register command: - -```bash -az provider register --namespace Microsoft.KubernetesConfiguration -az provider register --namespace Microsoft.ContainerService -``` - -#### Enable the Azure CLI extension for cluster extensions -You will also need the `k8s-extension` Azure CLI extension. Install this by running the following commands: - -```bash -az extension add --name k8s-extension -``` - -If the `k8s-extension` extension is already present, you can update it to the latest version using the below command: - -```bash -az extension update --name k8s-extension -``` - -#### Create the extension and install Dapr on your AKS cluster -After your subscription is registered to use Kubernetes extensions, install Dapr on your cluster by creating the Dapr extension. For example: - -```bash -az k8s-extension create --cluster-type managedClusters \ ---cluster-name myAKSCluster \ ---resource-group myResourceGroup \ ---name myDaprExtension \ ---extension-type Microsoft.Dapr -``` - -Additionally, Dapr can automatically update its minor version. To enable this, set the `--auto-upgrade-minor-version` parameter to true: - -```bash ---auto-upgrade-minor-version true -``` - -Once the k8-extension finishes provisioning, you can confirm that the Dapr control plane is installed on your AKS cluster by running: - -```bash -kubectl get pods -n dapr-system -``` - -In the example output below, note how the Dapr control plane is installed with high availability mode, enabled by default. - -``` -~  kubectl get pods -n dapr-system -NAME READY STATUS RESTARTS AGE -dapr-dashboard-5f49d48796-rnt5t 1/1 Running 0 1h -dapr-operator-98579b8b4-fpz7k 1/1 Running 0 1h -dapr-operator-98579b8b4-nn5vm 1/1 Running 0 1h -dapr-operator-98579b8b4-pplqr 1/1 Running 0 1h -dapr-placement-server-0 1/1 Running 0 1h -dapr-placement-server-1 1/1 Running 0 1h -dapr-placement-server-2 1/1 Running 0 1h -dapr-sentry-775bccdddb-htcl7 1/1 Running 0 1h -dapr-sentry-775bccdddb-vtfxj 1/1 Running 0 1h -dapr-sentry-775bccdddb-w4l8x 1/1 Running 0 1h -dapr-sidecar-injector-9555889bc-klb9g 1/1 Running 0 1h -dapr-sidecar-injector-9555889bc-rpjwl 1/1 Running 0 1h -dapr-sidecar-injector-9555889bc-rqjgt 1/1 Running 0 1h -``` - -For more information about configuration options and targeting specific Dapr versions, see the official [AKS Dapr Extension Docs](https://docs.microsoft.com/azure/aks/dapr). +{{< button text="Learn more about the Dapr extension for AKS" link="https://learn.microsoft.com/azure/aks/dapr" >}} diff --git a/daprdocs/content/en/developing-applications/integrations/Azure/workflows.md b/daprdocs/content/en/developing-applications/integrations/Azure/workflows.md deleted file mode 100644 index a7efed37c29..00000000000 --- a/daprdocs/content/en/developing-applications/integrations/Azure/workflows.md +++ /dev/null @@ -1,231 +0,0 @@ ---- -type: docs -title: "Build workflow applications with Logic Apps" -linkTitle: "Logic Apps workflows" -description: "Learn how to build workflows applications using Dapr Workflows and Logic Apps runtime" -weight: 3000 ---- - -Dapr Workflows is a lightweight host that allows developers to run cloud-native workflows locally, on-premises or any cloud environment using the [Azure Logic Apps](https://docs.microsoft.com/azure/logic-apps/logic-apps-overview) workflow engine and Dapr. - -## Benefits - -By using a workflow engine, business logic can be defined in a declarative, no-code fashion so application code doesn't need to change when a workflow changes. Dapr Workflows allows you to use workflows in a distributed application along with these added benefits: - -- **Run workflows anywhere**: on your local machine, on-premises, on Kubernetes or in the cloud -- **Built-in observability**: tracing, metrics and mTLS through Dapr -- **gRPC and HTTP endpoints** for your workflows -- Kick off workflows based on **Dapr bindings** events -- Orchestrate complex workflows by **calling back to Dapr** to save state, publish a message and more - -Diagram of Dapr Workflows - -## How it works - -Dapr Workflows hosts a gRPC server that implements the Dapr Client API. - -This allows users to start workflows using gRPC and HTTP endpoints through Dapr, or start a workflow asynchronously using Dapr bindings. -Once a workflow request comes in, Dapr Workflows uses the Logic Apps SDK to execute the workflow. - -## Supported workflow features - -### Supported actions and triggers - -- [HTTP](https://docs.microsoft.com/azure/connectors/connectors-native-http) -- [Schedule](https://docs.microsoft.com/azure/logic-apps/concepts-schedule-automated-recurring-tasks-workflows) -- [Request / Response](https://docs.microsoft.com/azure/connectors/connectors-native-reqres) - -### Supported control workflows - -- [All control workflows](https://docs.microsoft.com/azure/connectors/apis-list#control-workflow) - -### Supported data manipulation - -- [All data operations](https://docs.microsoft.com/azure/connectors/apis-list#manage-or-manipulate-data) - -### Not supported - -- [Managed connectors](https://docs.microsoft.com/azure/connectors/apis-list#managed-connectors) - -## Example - -Dapr Workflows can be used as the orchestrator for many otherwise complex activities. For example, invoking an external endpoint, saving the data to a state store, publishing the result to a different app or invoking a binding can all be done by calling back into Dapr from the workflow itself. - -This is due to the fact Dapr runs as a sidecar next to the workflow host just as if it was any other app. - -Examine [workflow2.json](/code/workflow.json) as an example of a workflow that does the following: - -1. Calls into Azure Functions to get a JSON response -2. Saves the result to a Dapr state store -3. Sends the result to a Dapr binding -4. Returns the result to the caller - -Since Dapr supports many pluggable state stores and bindings, the workflow becomes portable between different environments (cloud, edge or on-premises) without the user changing the code - *because there is no code involved*. - -## Get started - -Prerequisites: - -1. Install the [Dapr CLI]({{< ref install-dapr-cli.md >}}) -2. [Azure blob storage account](https://docs.microsoft.com/azure/storage/blobs/storage-blob-create-account-block-blob?tabs=azure-portal) - -### Self-hosted - -1. Make sure you have the Dapr runtime initialized: - - ```bash - dapr init - ``` - -1. Set up the environment variables containing the Azure Storage Account credentials: - - {{< tabs Windows "macOS/Linux" >}} - - {{% codetab %}} - ```bash - export STORAGE_ACCOUNT_KEY= - export STORAGE_ACCOUNT_NAME= - ``` - {{% /codetab %}} - - {{% codetab %}} - ```bash - set STORAGE_ACCOUNT_KEY= - set STORAGE_ACCOUNT_NAME= - ``` - {{% /codetab %}} - - {{< /tabs >}} - -1. Move to the workflows directory and run the sample runtime: - - ```bash - cd src/Dapr.Workflows - - dapr run --app-id workflows --protocol grpc --port 3500 --app-port 50003 -- dotnet run --workflows-path ../../samples - ``` - -1. Invoke a workflow: - - ```bash - curl http://localhost:3500/v1.0/invoke/workflows/method/workflow1 - - {"value":"Hello from Logic App workflow running with Dapr!"} - ``` - -### Kubernetes - -1. Make sure you have a running Kubernetes cluster and `kubectl` in your path. - -1. Once you have the Dapr CLI installed, run: - - ```bash - dapr init --kubernetes - ``` - -1. Wait until the Dapr pods have the status `Running`. - -1. Create a Config Map for the workflow: - - ```bash - kubectl create configmap workflows --from-file ./samples/workflow1.json - ``` - -1. Create a secret containing the Azure Storage Account credentials. Replace the account name and key values below with the actual credentials: - - ```bash - kubectl create secret generic dapr-workflows --from-literal=accountName= --from-literal=accountKey= - ``` - -1. Deploy Dapr Worfklows: - - ```bash - kubectl apply -f deploy/deploy.yaml - ``` - -1. Create a port-forward to the dapr workflows container: - - ```bash - kubectl port-forward deploy/dapr-workflows-host 3500:3500 - ``` - -1. Invoke logic apps through Dapr: - - ```bash - curl http://localhost:3500/v1.0/invoke/workflows/method/workflow1 - - {"value":"Hello from Logic App workflow running with Dapr!"} - ``` - -## Invoking workflows using Dapr bindings - -1. First, create any [Dapr binding]({{< ref components-reference >}}) of your choice. See [this]({{< ref howto-triggers >}}) How-To tutorial. - - In order for Dapr Workflows to be able to start a workflow from a Dapr binding event, simply name the binding with the name of the workflow you want it to trigger. - - Here's an example of a Kafka binding that will trigger a workflow named `workflow1`: - - ```yaml - apiVersion: dapr.io/v1alpha1 - kind: Component - metadata: - name: workflow1 - spec: - type: bindings.kafka - metadata: - - name: topics - value: topic1 - - name: brokers - value: localhost:9092 - - name: consumerGroup - value: group1 - - name: authRequired - value: "false" - ``` - -1. Next, apply the Dapr component: - - {{< tabs Self-hosted Kubernetes >}} - - {{% codetab %}} - Place the binding yaml file above in a `components` directory at the root of your application. - {{% /codetab %}} - - {{% codetab %}} - ```bash - kubectl apply -f my_binding.yaml - ``` - {{% /codetab %}} - - {{< /tabs >}} - -1. Once an event is sent to the bindings component, check the logs Dapr Workflows to see the output. - - {{< tabs Self-hosted Kubernetes >}} - - {{% codetab %}} - In standalone mode, the output will be printed to the local terminal. - {{% /codetab %}} - - {{% codetab %}} - On Kubernetes, run the following command: - - ```bash - kubectl logs -l app=dapr-workflows-host -c host - ``` - {{% /codetab %}} - - {{< /tabs >}} - -## Example - -Watch an example from the Dapr community call: - -
- -
- -## Additional resources - -- [Blog announcement](https://cloudblogs.microsoft.com/opensource/2020/05/26/announcing-cloud-native-workflows-dapr-logic-apps/) -- [Repo](https://github.com/dapr/workflows) diff --git a/daprdocs/content/en/developing-applications/integrations/autoscale-keda.md b/daprdocs/content/en/developing-applications/integrations/autoscale-keda.md index 7e685010ffc..b0b70928c8f 100644 --- a/daprdocs/content/en/developing-applications/integrations/autoscale-keda.md +++ b/daprdocs/content/en/developing-applications/integrations/autoscale-keda.md @@ -1,22 +1,27 @@ --- type: docs -title: "Autoscaling a Dapr app with KEDA" -linkTitle: "Autoscale with KEDA" +title: "How to: Autoscale a Dapr app with KEDA" +linkTitle: "How to: Autoscale with KEDA" description: "How to configure your Dapr application to autoscale using KEDA" -weight: 2000 +weight: 3000 --- -Dapr, with its modular building-block approach, along with the 10+ different [pub/sub components]({{< ref pubsub >}}), make it easy to write message processing applications. Since Dapr can run in many environments (e.g. VM, bare-metal, Cloud, or Edge) the autoscaling of Dapr applications is managed by the hosting layer. +Dapr, with its building-block API approach, along with the many [pub/sub components]({{< ref pubsub >}}), makes it easy to write message processing applications. Since Dapr can run in many environments (for example VMs, bare-metal, Cloud or Edge Kubernetes) the autoscaling of Dapr applications is managed by the hosting layer. -For Kubernetes, Dapr integrates with [KEDA](https://github.com/kedacore/keda), an event driven autoscaler for Kubernetes. Many of Dapr's pub/sub components overlap with the scalers provided by [KEDA](https://github.com/kedacore/keda) so it's easy to configure your Dapr deployment on Kubernetes to autoscale based on the back pressure using KEDA. +For Kubernetes, Dapr integrates with [KEDA](https://github.com/kedacore/keda), an event driven autoscaler for Kubernetes. Many of Dapr's pub/sub components overlap with the scalers provided by [KEDA](https://github.com/kedacore/keda), so it's easy to configure your Dapr deployment on Kubernetes to autoscale based on the back pressure using KEDA. -This how-to walks through the configuration of a scalable Dapr application along with the back pressure on Kafka topic, however you can apply this approach to any [pub/sub components]({{< ref pubsub >}}) offered by Dapr. +In this guide, you configure a scalable Dapr application, along with the back pressure on Kafka topic. However, you can apply this approach to _any_ [pub/sub components]({{< ref pubsub >}}) offered by Dapr. + +{{% alert title="Note" color="primary" %}} + If you're working with Azure Container Apps, refer to the official Azure documentation for [scaling Dapr applications using KEDA scalers](https://learn.microsoft.com/azure/container-apps/dapr-keda-scaling). + +{{% /alert %}} ## Install KEDA To install KEDA, follow the [Deploying KEDA](https://keda.sh/docs/latest/deploy/) instructions on the KEDA website. -## Install Kafka (optional) +## Install and deploy Kafka If you don't have access to a Kafka service, you can install it into your Kubernetes cluster for this example by using Helm: @@ -39,16 +44,16 @@ kubectl rollout status statefulset.apps/kafka-cp-kafka -n kafka kubectl rollout status statefulset.apps/kafka-cp-zookeeper -n kafka ``` -When done, also deploy the Kafka client and wait until it's ready: +Once installed, deploy the Kafka client and wait until it's ready: ```shell kubectl apply -n kafka -f deployment/kafka-client.yaml kubectl wait -n kafka --for=condition=ready pod kafka-client --timeout=120s ``` -Next, create the topic which is used in this example (for example `demo-topic`): +## Create the Kafka topic -> The number of topic partitions is related to the maximum number of replicas KEDA creates for your deployments +Create the topic used in this example (`demo-topic`): ```shell kubectl -n kafka exec -it kafka-client -- kafka-topics \ @@ -60,9 +65,11 @@ kubectl -n kafka exec -it kafka-client -- kafka-topics \ --if-not-exists ``` -## Deploy a Dapr Pub/Sub component +> The number of topic `partitions` is related to the maximum number of replicas KEDA creates for your deployments. + +## Deploy a Dapr pub/sub component -Next, we'll deploy the Dapr Kafka pub/sub component for Kubernetes. Paste the following YAML into a file named `kafka-pubsub.yaml`: +Deploy the Dapr Kafka pub/sub component for Kubernetes. Paste the following YAML into a file named `kafka-pubsub.yaml`: ```yaml apiVersion: dapr.io/v1alpha1 @@ -81,9 +88,11 @@ spec: value: autoscaling-subscriber ``` -The above YAML defines the pub/sub component that your application subscribes to, the `demo-topic` we created above. If you used the Kafka Helm install instructions above you can leave the `brokers` value as is. Otherwise, change this to the connection string to your Kafka brokers. +The above YAML defines the pub/sub component that your application subscribes to and that [you created earlier (`demo-topic`)]({{< ref "#create-the-kakfa-topic" >}}). -Also notice the `autoscaling-subscriber` value set for `consumerID` which is used later to make sure that KEDA and your deployment use the same [Kafka partition offset](http://cloudurable.com/blog/kafka-architecture-topics/index.html#:~:text=Kafka%20continually%20appended%20to%20partitions,fit%20on%20a%20single%20server.). +If you used the [Kafka Helm install instructions]({{< ref "#install-and-deploy-kafka" >}}), you can leave the `brokers` value as-is. Otherwise, change this value to the connection string to your Kafka brokers. + +Notice the `autoscaling-subscriber` value set for `consumerID`. This value is used later to ensure that KEDA and your deployment use the same [Kafka partition offset](http://cloudurable.com/blog/kafka-architecture-topics/index.html#:~:text=Kafka%20continually%20appended%20to%20partitions,fit%20on%20a%20single%20server.). Now, deploy the component to the cluster: @@ -93,7 +102,9 @@ kubectl apply -f kafka-pubsub.yaml ## Deploy KEDA autoscaler for Kafka -Next, we will deploy the KEDA scaling object that monitors the lag on the specified Kafka topic and configures the Kubernetes Horizontal Pod Autoscaler (HPA) to scale your Dapr deployment in and out. +Deploy the KEDA scaling object that: +- Monitors the lag on the specified Kafka topic +- Configures the Kubernetes Horizontal Pod Autoscaler (HPA) to scale your Dapr deployment in and out Paste the following into a file named `kafka_scaler.yaml`, and configure your Dapr deployment in the required place: @@ -117,19 +128,25 @@ spec: lagThreshold: "5" ``` -A few things to review here in the above file: +Let's review a few metadata values in the file above: + +| Values | Description | +| ------ | ----------- | +| `scaleTargetRef`/`name` | The Dapr ID of your app defined in the Deployment (The value of the `dapr.io/id` annotation). | +| `pollingInterval` | The frequency in seconds with which KEDA checks Kafka for current topic partition offset. | +| `minReplicaCount` | The minimum number of replicas KEDA creates for your deployment. If your application takes a long time to start, it may be better to set this to `1` to ensure at least one replica of your deployment is always running. Otherwise, set to `0` and KEDA creates the first replica for you. | +| `maxReplicaCount` | The maximum number of replicas for your deployment. Given how [Kafka partition offset](http://cloudurable.com/blog/kafka-architecture-topics/index.html#:~:text=Kafka%20continually%20appended%20to%20partitions,fit%20on%20a%20single%20server.) works, you shouldn't set that value higher than the total number of topic partitions. | +| `triggers`/`metadata`/`topic` | Should be set to the same topic to which your Dapr deployment subscribed (in this example, `demo-topic`). | +| `triggers`/`metadata`/`bootstrapServers` | Should be set to the same broker connection string used in the `kafka-pubsub.yaml` file. | +| `triggers`/`metadata`/`consumerGroup` | Should be set to the same value as the `consumerID` in the `kafka-pubsub.yaml` file. | -* `name` in the `scaleTargetRef` section in the `spec:` is the Dapr ID of your app defined in the Deployment (The value of the `dapr.io/id` annotation) -* `pollingInterval` is the frequency in seconds with which KEDA checks Kafka for current topic partition offset -* `minReplicaCount` is the minimum number of replicas KEDA creates for your deployment. (Note, if your application takes a long time to start it may be better to set that to `1` to ensure at least one replica of your deployment is always running. Otherwise, set that to `0` and KEDA creates the first replica for you) -* `maxReplicaCount` is the maximum number of replicas for your deployment. Given how [Kafka partition offset](http://cloudurable.com/blog/kafka-architecture-topics/index.html#:~:text=Kafka%20continually%20appended%20to%20partitions,fit%20on%20a%20single%20server.) works, you shouldn't set that value higher than the total number of topic partitions -* `topic` in the Kafka `metadata` section which should be set to the same topic to which your Dapr deployment subscribe (In this example `demo-topic`) -* Similarly the `bootstrapServers` should be set to the same broker connection string used in the `kafka-pubsub.yaml` file -* The `consumerGroup` should be set to the same value as the `consumerID` in the `kafka-pubsub.yaml` file +{{% alert title="Important" color="warning" %}} + Setting the connection string, topic, and consumer group to the *same* values for both the Dapr service subscription and the KEDA scaler configuration is critical to ensure the autoscaling works correctly. -> Note: setting the connection string, topic, and consumer group to the *same* values for both the Dapr service subscription and the KEDA scaler configuration is critical to ensure the autoscaling works correctly. +{{% /alert %}} -Next, deploy the KEDA scaler to Kubernetes: + +Deploy the KEDA scaler to Kubernetes: ```bash kubectl apply -f kafka_scaler.yaml @@ -137,6 +154,12 @@ kubectl apply -f kafka_scaler.yaml All done! -Now, that the `ScaledObject` KEDA object is configured, your deployment will scale based on the lag of the Kafka topic. More information on configuring KEDA for Kafka topics is available [here](https://keda.sh/docs/2.0/scalers/apache-kafka/). +## See the KEDA scaler work + +Now that the `ScaledObject` KEDA object is configured, your deployment will scale based on the lag of the Kafka topic. [Learn more about configuring KEDA for Kafka topics](https://keda.sh/docs/2.0/scalers/apache-kafka/). + +As defined in the KEDA scaler manifest, you can now start publishing messages to your Kafka topic `demo-topic` and watch the pods autoscale when the lag threshold is higher than `5` topics. Publish messages to the Kafka Dapr component by using the Dapr [Publish]({{< ref dapr-publish >}}) CLI command. + +## Next steps -You can now start publishing messages to your Kafka topic `demo-topic` and watch the pods autoscale when the lag threshold is higher than `5` topics, as we have defined in the KEDA scaler manifest. You can publish messages to the Kafka Dapr component by using the Dapr [Publish]({{< ref dapr-publish >}}) CLI command +[Learn about scaling your Dapr pub/sub or binding application with KEDA in Azure Container Apps](https://learn.microsoft.com/azure/container-apps/dapr-keda-scaling) \ No newline at end of file diff --git a/daprdocs/content/en/developing-applications/integrations/gRPC-integration.md b/daprdocs/content/en/developing-applications/integrations/gRPC-integration.md index 40b53b07384..c7999a63798 100644 --- a/daprdocs/content/en/developing-applications/integrations/gRPC-integration.md +++ b/daprdocs/content/en/developing-applications/integrations/gRPC-integration.md @@ -1,35 +1,40 @@ --- type: docs -title: "Dapr's gRPC Interface" -linkTitle: "gRPC interface" +title: "How to: Use the gRPC interface in your Dapr application" +linkTitle: "How to: gRPC interface" weight: 6000 description: "Use the Dapr gRPC API in your application" type: docs --- -# Dapr and gRPC +Dapr implements both an HTTP and a gRPC API for local calls. [gRPC](https://grpc.io/) is useful for low-latency, high performance scenarios and has language integration using the proto clients. -Dapr implements both an HTTP and a gRPC API for local calls. gRPC is useful for low-latency, high performance scenarios and has language integration using the proto clients. - -You can find a list of auto-generated clients [here](https://github.com/dapr/docs#sdks). +[Find a list of auto-generated clients in the Dapr SDK documentation]({{< ref sdks >}}). The Dapr runtime implements a [proto service](https://github.com/dapr/dapr/blob/master/dapr/proto/runtime/v1/dapr.proto) that apps can communicate with via gRPC. -In addition to calling Dapr via gRPC, Dapr supports service to service calls with gRPC by acting as a proxy. See more information [here]({{< ref howto-invoke-services-grpc.md >}}). +In addition to calling Dapr via gRPC, Dapr supports service-to-service calls with gRPC by acting as a proxy. [Learn more in the gRPC service invocation how-to guide]({{< ref howto-invoke-services-grpc.md >}}). + +This guide demonstrates configuring and invoking Dapr with gRPC using a Go SDK application. -## Configuring Dapr to communicate with an app via gRPC +## Configure Dapr to communicate with an app via gRPC -### Self hosted +{{< tabs "Self-hosted" "Kubernetes">}} + +{{% codetab %}} -When running in self hosted mode, use the `--app-protocol` flag to tell Dapr to use gRPC to talk to the app: +When running in self-hosted mode, use the `--app-protocol` flag to tell Dapr to use gRPC to talk to the app. ```bash dapr run --app-protocol grpc --app-port 5005 node app.js ``` + This tells Dapr to communicate with your app via gRPC over port `5005`. +{{% /codetab %}} -### Kubernetes + +{{% codetab %}} On Kubernetes, set the following annotations in your deployment YAML: @@ -58,178 +63,195 @@ spec: ... ``` -## Invoking Dapr with gRPC - Go example - -The following steps show you how to create a Dapr client and call the `SaveStateData` operation on it: - -1. Import the package - -```go -package main - -import ( - "context" - "log" - "os" - - dapr "github.com/dapr/go-sdk/client" -) -``` - -2. Create the client - -```go -// just for this demo -ctx := context.Background() -data := []byte("ping") - -// create the client -client, err := dapr.NewClient() -if err != nil { - log.Panic(err) -} -defer client.Close() -``` - -3. Invoke the Save State method - -```go -// save state with the key key1 -err = client.SaveState(ctx, "statestore", "key1", data) -if err != nil { - log.Panic(err) -} -log.Println("data saved") -``` - -Hooray! +{{% /codetab %}} + +{{< /tabs >}} + +## Invoke Dapr with gRPC + +The following steps show how to create a Dapr client and call the `SaveStateData` operation on it. + +1. Import the package: + + ```go + package main + + import ( + "context" + "log" + "os" + + dapr "github.com/dapr/go-sdk/client" + ) + ``` + +1. Create the client: + + ```go + // just for this demo + ctx := context.Background() + data := []byte("ping") + + // create the client + client, err := dapr.NewClient() + if err != nil { + log.Panic(err) + } + defer client.Close() + ``` + + 3. Invoke the `SaveState` method: + + ```go + // save state with the key key1 + err = client.SaveState(ctx, "statestore", "key1", data) + if err != nil { + log.Panic(err) + } + log.Println("data saved") + ``` Now you can explore all the different methods on the Dapr client. -## Creating a gRPC app with Dapr - -The following steps will show you how to create an app that exposes a server for Dapr to communicate with. - -1. Import the package +## Create a gRPC app with Dapr + +The following steps will show how to create an app that exposes a server for with which Dapr can communicate. + +1. Import the package: + + ```go + package main + + import ( + "context" + "fmt" + "log" + "net" + + "github.com/golang/protobuf/ptypes/any" + "github.com/golang/protobuf/ptypes/empty" + + commonv1pb "github.com/dapr/dapr/pkg/proto/common/v1" + pb "github.com/dapr/go-sdk/dapr/proto/runtime/v1" + "google.golang.org/grpc" + ) + ``` + +1. Implement the interface: + + ```go + // server is our user app + type server struct { + pb.UnimplementedAppCallbackServer + } + + // EchoMethod is a simple demo method to invoke + func (s *server) EchoMethod() string { + return "pong" + } + + // This method gets invoked when a remote service has called the app through Dapr + // The payload carries a Method to identify the method, a set of metadata properties and an optional payload + func (s *server) OnInvoke(ctx context.Context, in *commonv1pb.InvokeRequest) (*commonv1pb.InvokeResponse, error) { + var response string + + switch in.Method { + case "EchoMethod": + response = s.EchoMethod() + } + + return &commonv1pb.InvokeResponse{ + ContentType: "text/plain; charset=UTF-8", + Data: &any.Any{Value: []byte(response)}, + }, nil + } + + // Dapr will call this method to get the list of topics the app wants to subscribe to. In this example, we are telling Dapr + // To subscribe to a topic named TopicA + func (s *server) ListTopicSubscriptions(ctx context.Context, in *empty.Empty) (*pb.ListTopicSubscriptionsResponse, error) { + return &pb.ListTopicSubscriptionsResponse{ + Subscriptions: []*pb.TopicSubscription{ + {Topic: "TopicA"}, + }, + }, nil + } + + // Dapr will call this method to get the list of bindings the app will get invoked by. In this example, we are telling Dapr + // To invoke our app with a binding named storage + func (s *server) ListInputBindings(ctx context.Context, in *empty.Empty) (*pb.ListInputBindingsResponse, error) { + return &pb.ListInputBindingsResponse{ + Bindings: []string{"storage"}, + }, nil + } + + // This method gets invoked every time a new event is fired from a registered binding. The message carries the binding name, a payload and optional metadata + func (s *server) OnBindingEvent(ctx context.Context, in *pb.BindingEventRequest) (*pb.BindingEventResponse, error) { + fmt.Println("Invoked from binding") + return &pb.BindingEventResponse{}, nil + } + + // This method is fired whenever a message has been published to a topic that has been subscribed. Dapr sends published messages in a CloudEvents 0.3 envelope. + func (s *server) OnTopicEvent(ctx context.Context, in *pb.TopicEventRequest) (*pb.TopicEventResponse, error) { + fmt.Println("Topic message arrived") + return &pb.TopicEventResponse{}, nil + } + + ``` + +1. Create the server: + + ```go + func main() { + // create listener + lis, err := net.Listen("tcp", ":50001") + if err != nil { + log.Fatalf("failed to listen: %v", err) + } + + // create grpc server + s := grpc.NewServer() + pb.RegisterAppCallbackServer(s, &server{}) + + fmt.Println("Client starting...") + + // and start... + if err := s.Serve(lis); err != nil { + log.Fatalf("failed to serve: %v", err) + } + } + ``` + + This creates a gRPC server for your app on port 50001. + +## Run the application + +{{< tabs "Self-hosted" "Kubernetes">}} + +{{% codetab %}} -```go -package main - -import ( - "context" - "fmt" - "log" - "net" - - "github.com/golang/protobuf/ptypes/any" - "github.com/golang/protobuf/ptypes/empty" - - commonv1pb "github.com/dapr/dapr/pkg/proto/common/v1" - pb "github.com/dapr/go-sdk/dapr/proto/runtime/v1" - "google.golang.org/grpc" -) -``` - -2. Implement the interface - -```go -// server is our user app -type server struct { - pb.UnimplementedAppCallbackServer -} - -// EchoMethod is a simple demo method to invoke -func (s *server) EchoMethod() string { - return "pong" -} - -// This method gets invoked when a remote service has called the app through Dapr -// The payload carries a Method to identify the method, a set of metadata properties and an optional payload -func (s *server) OnInvoke(ctx context.Context, in *commonv1pb.InvokeRequest) (*commonv1pb.InvokeResponse, error) { - var response string - - switch in.Method { - case "EchoMethod": - response = s.EchoMethod() - } - - return &commonv1pb.InvokeResponse{ - ContentType: "text/plain; charset=UTF-8", - Data: &any.Any{Value: []byte(response)}, - }, nil -} - -// Dapr will call this method to get the list of topics the app wants to subscribe to. In this example, we are telling Dapr -// To subscribe to a topic named TopicA -func (s *server) ListTopicSubscriptions(ctx context.Context, in *empty.Empty) (*pb.ListTopicSubscriptionsResponse, error) { - return &pb.ListTopicSubscriptionsResponse{ - Subscriptions: []*pb.TopicSubscription{ - {Topic: "TopicA"}, - }, - }, nil -} - -// Dapr will call this method to get the list of bindings the app will get invoked by. In this example, we are telling Dapr -// To invoke our app with a binding named storage -func (s *server) ListInputBindings(ctx context.Context, in *empty.Empty) (*pb.ListInputBindingsResponse, error) { - return &pb.ListInputBindingsResponse{ - Bindings: []string{"storage"}, - }, nil -} - -// This method gets invoked every time a new event is fired from a registered binding. The message carries the binding name, a payload and optional metadata -func (s *server) OnBindingEvent(ctx context.Context, in *pb.BindingEventRequest) (*pb.BindingEventResponse, error) { - fmt.Println("Invoked from binding") - return &pb.BindingEventResponse{}, nil -} - -// This method is fired whenever a message has been published to a topic that has been subscribed. Dapr sends published messages in a CloudEvents 0.3 envelope. -func (s *server) OnTopicEvent(ctx context.Context, in *pb.TopicEventRequest) (*pb.TopicEventResponse, error) { - fmt.Println("Topic message arrived") - return &pb.TopicEventResponse{}, nil -} - -``` - -3. Create the server - -```go -func main() { - // create listener - lis, err := net.Listen("tcp", ":50001") - if err != nil { - log.Fatalf("failed to listen: %v", err) - } - - // create grpc server - s := grpc.NewServer() - pb.RegisterAppCallbackServer(s, &server{}) - - fmt.Println("Client starting...") +To run locally, use the Dapr CLI: - // and start... - if err := s.Serve(lis); err != nil { - log.Fatalf("failed to serve: %v", err) - } -} +```bash +dapr run --app-id goapp --app-port 50001 --app-protocol grpc go run main.go ``` -This creates a gRPC server for your app on port 50001. +{{% /codetab %}} -4. Run your app + +{{% codetab %}} -To run locally, use the Dapr CLI: +On Kubernetes, set the required `dapr.io/app-protocol: "grpc"` and `dapr.io/app-port: "50001` annotations in your pod spec template, as mentioned above. -``` -dapr run --app-id goapp --app-port 50001 --app-protocol grpc go run main.go -``` +{{% /codetab %}} -On Kubernetes, set the required `dapr.io/app-protocol: "grpc"` and `dapr.io/app-port: "50001` annotations in your pod spec template as mentioned above. +{{< /tabs >}} + ## Other languages You can use Dapr with any language supported by Protobuf, and not just with the currently available generated SDKs. -Using the [protoc](https://developers.google.com/protocol-buffers/docs/downloads) tool you can generate the Dapr clients for other languages like Ruby, C++, Rust and others. + +Using the [protoc](https://developers.google.com/protocol-buffers/docs/downloads) tool, you can generate the Dapr clients for other languages like Ruby, C++, Rust, and others. ## Related Topics - [Service invocation building block]({{< ref service-invocation >}}) diff --git a/daprdocs/content/en/developing-applications/integrations/github_actions.md b/daprdocs/content/en/developing-applications/integrations/github_actions.md index 781ed49a96e..15f87ffb484 100644 --- a/daprdocs/content/en/developing-applications/integrations/github_actions.md +++ b/daprdocs/content/en/developing-applications/integrations/github_actions.md @@ -1,14 +1,16 @@ --- type: docs weight: 5000 -title: "Use the Dapr CLI in a GitHub Actions workflow" -linkTitle: "GitHub Actions" +title: "How to: Use the Dapr CLI in a GitHub Actions workflow" +linkTitle: "How to: GitHub Actions" description: "Add the Dapr CLI to your GitHub Actions to deploy and manage Dapr in your environments." --- Dapr can be integrated with GitHub Actions via the [Dapr tool installer](https://github.com/marketplace/actions/dapr-tool-installer) available in the GitHub Marketplace. This installer adds the Dapr CLI to your workflow, allowing you to deploy, manage, and upgrade Dapr across your environments. -Copy and paste the following installer snippet into your applicatin's YAML file to get started: +## Install the Dapr CLI via the Dapr tool installer + +Copy and paste the following installer snippet into your application's YAML file: ```yaml - name: Dapr tool installer @@ -21,6 +23,8 @@ Refer to the [`action.yml` metadata file](https://github.com/dapr/setup-dapr/blo ## Example +For example, for an application using the [Dapr extention for Azure Kubernetes Service (AKS)]({{< ref azure-kubernetes-service-extension.md >}}), your application YAML will look like the following: + ```yaml - name: Install Dapr uses: dapr/setup-dapr@v1 @@ -45,4 +49,5 @@ Refer to the [`action.yml` metadata file](https://github.com/dapr/setup-dapr/blo ## Next steps -Learn more about [GitHub Actions](https://docs.github.com/en/actions). \ No newline at end of file +- Learn more about [GitHub Actions](https://docs.github.com/en/actions). +- Follow the tutorial to learn how [GitHub Actions works with your Dapr container app (Azure Container Apps)](https://learn.microsoft.com/azure/container-apps/dapr-github-actions?tabs=azure-cli) \ No newline at end of file diff --git a/daprdocs/content/en/developing-applications/integrations/open-service-mesh.md b/daprdocs/content/en/developing-applications/integrations/open-service-mesh.md deleted file mode 100644 index 7df6b35f292..00000000000 --- a/daprdocs/content/en/developing-applications/integrations/open-service-mesh.md +++ /dev/null @@ -1,31 +0,0 @@ ---- -type: docs -title: "Running Dapr and Open Service Mesh together" -linkTitle: "Open Service Mesh" -weight: 4000 -description: "Learn how to run both Open Service Mesh and Dapr on the same Kubernetes cluster" ---- - -## Overview - -[Open Service Mesh (OSM)](https://openservicemesh.io/) is a lightweight, extensible, cloud native service mesh that allows users to uniformly manage, secure, and get out-of-the-box observability features for highly dynamic microservice environments. - -{{< button text="Learn more" link="https://openservicemesh.io/" >}} - -## Dapr integration - -Users are able to leverage both OSM SMI traffic policies and Dapr capabilities on the same Kubernetes cluster. Visit [this guide](https://docs.openservicemesh.io/docs/integrations/demo_dapr/) to get started. - -{{< button text="Deploy OSM and Dapr" link="https://docs.openservicemesh.io/docs/integrations/demo_dapr/" >}} - -## Example - -Watch the OSM team present the OSM and Dapr integration in the 05/18/2021 community call: - -
- -
- -## Additional resources - -- [Dapr and service meshes]({{< ref service-mesh.md >}}) \ No newline at end of file diff --git a/daprdocs/content/en/developing-applications/local-development/ides/vscode/vscode-how-to-debug-multiple-dapr-apps.md b/daprdocs/content/en/developing-applications/local-development/ides/vscode/vscode-how-to-debug-multiple-dapr-apps.md index df5b1089fcf..a2085158100 100644 --- a/daprdocs/content/en/developing-applications/local-development/ides/vscode/vscode-how-to-debug-multiple-dapr-apps.md +++ b/daprdocs/content/en/developing-applications/local-development/ides/vscode/vscode-how-to-debug-multiple-dapr-apps.md @@ -173,8 +173,7 @@ Below are the supported parameters for VS Code tasks. These parameters are equiv | `appId`| The unique ID of the application. Used for service discovery, state encapsulation and the pub/sub consumer ID | Yes | `"appId": "divideapp"` | `appMaxConcurrency` | Limit the concurrency of your application. A valid value is any number larger than 0 | No | `"appMaxConcurrency": -1` | `appPort` | This parameter tells Dapr which port your application is listening on | Yes | `"appPort": 4000` -| `appProtocol` | Tells Dapr which protocol your application is using. Valid options are http and grpc. Default is http | No | `"appProtocol": "http"` -| `appSsl` | Sets the URI scheme of the app to https and attempts an SSL connection | No | `"appSsl": true` +| `appProtocol` | Tells Dapr which protocol your application is using. Valid options are `http`, `grpc`, `https`, `grpcs`, `h2c`. Default is `http`. | No | `"appProtocol": "http"` | `args` | Sets a list of arguments to pass on to the Dapr app | No | "args": [] | `componentsPath` | Path for components directory. If empty, components will not be loaded. | No | `"componentsPath": "./components"` | `config` | Tells Dapr which Configuration CRD to use | No | `"config": "./config"` diff --git a/daprdocs/content/en/getting-started/quickstarts/bindings-quickstart.md b/daprdocs/content/en/getting-started/quickstarts/bindings-quickstart.md index fd8cb0d3707..bac891b5ae3 100644 --- a/daprdocs/content/en/getting-started/quickstarts/bindings-quickstart.md +++ b/daprdocs/content/en/getting-started/quickstarts/bindings-quickstart.md @@ -11,7 +11,7 @@ Let's take a look at Dapr's [Bindings building block]({{< ref bindings >}}). Usi - Trigger your app with events coming in from external systems. - Interface with external systems. -In this Quickstart, you will schedule a batch script to run every 10 seconds using an input [Cron](https://docs.dapr.io/reference/components-reference/supported-bindings/cron/) binding. The script processes a JSON file and outputs data to a SQL database using the [PostgreSQL](https://docs.dapr.io/reference/components-reference/supported-bindings/postgres) Dapr binding. +In this Quickstart, you will schedule a batch script to run every 10 seconds using an input [Cron]({{< ref cron.md >}}) binding. The script processes a JSON file and outputs data to a SQL database using the [PostgreSQL]({{< ref postgresql.md >}}) Dapr binding. @@ -98,7 +98,7 @@ The code inside the `process_batch` function is executed every 10 seconds (defin def process_batch(): ``` -The `batch-sdk` service uses the PostgreSQL output binding defined in the [`binding-postgres.yaml`]({{< ref "#componentbinding-postgresyaml-component-file" >}}) component to insert the `OrderId`, `Customer`, and `Price` records into the `orders` table. +The `batch-sdk` service uses the PostgreSQL output binding defined in the [`binding-postgresql.yaml`]({{< ref "#componentbinding-postgresyaml-component-file" >}}) component to insert the `OrderId`, `Customer`, and `Price` records into the `orders` table. ```python with DaprClient() as d: @@ -140,7 +140,7 @@ In a new terminal, verify the same data has been inserted into the database. Nav cd bindings/db ``` -Run the following to start the interactive Postgres CLI: +Run the following to start the interactive *psql* CLI: ```bash docker exec -i -t postgres psql --username postgres -p 5432 -h localhost --no-password @@ -193,16 +193,16 @@ spec: **Note:** The `metadata` section of `binding-cron.yaml` contains a [Cron expression]({{< ref cron.md >}}) that specifies how often the binding is invoked. -#### `component\binding-postgres.yaml` component file +#### `component\binding-postgresql.yaml` component file When you execute the `dapr run` command and specify the component path, the Dapr sidecar: -- Initiates the PostgreSQL [binding building block]({{< ref postgres.md >}}) -- Connects to PostgreSQL using the settings specified in the `binding-postgres.yaml` file +- Initiates the PostgreSQL [binding building block]({{< ref postgresql.md >}}) +- Connects to PostgreSQL using the settings specified in the `binding-postgresql.yaml` file -With the `binding-postgres.yaml` component, you can easily swap out the backend database [binding]({{< ref supported-bindings.md >}}) without making code changes. +With the `binding-postgresql.yaml` component, you can easily swap out the backend database [binding]({{< ref supported-bindings.md >}}) without making code changes. -The PostgreSQL `binding-postgres.yaml` file included for this Quickstart contains the following: +The PostgreSQL `binding-postgresql.yaml` file included for this Quickstart contains the following: ```yaml apiVersion: dapr.io/v1alpha1 @@ -211,7 +211,7 @@ metadata: name: sqldb namespace: quickstarts spec: - type: bindings.postgres + type: bindings.postgresql version: v1 metadata: - name: url # Required @@ -304,7 +304,7 @@ async function start() { } ``` -The `batch-sdk` service uses the PostgreSQL output binding defined in the [`binding-postgres.yaml`]({{< ref "##componentsbinding-postgresyaml-component-file" >}}) component to insert the `OrderId`, `Customer`, and `Price` records into the `orders` table. +The `batch-sdk` service uses the PostgreSQL output binding defined in the [`binding-postgresql.yaml`]({{< ref "##componentsbinding-postgresyaml-component-file" >}}) component to insert the `OrderId`, `Customer`, and `Price` records into the `orders` table. ```javascript async function processBatch(){ @@ -395,16 +395,16 @@ spec: **Note:** The `metadata` section of `binding-cron.yaml` contains a [Cron expression]({{< ref cron.md >}}) that specifies how often the binding is invoked. -#### `component\binding-postgres.yaml` component file +#### `component\binding-postgresql.yaml` component file When you execute the `dapr run` command and specify the component path, the Dapr sidecar: -- Initiates the PostgreSQL [binding building block]({{< ref postgres.md >}}) -- Connects to PostgreSQL using the settings specified in the `binding-postgres.yaml` file +- Initiates the PostgreSQL [binding building block]({{< ref postgresql.md >}}) +- Connects to PostgreSQL using the settings specified in the `binding-postgresql.yaml` file -With the `binding-postgres.yaml` component, you can easily swap out the backend database [binding]({{< ref supported-bindings.md >}}) without making code changes. +With the `binding-postgresql.yaml` component, you can easily swap out the backend database [binding]({{< ref supported-bindings.md >}}) without making code changes. -The PostgreSQL `binding-postgres.yaml` file included for this Quickstart contains the following: +The PostgreSQL `binding-postgresql.yaml` file included for this Quickstart contains the following: ```yaml apiVersion: dapr.io/v1alpha1 @@ -413,7 +413,7 @@ metadata: name: sqldb namespace: quickstarts spec: - type: bindings.postgres + type: bindings.postgresql version: v1 metadata: - name: url # Required @@ -506,7 +506,7 @@ app.MapPost("/" + cronBindingName, async () => { }); ``` -The `batch-sdk` service uses the PostgreSQL output binding defined in the [`binding-postgres.yaml`]({{< ref "#componentbinding-postgresyaml-component-file" >}}) component to insert the `OrderId`, `Customer`, and `Price` records into the `orders` table. +The `batch-sdk` service uses the PostgreSQL output binding defined in the [`binding-postgresql.yaml`]({{< ref "#componentbinding-postgresyaml-component-file" >}}) component to insert the `OrderId`, `Customer`, and `Price` records into the `orders` table. ```csharp // ... @@ -599,16 +599,16 @@ spec: **Note:** The `metadata` section of `binding-cron.yaml` contains a [Cron expression]({{< ref cron.md >}}) that specifies how often the binding is invoked. -#### `component\binding-postgres.yaml` component file +#### `component\binding-postgresql.yaml` component file When you execute the `dapr run` command and specify the component path, the Dapr sidecar: -- Initiates the PostgreSQL [binding building block]({{< ref postgres.md >}}) -- Connects to PostgreSQL using the settings specified in the `binding-postgres.yaml` file +- Initiates the PostgreSQL [binding building block]({{< ref postgresql.md >}}) +- Connects to PostgreSQL using the settings specified in the `binding-postgresql.yaml` file -With the `binding-postgres.yaml` component, you can easily swap out the backend database [binding]({{< ref supported-bindings.md >}}) without making code changes. +With the `binding-postgresql.yaml` component, you can easily swap out the backend database [binding]({{< ref supported-bindings.md >}}) without making code changes. -The PostgreSQL `binding-postgres.yaml` file included for this Quickstart contains the following: +The PostgreSQL `binding-postgresql.yaml` file included for this Quickstart contains the following: ```yaml apiVersion: dapr.io/v1alpha1 @@ -617,7 +617,7 @@ metadata: name: sqldb namespace: quickstarts spec: - type: bindings.postgres + type: bindings.postgresql version: v1 metadata: - name: url # Required @@ -711,7 +711,7 @@ The code inside the `process_batch` function is executed every 10 seconds (defin public ResponseEntity processBatch() throws IOException, Exception ``` -The `batch-sdk` service uses the PostgreSQL output binding defined in the [`binding-postgres.yaml`]({{< ref "#componentbinding-postgresyaml-component-file" >}}) component to insert the `OrderId`, `Customer`, and `Price` records into the `orders` table. +The `batch-sdk` service uses the PostgreSQL output binding defined in the [`binding-postgresql.yaml`]({{< ref "#componentbinding-postgresyaml-component-file" >}}) component to insert the `OrderId`, `Customer`, and `Price` records into the `orders` table. ```java try (DaprClient client = new DaprClientBuilder().build()) { @@ -809,16 +809,16 @@ spec: **Note:** The `metadata` section of `binding-cron.yaml` contains a [Cron expression]({{< ref cron.md >}}) that specifies how often the binding is invoked. -#### `component\binding-postgres.yaml` component file +#### `component\binding-postgresql.yaml` component file When you execute the `dapr run` command and specify the component path, the Dapr sidecar: -- Initiates the PostgreSQL [binding building block]({{< ref postgres.md >}}) -- Connects to PostgreSQL using the settings specified in the `binding-postgres.yaml` file +- Initiates the PostgreSQL [binding building block]({{< ref postgresql.md >}}) +- Connects to PostgreSQL using the settings specified in the `binding-postgresql.yaml` file -With the `binding-postgres.yaml` component, you can easily swap out the backend database [binding]({{< ref supported-bindings.md >}}) without making code changes. +With the `binding-postgresql.yaml` component, you can easily swap out the backend database [binding]({{< ref supported-bindings.md >}}) without making code changes. -The PostgreSQL `binding-postgres.yaml` file included for this Quickstart contains the following: +The PostgreSQL `binding-postgresql.yaml` file included for this Quickstart contains the following: ```yaml apiVersion: dapr.io/v1alpha1 @@ -827,7 +827,7 @@ metadata: name: sqldb namespace: quickstarts spec: - type: bindings.postgres + type: bindings.postgresql version: v1 metadata: - name: url # Required @@ -918,7 +918,7 @@ The code inside the `process_batch` function is executed every 10 seconds (defin r.HandleFunc("/"+cronBindingName, processBatch).Methods("POST") ``` -The `batch-sdk` service uses the PostgreSQL output binding defined in the [`binding-postgres.yaml`]({{< ref "#componentbinding-postgresyaml-component-file" >}}) component to insert the `OrderId`, `Customer`, and `Price` records into the `orders` table. +The `batch-sdk` service uses the PostgreSQL output binding defined in the [`binding-postgresql.yaml`]({{< ref "#componentbinding-postgresyaml-component-file" >}}) component to insert the `OrderId`, `Customer`, and `Price` records into the `orders` table. ```go func sqlOutput(order Order) (err error) { @@ -1021,16 +1021,16 @@ spec: **Note:** The `metadata` section of `binding-cron.yaml` contains a [Cron expression]({{< ref cron.md >}}) that specifies how often the binding is invoked. -#### `component\binding-postgres.yaml` component file +#### `component\binding-postgresql.yaml` component file When you execute the `dapr run` command and specify the component path, the Dapr sidecar: -- Initiates the PostgreSQL [binding building block]({{< ref postgres.md >}}) -- Connects to PostgreSQL using the settings specified in the `binding-postgres.yaml` file +- Initiates the PostgreSQL [binding building block]({{< ref postgresql.md >}}) +- Connects to PostgreSQL using the settings specified in the `binding-postgresql.yaml` file -With the `binding-postgres.yaml` component, you can easily swap out the backend database [binding]({{< ref supported-bindings.md >}}) without making code changes. +With the `binding-postgresql.yaml` component, you can easily swap out the backend database [binding]({{< ref supported-bindings.md >}}) without making code changes. -The PostgreSQL `binding-postgres.yaml` file included for this Quickstart contains the following: +The PostgreSQL `binding-postgresql.yaml` file included for this Quickstart contains the following: ```yaml apiVersion: dapr.io/v1alpha1 @@ -1039,7 +1039,7 @@ metadata: name: sqldb namespace: quickstarts spec: - type: bindings.postgres + type: bindings.postgresql version: v1 metadata: - name: url # Required diff --git a/daprdocs/content/en/getting-started/quickstarts/workflow-quickstart.md b/daprdocs/content/en/getting-started/quickstarts/workflow-quickstart.md index ceeaba42a0f..0ee6fa8091d 100644 --- a/daprdocs/content/en/getting-started/quickstarts/workflow-quickstart.md +++ b/daprdocs/content/en/getting-started/quickstarts/workflow-quickstart.md @@ -151,25 +151,29 @@ string orderId = Guid.NewGuid().ToString()[..8]; string itemToPurchase = "Cars"; int ammountToPurchase = 10; -//... +// Construct the order +OrderPayload orderInfo = new OrderPayload(itemToPurchase, 15000, ammountToPurchase); // Start the workflow Console.WriteLine("Starting workflow {0} purchasing {1} {2}", orderId, ammountToPurchase, itemToPurchase); -await workflowClient.ScheduleNewWorkflowAsync( - name: nameof(OrderProcessingWorkflow), - instanceId: orderId, - input: orderInfo); - -//... +await daprClient.StartWorkflowAsync( + workflowComponent: DaprWorkflowComponent, + workflowName: nameof(OrderProcessingWorkflow), + input: orderInfo, + instanceId: orderId); -WorkflowState state = await workflowClient.GetWorkflowStateAsync( +// Wait for the workflow to start and confirm the input +GetWorkflowResponse state = await daprClient.WaitForWorkflowStartAsync( instanceId: orderId, - getInputsAndOutputs: true); + workflowComponent: DaprWorkflowComponent); -Console.WriteLine("Your workflow has started. Here is the status of the workflow: {0}", state); +Console.WriteLine("Your workflow has started. Here is the status of the workflow: {0}", state.RuntimeStatus); -//... +// Wait for the workflow to complete +state = await daprClient.WaitForWorkflowCompletionAsync( + instanceId: orderId, + workflowComponent: DaprWorkflowComponent); Console.WriteLine("Workflow Status: {0}", state.RuntimeStatus); ``` diff --git a/daprdocs/content/en/operations/configuration/configuration-overview.md b/daprdocs/content/en/operations/configuration/configuration-overview.md index 40eb09427f5..1f04714d7ba 100644 --- a/daprdocs/content/en/operations/configuration/configuration-overview.md +++ b/daprdocs/content/en/operations/configuration/configuration-overview.md @@ -79,13 +79,22 @@ The following table lists the properties for tracing: | `samplingRate` | string | Set sampling rate for tracing to be enabled or disabled. | `stdout` | bool | True write more verbose information to the traces | `otel.endpointAddress` | string | Set the Open Telemetry (OTEL) server address to send traces to -| `otel.isSecure` | bool | Is the connection to the endpoint address encryped +| `otel.isSecure` | bool | Is the connection to the endpoint address encrypted | `otel.protocol` | string | Set to `http` or `grpc` protocol | `zipkin.endpointAddress` | string | Set the Zipkin server address to send traces to `samplingRate` is used to enable or disable the tracing. To disable the sampling rate , set `samplingRate : "0"` in the configuration. The valid range of samplingRate is between 0 and 1 inclusive. The sampling rate determines whether a trace span should be sampled or not based on value. `samplingRate : "1"` samples all traces. By default, the sampling rate is (0.0001) or 1 in 10,000 traces. +The OpenTelemetry (otel) endpoint can also be configured via an environment variables. The presence of the OTEL_EXPORTER_OTLP_ENDPOINT environment variable +turns on tracing for the sidecar. + +| Environment Variable | Description | +|----------------------|-------------| +| `OTEL_EXPORTER_OTLP_ENDPOINT` | Sets the Open Telemetry (OTEL) server address, turns on tracing | +| `OTEL_EXPORTER_OTLP_INSECURE` | Sets the connection to the endpoint as unencrypted (true/false) | +| `OTEL_EXPORTER_OTLP_PROTOCOL` | Transport protocol (`grpc`, `http/protobuf`, `http/json`) | + See [Observability distributed tracing]({{< ref "tracing-overview.md" >}}) for more information. #### Metrics diff --git a/daprdocs/content/en/operations/hosting/self-hosted/self-hosted-with-podman.md b/daprdocs/content/en/operations/hosting/self-hosted/self-hosted-with-podman.md index b53aaf9232b..fda567276bf 100644 --- a/daprdocs/content/en/operations/hosting/self-hosted/self-hosted-with-podman.md +++ b/daprdocs/content/en/operations/hosting/self-hosted/self-hosted-with-podman.md @@ -11,7 +11,7 @@ This article provides guidance on running Dapr with Podman on a Windows/Linux/ma ## Prerequisites - [Dapr CLI]({{< ref install-dapr-cli.md >}}) -- [Podman](https://podman.io/docs/tutorials/installation) +- [Podman](https://podman-desktop.io/downloads) ## Initialize Dapr environment diff --git a/daprdocs/content/en/operations/monitoring/tracing/setup-tracing.md b/daprdocs/content/en/operations/monitoring/tracing/setup-tracing.md index 5e5e7ce92f7..52147498cf8 100644 --- a/daprdocs/content/en/operations/monitoring/tracing/setup-tracing.md +++ b/daprdocs/content/en/operations/monitoring/tracing/setup-tracing.md @@ -33,7 +33,7 @@ The following table lists the properties for tracing: | `samplingRate` | string | Set sampling rate for tracing to be enabled or disabled. | `stdout` | bool | True write more verbose information to the traces | `otel.endpointAddress` | string | Set the Open Telemetry (OTEL) server address. -| `otel.isSecure` | bool | Is the connection to the endpoint address encryped. +| `otel.isSecure` | bool | Is the connection to the endpoint address encrypted. | `otel.protocol` | string | Set to `http` or `grpc` protocol. | `zipkin.endpointAddress` | string | Set the Zipkin server address. If this is used, you do not need to specify the `otel` section. @@ -58,3 +58,14 @@ spec: Dapr uses probabilistic sampling. The sample rate defines the probability a tracing span will be sampled and can have a value between 0 and 1 (inclusive). The default sample rate is 0.0001 (i.e. 1 in 10,000 spans is sampled). Changing `samplingRate` to 0 disables tracing altogether. + +## Environment variables + +The OpenTelemetry (otel) endpoint can also be configured via an environment variables. The presence of the OTEL_EXPORTER_OTLP_ENDPOINT environment variable +turns on tracing for the sidecar. + +| Environment Variable | Description | +|----------------------|-------------| +| `OTEL_EXPORTER_OTLP_ENDPOINT` | Sets the Open Telemetry (OTEL) server address, turns on tracing | +| `OTEL_EXPORTER_OTLP_INSECURE` | Sets the connection to the endpoint as unencrypted (true/false) | +| `OTEL_EXPORTER_OTLP_PROTOCOL` | Transport protocol (`grpc`, `http/protobuf`, `http/json`) | \ No newline at end of file diff --git a/daprdocs/content/en/operations/support/support-preview-features.md b/daprdocs/content/en/operations/support/support-preview-features.md index 71c649fe62f..92522d4969a 100644 --- a/daprdocs/content/en/operations/support/support-preview-features.md +++ b/daprdocs/content/en/operations/support/support-preview-features.md @@ -17,10 +17,10 @@ For CLI there is no explicit opt-in, just the version that this was first made a | --- | --- | --- | --- | --- | | **App Middleware** | Allow middleware components to be executed when making service-to-service calls | N/A | [App Middleware]({{}}) | v1.9 | | **Streaming for HTTP service invocation** | Enables (partial) support for using streams in HTTP service invocation; see below for more details. | `ServiceInvocationStreaming` | [Details]({{< ref "support-preview-features.md#streaming-for-http-service-invocation" >}}) | v1.10 | -| **App health checks** | Allows configuring app health checks | `AppHealthCheck` | [App health checks]({{}}) | v1.9 | | **Pluggable components** | Allows creating self-hosted gRPC-based components written in any language that supports gRPC. The following component APIs are supported: State stores, Pub/sub, Bindings | N/A | [Pluggable components concept]({{}})| v1.9 | | **Multi-App Run** | Configure multiple Dapr applications from a single configuration file and run from a single command | `dapr run -f` | [Multi-App Run]({{< ref multi-app-dapr-run.md >}}) | v1.10 | | **Workflows** | Author workflows as code to automate and orchestrate tasks within your application, like messaging, state management, and failure handling | N/A | [Workflows concept]({{< ref "components-concept#workflows" >}})| v1.10 | +| **Service invocation for non-Dapr endpoints** | Allow the invocation of non-Dapr endpoints by Dapr using the [Service invocation API]({{< ref service_invocation_api.md >}}). Read ["How-To: Invoke Non-Dapr Endpoints using HTTP"]({{< ref howto-invoke-non-dapr-endpoints.md >}}) for more information. | N/A | [Service invocation API]({{< ref service_invocation_api.md >}}) | v1.11 | ### Streaming for HTTP service invocation @@ -30,14 +30,14 @@ The table below summarizes the current state of support for streaming in HTTP se Diagram showing the steps of service invocation described in the table below -| Step | Handles data as a stream | Dapr 1.10 | Dapr 1.10 with
`ServiceInvocationStreaming` | +| Step | Handles data as a stream | Dapr 1.11 | Dapr 1.11 with
`ServiceInvocationStreaming` | |:---:|---|:---:|:---:| | 1 | Request: "App A" to "Dapr sidecar A | ❌ | ❌ | | 2 | Request: "Dapr sidecar A" to "Dapr sidecar B | ❌ | ✅ | | 3 | Request: "Dapr sidecar B" to "App B" | ✅ | ✅ | | 4 | Response: "App B" to "Dapr sidecar B" | ✅ | ✅ | | 5 | Response: "Dapr sidecar B" to "Dapr sidecar A | ❌ | ✅ | -| 6 | Response: "Dapr sidecar A" to "App A | ❌ | ✅ | +| 6 | Response: "Dapr sidecar A" to "App A | ❌ | ❌ | Important notes: diff --git a/daprdocs/content/en/reference/api/actors_api.md b/daprdocs/content/en/reference/api/actors_api.md index 81ee3cf7950..180bea672bf 100644 --- a/daprdocs/content/en/reference/api/actors_api.md +++ b/daprdocs/content/en/reference/api/actors_api.md @@ -75,6 +75,14 @@ Persists the change to the state for an actor as a multi-item transaction. ***Note that this operation is dependant on a using state store component that supports multi-item transactions.*** +When putting state, _always_ set the `ttlInSeconds` field in the +metadata for each value, unless there is a state clean up process out of band of +Dapr. Omitting this field will result in the underlying Actor state store to +grow indefinitely. + +See the Dapr Community Call 80 recording for more details on actor state TTL. + + #### HTTP Request ``` @@ -109,7 +117,10 @@ curl -X POST http://localhost:3500/v1.0/actors/stormtrooper/50/state \ "operation": "upsert", "request": { "key": "key1", - "value": "myData" + "value": "myData", + "metadata": { + "ttlInSeconds": "3600" + } } }, { @@ -176,7 +187,7 @@ Creates a persistent reminder for an actor. POST/PUT http://localhost:/v1.0/actors///reminders/ ``` -#### Request Body +#### Reminder request body A JSON object with the following fields: @@ -340,7 +351,8 @@ Creates a timer for an actor. POST/PUT http://localhost:/v1.0/actors///timers/ ``` -Body: +#### Timer request body: +The format for the timer request body is the same as for [actor reminders]({{< ref "#reminder-request-body" >}}). For example: The following specifies a `dueTime` of 3 seconds and a period of 7 seconds. @@ -473,6 +485,16 @@ Parameter | Description `maxStackDepth` | A value in the reentrancy configuration that controls how many reentrant calls be made to the same actor. `entitiesConfig` | Array of entity configurations that allow per actor type settings. Any configuration defined here must have an entity that maps back into the root level entities. + +{{% alert title="Note" color="primary" %}} +Actor settings in configuration for timeouts and intervals use [time.ParseDuration](https://pkg.go.dev/time#ParseDuration) format. You can use string formats to represent durations. For example: +- `1h30m` or `1.5h`: A duration of 1 hour and 30 minutes +- `1d12h`: A duration of 1 day and 12 hours +- `500ms`: A duration of 500 milliseconds +- `-30m`: A negative duration of 30 minutes + +{{% /alert %}} + ```json { "entities":["actorType1", "actorType2"], diff --git a/daprdocs/content/en/reference/api/metadata_api.md b/daprdocs/content/en/reference/api/metadata_api.md index 33671101361..b0c39c85513 100644 --- a/daprdocs/content/en/reference/api/metadata_api.md +++ b/daprdocs/content/en/reference/api/metadata_api.md @@ -6,7 +6,7 @@ description: "Detailed documentation on the Metadata API" weight: 1100 --- -Dapr has a metadata API that returns information about the sidecar allowing runtime discoverability. The metadata endpoint returns a list of the components loaded, the activated actors (if present) and attributes with information attached. +Dapr has a metadata API that returns information about the sidecar allowing runtime discoverability. The metadata endpoint returns a list of the resources (components and HttpEndpoints loaded), the activated actors (if present), and attributes with information attached. ## Components Each loaded component provides its name, type and version and also information about supported features in the form of component capabilities. @@ -17,6 +17,9 @@ Component type | Capabilities State Store | ETAG, TRANSACTION, ACTOR, QUERY_API Binding | INPUT_BINDING, OUTPUT_BINDING +## HTTPEndpoints +Each loaded `HttpEndpoint` provides a name to easily identify the Dapr resource associated with the runtime. + ## Attributes The metadata API allows you to store additional attribute information in the format of key-value pairs. These are ephemeral in-memory and are not persisted if a sidecar is reloaded. This information should be added at the time of a sidecar creation, for example, after the application has started. diff --git a/daprdocs/content/en/reference/api/service_invocation_api.md b/daprdocs/content/en/reference/api/service_invocation_api.md index 15de8235141..8ecd243ef36 100644 --- a/daprdocs/content/en/reference/api/service_invocation_api.md +++ b/daprdocs/content/en/reference/api/service_invocation_api.md @@ -6,17 +6,29 @@ description: "Detailed documentation on the service invocation API" weight: 100 --- -Dapr provides users with the ability to call other applications that have unique ids. -This functionality allows apps to interact with one another via named identifiers and puts the burden of service discovery on the Dapr runtime. +Dapr provides users with the ability to call other applications that are using Dapr with a unique named identifier (appId), or HTTP endpoints that are not using Dapr. +This allows applications to interact with one another via named identifiers and puts the burden of service discovery on the Dapr runtime. -## Invoke a method on a remote dapr app +## Invoke a method on a remote Dapr app This endpoint lets you invoke a method in another Dapr enabled app. ### HTTP Request ``` -PATCH/POST/GET/PUT/DELETE http://localhost:/v1.0/invoke//method/ +PATCH/POST/GET/PUT/DELETE http://localhost:/v1.0/invoke//method/ +``` + +## Invoke a method on a non-Dapr endpoint + +This endpoint lets you invoke a method on a non-Dapr endpoint using an `HTTPEndpoint` resource name, or a Fully Qualified Domain Name (FQDN) URL. + +### HTTP Request + +``` +PATCH/POST/GET/PUT/DELETE http://localhost:/v1.0/invoke//method/ + +PATCH/POST/GET/PUT/DELETE http://localhost:/v1.0/invoke//method/ ``` ### HTTP Response codes @@ -38,7 +50,9 @@ XXX | Upstream status returned Parameter | Description --------- | ----------- daprPort | the Dapr port -appId | the App ID associated with the remote app +appID | the App ID associated with the remote app +HTTPEndpoint name | the HTTPEndpoint resource associated with the external endpoint +FQDN URL | Fully Qualified Domain Name URL to invoke on the external endpoint method-name | the name of the method or url to invoke on the remote app > Note, all URL parameters are case-sensitive. @@ -65,9 +79,9 @@ Within the body of the request place the data you want to send to the service: ### Request received by invoked service -Once your service code invokes a method in another Dapr enabled app, Dapr will send the request, along with the headers and body, to the app on the `` endpoint. +Once your service code invokes a method in another Dapr enabled app or non-Dapr endpoint, Dapr sends the request, along with the headers and body, on the `` endpoint. -The Dapr app being invoked will need to be listening for and responding to requests on that endpoint. +The Dapr app or non-Dapr endpoint being invoked will need to be listening for and responding to requests on that endpoint. ### Cross namespace invocation @@ -120,5 +134,19 @@ In case you are invoking `mathService` on a different namespace, you can use the In this URL, `testing` is the namespace that `mathService` is running in. +#### Non-Dapr Endpoint Example + +If the `mathService` service was a non-Dapr application, then it could be invoked using service invocation via an `HTTPEndpoint`, as well as a Fully Qualified Domain Name (FQDN) URL. + +```shell +curl http://localhost:3500/v1.0/invoke/mathHTTPEndpoint/method/add \ + -H "Content-Type: application/json" + -d '{ "arg1": 10, "arg2": 23}' + +curl http://localhost:3500/v1.0/invoke/http://mathServiceURL.com/method/add \ + -H "Content-Type: application/json" + -d '{ "arg1": 10, "arg2": 23}' +``` + ## Next Steps - [How-To: Invoke and discover services]({{< ref howto-invoke-discover-services.md >}}) diff --git a/daprdocs/content/en/reference/api/workflow_api.md b/daprdocs/content/en/reference/api/workflow_api.md index 442b206ad29..4ddf8b7209f 100644 --- a/daprdocs/content/en/reference/api/workflow_api.md +++ b/daprdocs/content/en/reference/api/workflow_api.md @@ -12,7 +12,7 @@ Dapr provides users with the ability to interact with workflows and comes with a Start a workflow instance with the given name and optionally, an instance ID. -```bash +```http POST http://localhost:3500/v1.0-alpha1/workflows///start[?instanceId=] ``` @@ -22,7 +22,7 @@ Note that workflow instance IDs can only contain alphanumeric characters, unders Parameter | Description --------- | ----------- -`workflowComponentName` | Current default is `dapr` for Dapr Workflows +`workflowComponentName` | Use `dapr` for Dapr Workflows `workflowName` | Identify the workflow type `instanceId` | (Optional) Unique value created for each run of a specific workflow @@ -52,7 +52,7 @@ The API call will provide a response similar to this: Terminate a running workflow instance with the given name and instance ID. -```bash +```http POST http://localhost:3500/v1.0-alpha1/workflows//terminate ``` @@ -60,7 +60,7 @@ POST http://localhost:3500/v1.0-alpha1/workflows//terminate Parameter | Description --------- | ----------- -`workflowComponentName` | Current default is `dapr` for Dapr Workflows +`workflowComponentName` | Use `dapr` for Dapr Workflows `instanceId` | Unique value created for each run of a specific workflow ### HTTP response codes @@ -75,11 +75,125 @@ Code | Description This API does not return any content. -### Get workflow request +## Raise Event request + +For workflow components that support subscribing to external events, such as the Dapr Workflow engine, you can use the following "raise event" API to deliver a named event to a specific workflow instance. + +```http +POST http://localhost:3500/v1.0-alpha1/workflows///raiseEvent/ +``` + +{{% alert title="Note" color="primary" %}} + The exact mechanism for subscribing to an event depends on the workflow component that you're using. Dapr Workflow has one way of subscribing to external events but other workflow components might have different ways. + +{{% /alert %}} + +### URL parameters + +Parameter | Description +--------- | ----------- +`workflowComponentName` | Use `dapr` for Dapr Workflows +`instanceId` | Unique value created for each run of a specific workflow +`eventName` | The name of the event to raise + +### HTTP response codes + +Code | Description +---- | ----------- +`202` | Accepted +`400` | Request was malformed +`500` | Request formatted correctly, error in dapr code or underlying component + +### Response content + +None. + +## Pause workflow request + +Pause a running workflow instance. + +```http +POST http://localhost:3500/v1.0-alpha1/workflows///pause +``` + +### URL parameters + +Parameter | Description +--------- | ----------- +`workflowComponentName` | Use `dapr` for Dapr Workflows +`instanceId` | Unique value created for each run of a specific workflow + +### HTTP response codes + +Code | Description +---- | ----------- +`202` | Accepted +`400` | Request was malformed +`500` | Error in Dapr code or underlying component + +### Response content + +None. + +## Resume workflow request + +Resume a paused workflow instance. + +```http +POST http://localhost:3500/v1.0-alpha1/workflows///resume +``` + +### URL parameters + +Parameter | Description +--------- | ----------- +`workflowComponentName` | Use `dapr` for Dapr Workflows +`instanceId` | Unique value created for each run of a specific workflow + +### HTTP response codes + +Code | Description +---- | ----------- +`202` | Accepted +`400` | Request was malformed +`500` | Error in Dapr code or underlying component + +### Response content + +None. + +## Purge workflow request + +Purge the workflow state from your state store with the workflow's instance ID. + +```http +POST http://localhost:3500/v1.0-alpha1/workflows///purge +``` + +### URL parameters + +Parameter | Description +--------- | ----------- +`workflowComponentName` | Use `dapr` for Dapr Workflows +`instanceId` | Unique value created for each run of a specific workflow + +### HTTP response codes + +Code | Description +---- | ----------- +`202` | Accepted +`400` | Request was malformed +`500` | Error in Dapr code or underlying component + +### Response content + +None. + +## Get workflow request Get information about a given workflow instance. -```bash +```http GET http://localhost:3500/v1.0-alpha1/workflows// ``` @@ -87,7 +201,7 @@ GET http://localhost:3500/v1.0-alpha1/workflows//}}) directly, via daprd, or on [Kubernetes]({{< ref kubernetes-overview.md >}}) via annotations. +This table is meant to help users understand the equivalent options for running Dapr sidecars in different contexts: via the [CLI]({{< ref cli-overview.md >}}) directly, via daprd, or on [Kubernetes]({{< ref kubernetes-overview.md >}}) via annotations. | daprd | Dapr CLI | CLI shorthand | Kubernetes annotations | Description| |----- | ------- | -----------| ----------| ------------ | | `--allowed-origins` | not supported | | not supported | Allowed HTTP origins (default "*") | | `--app-id` | `--app-id` | `-i` | `dapr.io/app-id` | The unique ID of the application. Used for service discovery, state encapsulation and the pub/sub consumer ID | | `--app-port` | `--app-port` | `-p` | `dapr.io/app-port` | This parameter tells Dapr which port your application is listening on | -| `--app-ssl` | `--app-ssl` | | `dapr.io/app-ssl` | Sets the URI scheme of the app to https and attempts an SSL connection | | `--components-path` | `--components-path` | `-d` | not supported | **Deprecated** in favor of `--resources-path` | | `--resources-path` | `--resources-path` | `-d` | not supported | Path for components directory. If empty, components will not be loaded. | | `--config` | `--config` | `-c` | `dapr.io/config` | Tells Dapr which Configuration CRD to use | @@ -37,8 +36,8 @@ This table is meant to help users understand the equivalent options for running | `--metrics-port` | `--metrics-port` | | `dapr.io/metrics-port` | Sets the port for the sidecar metrics server. Default is `9090` | | `--mode` | not supported | | not supported | Runtime mode for Dapr (default "standalone") | | `--placement-host-address` | `--placement-host-address` | | `dapr.io/placement-host-address` | Comma separated list of addresses for Dapr Actor Placement servers. When no annotation is set, the default value is set by the Sidecar Injector. When the annotation is set and the value is empty, the sidecar does not connect to Placement server. This can be used when there are no actors running in the sidecar. When the annotation is set and the value is not empty, the sidecar connects to the configured address. For example: `127.0.0.1:50057,127.0.0.1:50058` | -| `--profiling-port` | `--profiling-port` | | not supported | The port for the profile server (default "7777") | -| `--app-protocol` | `--app-protocol` | `-P` | `dapr.io/app-protocol` | Tells Dapr which protocol your application is using. Valid options are `http` and `grpc`. Default is `http` | +| `--profiling-port` | `--profiling-port` | | not supported | The port for the profile server (default `7777`) | +| `--app-protocol` | `--app-protocol` | `-P` | `dapr.io/app-protocol` | Configures the protocol Dapr uses to communicate with your app. Valid options are `http`, `grpc`, `https` (HTTP with TLS), `grpcs` (gRPC with TLS), `h2c` (HTTP/2 Cleartext). Note that Dapr does not validate TLS certificates presented by the app. Default is `http` | | `--enable-app-health-check` | `--enable-app-health-check` | | `dapr.io/enable-app-health-check` | Boolean that enables the health checks. Default is `false`. | | `--app-health-check-path` | `--app-health-check-path` | | `dapr.io/app-health-check-path` | Path that Dapr invokes for health probes when the app channel is HTTP (this value is ignored if the app channel is using gRPC). Requires app health checks to be enabled. Default is `/health` | | `--app-health-probe-interval` | `--app-health-probe-interval` | | `dapr.io/app-health-probe-interval` | Number of *seconds* between each health probe. Requires app health checks to be enabled. Default is `5` | diff --git a/daprdocs/content/en/reference/cli/dapr-annotate.md b/daprdocs/content/en/reference/cli/dapr-annotate.md index 3af9c6610cf..e0e1ba8b90e 100644 --- a/daprdocs/content/en/reference/cli/dapr-annotate.md +++ b/daprdocs/content/en/reference/cli/dapr-annotate.md @@ -28,8 +28,7 @@ dapr annotate [flags] CONFIG-FILE | `--app-id, -a` | | | The app id to annotate | | `--app-max-concurrency` | | `-1` | The maximum number of concurrent requests to allow | | `--app-port, -p` | | `-1` | The port to expose the app on | -| `--app-protocol` | | | The protocol to use for the app | -| `--app-ssl` | | `false` | Enable SSL for the app | +| `--app-protocol` | | | The protocol to use for the app: `http` (default), `grpc`, `https`, `grpcs`, `h2c` | | `--app-token-secret` | | | The secret to use for the app token | | `--config, -c` | | | The config file to annotate | | `--cpu-limit` | | | The CPU limit to set for the sidecar. See valid values [here](https://kubernetes.io/docs/tasks/administer-cluster/manage-resources/quota-memory-cpu-namespace/). | diff --git a/daprdocs/content/en/reference/cli/dapr-run.md b/daprdocs/content/en/reference/cli/dapr-run.md index 3045a99cf65..ee16027b219 100644 --- a/daprdocs/content/en/reference/cli/dapr-run.md +++ b/daprdocs/content/en/reference/cli/dapr-run.md @@ -26,8 +26,7 @@ dapr run [flags] [command] | `--app-id`, `-a` | `APP_ID` | | The id for your application, used for service discovery. Cannot contain dots. | | `--app-max-concurrency` | | `unlimited` | The concurrency level of the application; default is unlimited | | `--app-port`, `-p` | `APP_PORT` | | The port your application is listening on | -| `--app-protocol`, `-P` | | `http` | The protocol Dapr uses to talk to the application. Valid values are: `http` or `grpc` | -| `--app-ssl` | | `false` | Enable https when Dapr invokes the application | +| `--app-protocol`, `-P` | | `http` | The protocol Dapr uses to talk to the application. Valid values are: `http`, `grpc`, `https` (HTTP with TLS), `grpcs` (gRPC with TLS), `h2c` (HTTP/2 Cleartext) | | `--resources-path`, `-d` | | Linux/Mac: `$HOME/.dapr/components`
Windows: `%USERPROFILE%\.dapr\components` | The path for components directory | | `--runtime-path` | | | Dapr runtime install path | | `--config`, `-c` | | Linux/Mac: `$HOME/.dapr/config.yaml`
Windows: `%USERPROFILE%\.dapr\config.yaml` | Dapr configuration file | diff --git a/daprdocs/content/en/reference/components-reference/supported-bindings/http.md b/daprdocs/content/en/reference/components-reference/supported-bindings/http.md index b95099430f8..30de167050c 100644 --- a/daprdocs/content/en/reference/components-reference/supported-bindings/http.md +++ b/daprdocs/content/en/reference/components-reference/supported-bindings/http.md @@ -7,6 +7,10 @@ aliases: - "/operations/components/setup-bindings/supported-bindings/http/" --- +## Alternative + +The [service invocation API]({{< ref service_invocation_api.md >}}) allows for the invocation of non-Dapr HTTP endpoints and is the recommended approach. Read ["How-To: Invoke Non-Dapr Endpoints using HTTP"]({{< ref howto-invoke-non-dapr-endpoints.md >}}) for more information. + ## Setup Dapr component ```yaml diff --git a/daprdocs/content/en/reference/components-reference/supported-bindings/kitex.md b/daprdocs/content/en/reference/components-reference/supported-bindings/kitex.md new file mode 100644 index 00000000000..b262f8ebe54 --- /dev/null +++ b/daprdocs/content/en/reference/components-reference/supported-bindings/kitex.md @@ -0,0 +1,83 @@ +--- +type: docs +title: "Kitex" +linkTitle: "Kitex" +description: "Detailed documentation on the Kitex binding component" +aliases: +- "/operations/components/setup-bindings/supported-bindings/kitex/" +--- + +## Overview + +The binding for Kitex mainly utilizes the generic-call feature in Kitex. Learn more from the official documentation around [Kitex generic-call](https://www.cloudwego.io/docs/kitex/tutorials/advanced-feature/generic-call/). +Currently, Kitex only supports Thrift generic calls. The implementation integrated into [components-contrib](https://github.com/dapr/components-contrib/tree/master/bindings/kitex) adopts binary generic calls. + + +## Component format + +To setup an Kitex binding, create a component of type `bindings.kitex`. See the [How-to: Use output bindings to interface with external resources]({{< ref "howto-bindings.md#1-create-a-binding" >}}) guide on creating and applying a binding configuration. + +```yaml +apiVersion: dapr.io/v1alpha1 +kind: Component +metadata: + name: bindings.kitex +spec: + type: bindings.kitex + version: v1 + metadata: +``` + +## Spec metadata fields + +The `InvokeRequest.Metadata` for `bindings.kitex` requires the client to fill in four required items when making a call: +- `hostPorts` +- `destService` +- `methodName` +- `version` + +| Field | Required | Binding support | Details | Example | +|-------------|:--------:|--------|---------------------------------------------------------------------------------------------------------|--------------------| +| hostPorts | Y | Output | IP address and port information of the Kitex server (Thrift) | `"127.0.0.1:8888"` | +| destService | Y | Output | Service name of the Kitex server (Thrift) | `"echo"` | +| methodName | Y | Output | Method name under a specific service name of the Kitex server (Thrift) | `"echo"` | +| version | Y | Output | Kitex version | `"0.5.0"` | + + +## Binding support + +This component supports **output binding** with the following operations: + +- `get` + +## Example + +When using Kitex binding: +- The client needs to pass in the correct Thrift-encoded binary +- The server needs to be a Thrift Server. + +The [kitex_output_test](https://github.com/dapr/components-contrib/blob/master/bindings/kitex/kitex_output_test.go) can be used as a reference. +For example, the variable `reqData` needs to be _encoded_ by the Thrift protocol before sending, and the returned data needs to be decoded by the Thrift protocol. + +**Request** + +```json +{ + "operation": "get", + "metadata": { + "hostPorts": "127.0.0.1:8888", + "destService": "echo", + "methodName": "echo", + "version":"0.5.0" + }, + "data": reqdata +} +``` + +## Related links + +- [Basic schema for a Dapr component]({{< ref component-schema >}}) +- [Bindings building block]({{< ref bindings >}}) +- [How-To: Trigger application with input binding]({{< ref howto-triggers.md >}}) +- [How-To: Use bindings to interface with external resources]({{< ref howto-bindings.md >}}) +- [Bindings API reference]({{< ref bindings_api.md >}}) diff --git a/daprdocs/content/en/reference/components-reference/supported-bindings/postgres.md b/daprdocs/content/en/reference/components-reference/supported-bindings/postgresql.md similarity index 88% rename from daprdocs/content/en/reference/components-reference/supported-bindings/postgres.md rename to daprdocs/content/en/reference/components-reference/supported-bindings/postgresql.md index 2a423dfa706..fdefee5ccbf 100644 --- a/daprdocs/content/en/reference/components-reference/supported-bindings/postgres.md +++ b/daprdocs/content/en/reference/components-reference/supported-bindings/postgresql.md @@ -4,12 +4,13 @@ title: "PostgreSQL binding spec" linkTitle: "PostgreSQL" description: "Detailed documentation on the PostgreSQL binding component" aliases: + - "/operations/components/setup-bindings/supported-bindings/postgresql/" - "/operations/components/setup-bindings/supported-bindings/postgres/" --- ## Component format -To setup PostgreSQL binding create a component of type `bindings.postgres`. See [this guide]({{< ref "howto-bindings.md#1-create-a-binding" >}}) on how to create and apply a binding configuration. +To setup PostgreSQL binding create a component of type `bindings.postgresql`. See [this guide]({{< ref "howto-bindings.md#1-create-a-binding" >}}) on how to create and apply a binding configuration. ```yaml @@ -18,7 +19,7 @@ kind: Component metadata: name: spec: - type: bindings.postgres + type: bindings.postgresql version: v1 metadata: - name: url # Required @@ -33,7 +34,7 @@ The above example uses secrets as plain strings. It is recommended to use a secr | Field | Required | Binding support | Details | Example | |--------------------|:--------:|------------|-----|---------| -| url | Y | Output | Postgres connection string See [here](#url-format) for more details | `"user=dapr password=secret host=dapr.example.com port=5432 dbname=dapr sslmode=verify-ca"` | +| url | Y | Output | PostgreSQL connection string See [here](#url-format) for more details | `"user=dapr password=secret host=dapr.example.com port=5432 dbname=dapr sslmode=verify-ca"` | ### URL format @@ -144,8 +145,7 @@ Finally, the `close` operation can be used to explicitly close the DB connection } ``` - -> Note, the PostgreSql binding itself doesn't prevent SQL injection, like with any database application, validate the input before executing query. +> Note, the PostgreSQL binding itself doesn't prevent SQL injection, like with any database application, validate the input before executing query. ## Related links diff --git a/daprdocs/content/en/reference/components-reference/supported-bindings/rabbitmq.md b/daprdocs/content/en/reference/components-reference/supported-bindings/rabbitmq.md index 9a8924c39e9..89d150f43a4 100644 --- a/daprdocs/content/en/reference/components-reference/supported-bindings/rabbitmq.md +++ b/daprdocs/content/en/reference/components-reference/supported-bindings/rabbitmq.md @@ -41,6 +41,14 @@ spec: value: "text/plain" - name: reconnectWaitInSeconds value: 5 + - name: externalSasl + value: false + - name: caCert + value: null + - name: clientCert + value: null + - name: clientKey + value: null ``` {{% alert title="Warning" color="warning" %}} @@ -52,7 +60,7 @@ The above example uses secrets as plain strings. It is recommended to use a secr | Field | Required | Binding support | Details | Example | |--------------------|:--------:|------------|-----|---------| | queueName | Y | Input/Output | The RabbitMQ queue name | `"myqueue"` | -| host | Y | Input/Output | The RabbitMQ host address | `"amqp://[username][:password]@host.domain[:port]"` | +| host | Y | Input/Output | The RabbitMQ host address | `"amqp://[username][:password]@host.domain[:port]"` or with TLS: `"amqps://[username][:password]@host.domain[:port]"` | | durable | N | Output | Tells RabbitMQ to persist message in storage. Defaults to `"false"` | `"true"`, `"false"` | | deleteWhenUnused | N | Input/Output | Enables or disables auto-delete. Defaults to `"false"` | `"true"`, `"false"` | | ttlInSeconds | N | Output | Set the [default message time to live at RabbitMQ queue level](https://www.rabbitmq.com/ttl.html). If this parameter is omitted, messages won't expire, continuing to exist on the queue until processed. See [also](#specifying-a-ttl-per-message) | `60` | @@ -61,6 +69,10 @@ The above example uses secrets as plain strings. It is recommended to use a secr | maxPriority| N | Input/Output | Parameter to set the [priority queue](https://www.rabbitmq.com/priority.html). If this parameter is omitted, queue will be created as a general queue instead of a priority queue. Value between 1 and 255. See [also](#specifying-a-priority-per-message) | `"1"`, `"10"` | | contentType | N | Input/Output | The content type of the message. Defaults to "text/plain". | `"text/plain"`, `"application/cloudevent+json"` and so on | | reconnectWaitInSeconds | N | Input/Output | Represents the duration in seconds that the client should wait before attempting to reconnect to the server after a disconnection occurs. Defaults to `"5"`. | `"5"`, `"10"` | +| externalSasl | N | Input/Output | With TLS, should the username be taken from an additional field (e.g. CN.) See [RabbitMQ Authentication Mechanisms](https://www.rabbitmq.com/access-control.html#mechanisms). Defaults to `"false"`. | `"true"`, `"false"` | +| caCert | N | Input/Output | The CA certificate to use for TLS connection. Defaults to `null`. | `"-----BEGIN CERTIFICATE-----\nMI..."` | +| clientCert | N | Input/Output | The client certificate to use for TLS connection. Defaults to `null`. | `"-----BEGIN CERTIFICATE-----\nMI..."` | +| clientKey | N | Input/Output | The client key to use for TLS connection. Defaults to `null`. | `"-----BEGIN PRIVATE KEY-----\nMI..."` | ## Binding support This component supports both **input and output** binding interfaces. diff --git a/daprdocs/content/en/reference/components-reference/supported-bindings/wasm.md b/daprdocs/content/en/reference/components-reference/supported-bindings/wasm.md new file mode 100644 index 00000000000..0ca9a69d450 --- /dev/null +++ b/daprdocs/content/en/reference/components-reference/supported-bindings/wasm.md @@ -0,0 +1,103 @@ +--- +type: docs +title: "Wasm" +linkTitle: "Wasm" +description: "Detailed documentation on the WebAssembly binding component" +aliases: +- "/operations/components/setup-bindings/supported-bindings/wasm/" +--- + +## Overview + +With WebAssembly, you can safely run code compiled in other languages. Runtimes +execute WebAssembly Modules (Wasm), which are most often binaries with a `.wasm` +extension. + +The Wasm Binding allows you to invoke a program compiled to Wasm by passing +commandline args or environment variables to it, similar to how you would with +a normal subprocess. For example, you can satisfy an invocation using Python, +even though Dapr is written in Go and is running on a platform that doesn't have +Python installed! + +The Wasm binary must be a program compiled with the WebAssembly System +Interface (WASI). The binary can be a program you've written such as in Go, or +an interpreter you use to run inlined scripts, such as Python. + +Minimally, you must specify a Wasm binary compiled with the canonical WASI +version `wasi_snapshot_preview1` (a.k.a. `wasip1`), often abbreviated to `wasi`. + +> **Note:** If compiling in Go 1.21+, this is `GOOS=wasip1 GOARCH=wasm`. In TinyGo, Rust, and Zig, this is the target `wasm32-wasi`. + +You can also re-use an existing binary. For example, [Wasm Language Runtimes](https://github.com/vmware-labs/webassembly-language-runtimes) +distributes interpreters (including PHP, Python, and Ruby) already compiled to +WASI. + +Wasm binaries are loaded from a URL. For example, the URL `file://rewrite.wasm` +loads `rewrite.wasm` from the current directory of the process. On Kubernetes, +see [How to: Mount Pod volumes to the Dapr sidecar]({{< ref kubernetes-volume-mounts.md >}}) +to configure a filesystem mount that can contain Wasm binaries. + +Dapr uses [wazero](https://wazero.io) to run these binaries, because it has no +dependencies. This allows use of WebAssembly with no installation process +except Dapr itself. + +## Component format + +To configure a Wasm binding, create a component of type +`bindings.wasm`. See [this guide]({{< ref "howto-bindings.md#1-create-a-binding" >}}) +on how to create and apply a binding configuration. + +```yaml +apiVersion: dapr.io/v1alpha1 +kind: Component +metadata: + name: wasm +spec: + type: bindings.wasm + version: v1 + metadata: + - name: url + value: "file://uppercase.wasm" +``` + +## Spec metadata fields + +| Field | Details | Required | Example | +|-------|----------------------------------------------------------------|----------|----------------| +| url | The URL of the resource including the Wasm binary to instantiate. The supported schemes include `file://`. The path of a `file://` URL is relative to the Dapr process unless it begins with `/`. | true | `file://hello.wasm` | + + +## Binding support + +This component supports **output binding** with the following operations: + +- `execute` + +## Example request + +The `data` field, if present will be the program's STDIN. You can optionally +pass metadata properties with each request: + +- `args` any CLI arguments, comma-separated. This excludes the program name. + +For example, if the binding `url` was a Ruby interpreter, such as from +[webassembly-language-runtimes](https://github.com/vmware-labs/webassembly-language-runtimes/releases/tag/ruby%2F3.2.0%2B20230215-1349da9), +the following request would respond back with "Hello, salaboy": + +```json +{ + "operation": "execute", + "metadata": { + "args": "-ne,'print \"Hello, \"; print'" + }, + "data": "salaboy" +} +``` + +## Related links + +- [Basic schema for a Dapr component]({{< ref component-schema >}}) +- [Bindings building block]({{< ref bindings >}}) +- [How-To: Trigger application with input binding]({{< ref howto-triggers.md >}}) +- [How-To: Use bindings to interface with external resources]({{< ref howto-bindings.md >}}) +- [Bindings API reference]({{< ref bindings_api.md >}}) diff --git a/daprdocs/content/en/reference/components-reference/supported-configuration-stores/postgres-configuration-store.md b/daprdocs/content/en/reference/components-reference/supported-configuration-stores/postgres-configuration-store.md index 35c74f03040..4a699174855 100644 --- a/daprdocs/content/en/reference/components-reference/supported-configuration-stores/postgres-configuration-store.md +++ b/daprdocs/content/en/reference/components-reference/supported-configuration-stores/postgres-configuration-store.md @@ -1,15 +1,16 @@ --- type: docs -title: "Postgres" -linkTitle: "Postgres" -description: Detailed information on the Postgres configuration store component +title: "PostgreSQL" +linkTitle: "PostgreSQL" +description: Detailed information on the PostgreSQL configuration store component aliases: + - "/operations/components/setup-configuration-store/supported-configuration-stores/setup-postgresql/" - "/operations/components/setup-configuration-store/supported-configuration-stores/setup-postgres/" --- ## Component format -To set up an Postgres configuration store, create a component of type `configuration.postgres` +To set up an PostgreSQL configuration store, create a component of type `configuration.postgresql` ```yaml apiVersion: dapr.io/v1alpha1 @@ -17,7 +18,7 @@ kind: Component metadata: name: spec: - type: configuration.postgres + type: configuration.postgresql version: v1 metadata: - name: connectionString @@ -40,10 +41,10 @@ The above example uses secrets as plain strings. It is recommended to use a secr | connectionString | Y | The connection string for PostgreSQL. Default pool_max_conns = 5 | `"host=localhost user=postgres password=example port=5432 connect_timeout=10 database=dapr_test pool_max_conns=10"` | table | Y | table name for configuration information. | `configTable` -## Set up Postgres as Configuration Store +## Set up PostgreSQL as Configuration Store -1. Start Postgres Database -1. Connect to the Postgres database and setup a configuration table with following schema - +1. Start PostgreSQL Database +1. Connect to the PostgreSQL database and setup a configuration table with following schema - | Field | Datatype | Nullable |Details | |--------------------|:--------:|---------|---------| @@ -101,13 +102,13 @@ AFTER INSERT OR UPDATE OR DELETE ON configTable 7. In the subscribe request add an additional metadata field with key as `pgNotifyChannel` and value should be set to same `channel name` mentioned in `pg_notify`. From the above example, it should be set to `config` {{% alert title="Note" color="primary" %}} -When calling `subscribe` API, `metadata.pgNotifyChannel` should be used to specify the name of the channel to listen for notifications from Postgres configuration store. +When calling `subscribe` API, `metadata.pgNotifyChannel` should be used to specify the name of the channel to listen for notifications from PostgreSQL configuration store. Any number of keys can be added to a subscription request. Each subscription uses an exclusive database connection. It is strongly recommended to subscribe to multiple keys within a single subscription. This helps optimize the number of connections to the database. Example of subscribe HTTP API - ```ps -curl --location --request GET 'http://:/configuration/postgres/subscribe?key=&key=&metadata.pgNotifyChannel=' +curl --location --request GET 'http://:/configuration/mypostgresql/subscribe?key=&key=&metadata.pgNotifyChannel=' ``` {{% /alert %}} diff --git a/daprdocs/content/en/reference/components-reference/supported-middleware/middleware-bearer.md b/daprdocs/content/en/reference/components-reference/supported-middleware/middleware-bearer.md index bf637194e89..e2916a71384 100644 --- a/daprdocs/content/en/reference/components-reference/supported-middleware/middleware-bearer.md +++ b/daprdocs/content/en/reference/components-reference/supported-middleware/middleware-bearer.md @@ -8,7 +8,7 @@ aliases: - /developing-applications/middleware/supported-middleware/middleware-bearer/ --- -The bearer [HTTP middleware]({{< ref middleware.md >}}) verifies a [Bearer Token](https://tools.ietf.org/html/rfc6750) using [OpenID Connect](https://openid.net/connect/) on a Web API without modifying the application. This design separates authentication/authorization concerns from the application, so that application operators can adopt and configure authentication/authorization providers without impacting the application code. +The bearer [HTTP middleware]({{< ref middleware.md >}}) verifies a [Bearer Token](https://tools.ietf.org/html/rfc6750) using [OpenID Connect](https://openid.net/connect/) on a Web API, without modifying the application. This design separates authentication/authorization concerns from the application, so that application operators can adopt and configure authentication/authorization providers without impacting the application code. ## Component format @@ -21,17 +21,22 @@ spec: type: middleware.http.bearer version: v1 metadata: - - name: clientId - value: "" - - name: issuerURL - value: "https://accounts.google.com" + - name: audience + value: "" + - name: issuer + value: "" + + # Optional values + - name: jwksURL + value: "https://accounts.google.com/.well-known/openid-configuration" ``` ## Spec metadata fields -| Field | Details | Example | -|-------|---------|---------| -| clientId | The client ID of your application that is created as part of a credential hosted by a OpenID Connect platform -| issuerURL | URL identifier for the service. | `"https://accounts.google.com"`, `"https://login.salesforce.com"` +| Field | Required | Details | Example | +|-------|:--------:|---------|---------| +| `audience` | Y | The audience expected in the tokens. Usually, this corresponds to the client ID of your application that is created as part of a credential hosted by a OpenID Connect platform. | +| `issuer` | Y | The issuer authority, which is the value expected in the issuer claim in the tokens. | `"https://accounts.google.com"`, `"https://login.salesforce.com"` +| `jwksURL` | N | Address of the JWKS (JWK Set containing the public keys for verifying tokens). If empty, will try to fetch the URL set in the OpenID Configuration document `/.well-known/openid-configuration`. | `"https://accounts.google.com/.well-known/openid-configuration"` ## Dapr configuration diff --git a/daprdocs/content/en/reference/components-reference/supported-middleware/middleware-rate-limit.md b/daprdocs/content/en/reference/components-reference/supported-middleware/middleware-rate-limit.md index c1d0c6ea048..59dc11dfb77 100644 --- a/daprdocs/content/en/reference/components-reference/supported-middleware/middleware-rate-limit.md +++ b/daprdocs/content/en/reference/components-reference/supported-middleware/middleware-rate-limit.md @@ -38,7 +38,7 @@ Once the limit is reached, the requests will fail with HTTP Status code *429: To The rate limit is enforced independently in each Dapr sidecar, and not cluster-wide. {{% /alert %}} -Alternatively, the [max concurrency setting]({{< ref control-concurrency.md >}}) can be used to rate limit applications and applies to all traffic, regardless of remote IP, protocol, or path. +Alternatively, the [max concurrency setting]({{< ref control-concurrency.md >}}) can be used to rate-limit applications and applies to all traffic, regardless of remote IP, protocol, or path. ## Dapr configuration diff --git a/daprdocs/content/en/reference/components-reference/supported-middleware/middleware-wasm.md b/daprdocs/content/en/reference/components-reference/supported-middleware/middleware-wasm.md index 8d19d0b196d..d36ad948bb5 100644 --- a/daprdocs/content/en/reference/components-reference/supported-middleware/middleware-wasm.md +++ b/daprdocs/content/en/reference/components-reference/supported-middleware/middleware-wasm.md @@ -1,8 +1,8 @@ --- type: docs -title: "WASM" -linkTitle: "WASM" -description: "Use WASM middleware in your HTTP pipeline" +title: "Wasm" +linkTitle: "Wasm" +description: "Use Wasm middleware in your HTTP pipeline" aliases: - /developing-applications/middleware/supported-middleware/middleware-wasm/ --- @@ -17,9 +17,10 @@ binary. In other words, you can extend Dapr using external files that are not pre-compiled into the `daprd` binary. Dapr embeds [wazero](https://wazero.io) to accomplish this without CGO. -Wasm modules are loaded from a filesystem path. On Kubernetes, see [mounting -volumes to the Dapr sidecar]({{< ref kubernetes-volume-mounts.md >}}) to configure -a filesystem mount that can contain Wasm modules. +Wasm binaries are loaded from a URL. For example, the URL `file://rewrite.wasm` +loads `rewrite.wasm` from the current directory of the process. On Kubernetes, +see [How to: Mount Pod volumes to the Dapr sidecar]({{< ref kubernetes-volume-mounts.md >}}) +to configure a filesystem mount that can contain Wasm modules. ## Component format @@ -32,8 +33,8 @@ spec: type: middleware.http.wasm version: v1 metadata: - - name: path - value: "./router.wasm" + - name: url + value: "file://router.wasm" ``` ## Spec metadata fields @@ -41,9 +42,9 @@ spec: Minimally, a user must specify a Wasm binary implements the [http-handler](https://http-wasm.io/http-handler/). How to compile this is described later. -| Field | Details | Required | Example | -|----------|----------------------------------------------------------------|----------|----------------| -| path | A relative or absolute path to the Wasm binary to instantiate. | true | "./hello.wasm" | +| Field | Details | Required | Example | +|-------|----------------------------------------------------------------|----------|----------------| +| url | The URL of the resource including the Wasm binary to instantiate. The supported schemes include `file://`. The path of a `file://` URL is relative to the Dapr process unless it begins with `/`. | true | `file://hello.wasm` | ## Dapr configuration @@ -109,7 +110,7 @@ func handleRequest(req api.Request, resp api.Response) (next bool, reqCtx uint32 ``` If using TinyGo, compile as shown below and set the spec metadata field named -"path" to the location of the output (ex "router.wasm"): +"url" to the location of the output (for example, `file://router.wasm`): ```bash tinygo build -o router.wasm -scheduler=none --no-debug -target=wasi router.go` @@ -145,7 +146,7 @@ func rewrite(requestURI []byte) ([]byte, error) { ``` If using TinyGo, compile as shown below and set the spec metadata field named -"path" to the location of the output (ex "example.wasm"): +"url" to the location of the output (for example, `file://example.wasm`): ```bash tinygo build -o example.wasm -scheduler=none --no-debug -target=wasi example.go diff --git a/daprdocs/content/en/reference/components-reference/supported-pubsub/setup-rocketmq.md b/daprdocs/content/en/reference/components-reference/supported-pubsub/setup-rocketmq.md index c40d75e3dbc..1b6a5a87e35 100644 --- a/daprdocs/content/en/reference/components-reference/supported-pubsub/setup-rocketmq.md +++ b/daprdocs/content/en/reference/components-reference/supported-pubsub/setup-rocketmq.md @@ -58,12 +58,20 @@ The above example uses secrets as plain strings. It is recommended to use a secr | producerQueueSelector (queueSelector) | N | Producer Queue selector. There are five implementations of queue selector: `hash`, `random`, `manual`, `roundRobin`, `dapr`. | `dapr` | `hash` | | consumerModel | N | Message model that defines how messages are delivered to each consumer client. RocketMQ supports two message models: `clustering` and `broadcasting`. | `clustering` | `broadcasting` , `clustering` | | fromWhere (consumeFromWhere) | N | Consuming point on consumer booting. There are three consuming points: `CONSUME_FROM_LAST_OFFSET`, `CONSUME_FROM_FIRST_OFFSET`, `CONSUME_FROM_TIMESTAMP` | `CONSUME_FROM_LAST_OFFSET` | `CONSUME_FROM_LAST_OFFSET` | +| consumeTimestamp | N | Backtracks consumption time with second precision. Time format is `yyyymmddhhmmss`. For example, `20131223171201` implies the time of 17:12:01 and date of December 23, 2013 | ` time.Now().Add(time.Minute * (-30)).Format("20060102150405")` | `20131223171201` | | consumeOrderly | N | Determines if it's an ordered message using FIFO order. | `false` | `false` | | consumeMessageBatchMaxSize | N | Batch consumption size out of range `[1, 1024]` | `512` | `10` | +| consumeConcurrentlyMaxSpan | N | Concurrently max span offset. This has no effect on sequential consumption. Range: `[1, 65535]` | `1000` | `1000` | | maxReconsumeTimes | N | Max re-consume times. `-1` means 16 times. If messages are re-consumed more than {@link maxReconsumeTimes} before success, they'll be directed to a deletion queue. | Orderly message is `MaxInt32`; Concurrently message is `16` | `16` | | autoCommit | N | Enable auto commit | `true` | `false` | +| consumeTimeout | N | Maximum amount of time a message may block the consuming thread. Time unit: Minute | `15` | `15` | +| consumerPullTimeout | N | The socket timeout in milliseconds | | | | pullInterval | N | Message pull interval | `100` | `100` | | pullBatchSize | N | The number of messages pulled from the broker at a time. If `pullBatchSize` is `null`, use `ConsumerBatchSize`. `pullBatchSize` out of range `[1, 1024]` | `32` | `10` | +| pullThresholdForQueue | N | Flow control threshold on queue level. Each message queue will cache a maximum of 1000 messages by default. Consider the `PullBatchSize` - the instantaneous value may exceed the limit. Range: `[1, 65535]` | `1024` | `1000` | +| pullThresholdForTopic | N | Flow control threshold on topic level. The value of `pullThresholdForQueue` will be overwritten and calculated based on `pullThresholdForTopic` if it isn't unlimited. For example, if the value of `pullThresholdForTopic` is 1000 and 10 message queues are assigned to this consumer, then `pullThresholdForQueue` will be set to 100. Range: `[1, 6553500]` | `-1(Unlimited)` | `10` | +| pullThresholdSizeForQueue | N | Limit the cached message size on queue level. Consider the `pullBatchSize` - the instantaneous value may exceed the limit. The size of a message is only measured by message body, so it's not accurate. Range: `[1, 1024]` | `100` | `100` | +| pullThresholdSizeForTopic | N | Limit the cached message size on topic level. The value of `pullThresholdSizeForQueue` will be overwritten and calculated based on `pullThresholdSizeForTopic` if it isn't unlimited. For example, if the value of `pullThresholdSizeForTopic` is 1000 MiB and 10 message queues are assigned to this consumer, then `pullThresholdSizeForQueue` will be set to 100 MiB. Range: `[1, 102400]` | `-1` | `100` | | content-type | N | Message content type. | `"text/plain"` | `"application/cloudevents+json; charset=utf-8"`, `"application/octet-stream"` | | logLevel | N | Log level | `warn` | `info` | | sendTimeOut | N | Send message timeout to connect RocketMQ's broker, measured in nanoseconds. **Deprecated**. | 3 seconds | `10000000000` | diff --git a/daprdocs/content/en/reference/components-reference/supported-secret-stores/envvar-secret-store.md b/daprdocs/content/en/reference/components-reference/supported-secret-stores/envvar-secret-store.md index 0ee8be7290c..b4e67318946 100644 --- a/daprdocs/content/en/reference/components-reference/supported-secret-stores/envvar-secret-store.md +++ b/daprdocs/content/en/reference/components-reference/supported-secret-stores/envvar-secret-store.md @@ -26,7 +26,23 @@ spec: type: secretstores.local.env version: v1 metadata: + # - name: prefix + # value: "MYAPP_" ``` + +## Spec metadata fields + +| Field | Required | Details | Example | +|-------|:--------:|---------|---------| +| `prefix` | N | If set, limits operations to environment variables with the given prefix. The prefix is removed from the returned secrets' names.
The matching is case-insensitive on Windows and case-sensitive on all other operating systems. | `"MYAPP_"` + +## Notes + +For security reasons, this component cannot be used to access these environment variables: + +- `APP_API_TOKEN` +- Any variable whose name begines with the `DAPR_` prefix + ## Related Links - [Secrets building block]({{< ref secrets >}}) - [How-To: Retrieve a secret]({{< ref "howto-secrets.md" >}}) diff --git a/daprdocs/content/en/reference/components-reference/supported-state-stores/setup-etcd.md b/daprdocs/content/en/reference/components-reference/supported-state-stores/setup-etcd.md index 10e85af68ac..35834d1c4ce 100644 --- a/daprdocs/content/en/reference/components-reference/supported-state-stores/setup-etcd.md +++ b/daprdocs/content/en/reference/components-reference/supported-state-stores/setup-etcd.md @@ -9,8 +9,7 @@ aliases: ## Component format -To setup Etcd state store create a component of type `state.etcd`. See [this guide]({{< ref "howto-get-save-state.md#step-1-setup-a-state-store" >}}) on how to create and apply a state store configuration. - +To setup an Etcd state store create a component of type `state.etcd`. See [this guide]({{< ref "howto-get-save-state.md#step-1-setup-a-state-store" >}}) on how to create and apply a state store configuration. ```yaml apiVersion: dapr.io/v1alpha1 @@ -43,12 +42,12 @@ The above example uses secrets as plain strings. It is recommended to use a secr | Field | Required | Details | Example | |--------------------|:--------:|---------|---------| -| endpoints | Y | Connection string to Etcd server | `"192.168.0.1:2379,192.168.0.2:2379,192.168.0.3:2379"` -| keyPrefixPath | N | Key prefix path in Etcd. Default is `""` | `"dapr"` -| tlsEnable | N | Whether to enable tls | `"false"` -| ca | N | Contents of Etcd server CA file. Can be `secretKeyRef` to use a [secret reference]({{< ref component-secrets.md >}}).| `"-----BEGIN CERTIFICATE-----\nMIIC9TCCA..."` -| cert | N | Contents of Etcd server certificate file. Can be `secretKeyRef` to use a [secret reference]({{< ref component-secrets.md >}}).| `"-----BEGIN CERTIFICATE-----\nMIIDUTCC..."` -| key | N | Contents of Etcd server key file. Can be `secretKeyRef` to use a [secret reference]({{< ref component-secrets.md >}}).| `"-----BEGIN RSA PRIVATE KEY-----\nMIIEpAIB..."` +| `endpoints` | Y | Connection string to the Etcd cluster | `"192.168.0.1:2379,192.168.0.2:2379,192.168.0.3:2379"` +| `keyPrefixPath` | N | Key prefix path in Etcd. Default is no prefix. | `"dapr"` +| `tlsEnable` | N | Whether to enable TLS for connecting to Etcd. | `"false"` +| `ca` | N | CA certificate for connecting to Etcd, PEM-encoded. Can be `secretKeyRef` to use a [secret reference]({{< ref component-secrets.md >}}).| `"-----BEGIN CERTIFICATE-----\nMIIC9TCCA..."` +| `cert` | N | TLS certificate for connecting to Etcd, PEM-encoded. Can be `secretKeyRef` to use a [secret reference]({{< ref component-secrets.md >}}).| `"-----BEGIN CERTIFICATE-----\nMIIDUTCC..."` +| `key` | N | TLS key for connecting to Etcd, PEM-encoded. Can be `secretKeyRef` to use a [secret reference]({{< ref component-secrets.md >}}).| `"-----BEGIN PRIVATE KEY-----\nMIIEpAIB..."` ## Setup Etcd @@ -58,7 +57,7 @@ The above example uses secrets as plain strings. It is recommended to use a secr You can run Etcd database locally using Docker Compose. Create a new file called `docker-compose.yml` and add the following contents as an example: -``` +```yaml version: '2' services: etcd: @@ -67,15 +66,16 @@ services: - "2379:2379" command: etcd --listen-client-urls http://0.0.0.0:2379 --advertise-client-urls http://0.0.0.0:2379``` ``` + Save the `docker-compose.yml` file and run the following command to start the Etcd server: -``` +```sh docker-compose up -d ``` This starts the Etcd server in the background and expose the default Etcd port of `2379`. You can then interact with the server using the `etcdctl` command-line client on `localhost:12379`. For example: -``` +```sh etcdctl --endpoints=localhost:2379 put mykey myvalue ``` @@ -92,6 +92,7 @@ Follow the [Bitnami instructions](https://github.com/bitnami/charts/tree/main/bi {{< /tabs >}} ## Related links + - [Basic schema for a Dapr component]({{< ref component-schema >}}) - Read [this guide]({{< ref "howto-get-save-state.md#step-2-save-and-retrieve-a-single-state" >}}) for instructions on configuring state store components - [State management building block]({{< ref state-management >}}) diff --git a/daprdocs/content/en/reference/components-reference/supported-state-stores/setup-postgresql.md b/daprdocs/content/en/reference/components-reference/supported-state-stores/setup-postgresql.md index 263381df3ea..d8472e84038 100644 --- a/daprdocs/content/en/reference/components-reference/supported-state-stores/setup-postgresql.md +++ b/daprdocs/content/en/reference/components-reference/supported-state-stores/setup-postgresql.md @@ -11,7 +11,7 @@ This component allows using PostgreSQL (Postgres) as state store for Dapr. ## Create a Dapr component -Create a file called `postgres.yaml`, paste the following and replace the `` value with your connection string. The connection string is a standard PostgreSQL connection string. For example, `"host=localhost user=postgres password=example port=5432 connect_timeout=10 database=dapr_test"`. See the PostgreSQL [documentation on database connections](https://www.postgresql.org/docs/current/libpq-connect.html) for information on how to define a connection string. +Create a file called `postgresql.yaml`, paste the following and replace the `` value with your connection string. The connection string is a standard PostgreSQL connection string. For example, `"host=localhost user=postgres password=example port=5432 connect_timeout=10 database=dapr_test"`. See the PostgreSQL [documentation on database connections](https://www.postgresql.org/docs/current/libpq-connect.html) for information on how to define a connection string. If you want to also configure PostgreSQL to store actors, add the `actorStateStore` option as in the example below. diff --git a/daprdocs/content/en/reference/components-reference/supported-state-stores/setup-sqlite.md b/daprdocs/content/en/reference/components-reference/supported-state-stores/setup-sqlite.md index 1415e0d8bad..381e6bc4a78 100644 --- a/daprdocs/content/en/reference/components-reference/supported-state-stores/setup-sqlite.md +++ b/daprdocs/content/en/reference/components-reference/supported-state-stores/setup-sqlite.md @@ -9,7 +9,7 @@ aliases: This component allows using SQLite 3 as state store for Dapr. -> The component is currently compiled with SQLite version 3.40.1. +> The component is currently compiled with SQLite version 3.41.2. ## Create a Dapr component @@ -36,8 +36,11 @@ spec: #- name: tableName # value: "state" # Cleanup interval in seconds, to remove expired rows (optional) - #- name: cleanupIntervalInSeconds - # value: 3600 + #- name: cleanupInterval + # value: "1h" + # Set busy timeout for database operations + #- name: busyTimeout + # value: "2s" # Uncomment this if you wish to use SQLite as a state store for actors (optional) #- name: actorStateStore # value: "true" @@ -50,14 +53,17 @@ spec: | `connectionString` | Y | The connection string for the SQLite database. See below for more details. | `"path/to/data.db"`, `"file::memory:?cache=shared"` | `timeoutInSeconds` | N | Timeout, in seconds, for all database operations. Defaults to `20` | `30` | `tableName` | N | Name of the table where the data is stored. Defaults to `state`. | `"state"` -| `cleanupIntervalInSeconds` | N | Interval, in seconds, to clean up rows with an expired TTL. Default: `3600` (i.e. 1 hour). Setting this to values <=0 disables the periodic cleanup. | `1800`, `-1` +| `metadataTableName` | N | Name of the table used by Dapr to store metadata for the component. Defaults to `metadata`. | `"metadata"` +| `cleanupInterval` | N | Interval, as a [Go duration](https://pkg.go.dev/time#ParseDuration), to clean up rows with an expired TTL. Setting this to values <=0 disables the periodic cleanup. Default: `0` (i.e. disabled) | `"2h"`, `"30m"`, `-1` +| `busyTimeout` | N | Interval, as a [Go duration](https://pkg.go.dev/time#ParseDuration), to wait in case the SQLite database is currently busy serving another request, before returning a "database busy" error. Default: `2s` | `"100ms"`, `"5s"` +| `disableWAL` | N | If set to true, disables Write-Ahead Logging for journaling of the SQLite database. You should set this to `false` if the database is stored on a network file system (e.g. a folder mounted as a SMB or NFS share). This option is ignored for read-only or in-memory databases. | `"100ms"`, `"5s"` | `actorStateStore` | N | Consider this state store for actors. Defaults to `"false"` | `"true"`, `"false"` The **`connectionString`** parameter configures how to open the SQLite database. - Normally, this is the path to a file on disk, relative to the current working directory, or absolute. For example: `"data.db"` (relative to the working directory) or `"/mnt/data/mydata.db"`. - The path is interpreted by the SQLite library, so it's possible to pass additional options to the SQLite driver using "URI options" if the path begins with `file:`. For example: `"file:path/to/data.db?mode=ro"` opens the database at path `path/to/data.db` in read-only mode. [Refer to the SQLite documentation for all supported URI options](https://www.sqlite.org/uri.html). -- The special case `":memory:"` launches the component backed by an in-memory SQLite database. This database is not persisted on disk, not shared across multiple Dapr instances, and all data is lost when the Dapr sidecar is stopped. When using an in-memory database, you should always set the `?cache=shared` URI option: `"file::memory:?cache=shared"` +- The special case `":memory:"` launches the component backed by an in-memory SQLite database. This database is not persisted on disk, not shared across multiple Dapr instances, and all data is lost when the Dapr sidecar is stopped. When using an in-memory database, Dapr automatically sets the `cache=shared` URI option. ## Advanced @@ -67,10 +73,10 @@ This state store supports [Time-To-Live (TTL)]({{< ref state-store-ttl.md >}}) f Because SQLite doesn't have built-in support for TTLs, this is implemented in Dapr by adding a column in the state table indicating when the data is to be considered "expired". Records that are "expired" are not returned to the caller, even if they're still physically stored in the database. A background "garbage collector" periodically scans the state table for expired rows and deletes them. -The `cleanupIntervalInSeconds` metadata property sets the expired records deletion interval, which defaults to 3600 seconds (that is, 1 hour). +The `cleanupInterval` metadata property sets the expired records deletion interval, which is disabled by default. -- Longer intervals require less frequent scans for expired rows, but can require storing expired records for longer, potentially requiring more storage space. If you plan to store many records in your state table, with short TTLs, consider setting `cleanupIntervalInSeconds` to a smaller value, for example `300` (300 seconds, or 5 minutes). -- If you do not plan to use TTLs with Dapr and the SQLite state store, you should consider setting `cleanupIntervalInSeconds` to a value <= 0 (e.g. `0` or `-1`) to disable the periodic cleanup and reduce the load on the database. +- Longer intervals require less frequent scans for expired rows, but can cause the database to store expired records for longer, potentially requiring more storage space. If you plan to store many records in your state table, with short TTLs, consider setting `cleanupInterval` to a smaller value, for example `5m`. +- If you do not plan to use TTLs with Dapr and the SQLite state store, you should consider setting `cleanupInterval` to a value <= 0 (e.g. `0` or `-1`) to disable the periodic cleanup and reduce the load on the database. This is the default behavior. The `expiration_time` column in the state table, where the expiration date for records is stored, **does not have an index by default**, so each periodic cleanup must perform a full-table scan. If you have a table with a very large number of records, and only some of them use a TTL, you may find it useful to create an index on that column. Assuming that your state table name is `state` (the default), you can use this query: @@ -79,6 +85,18 @@ CREATE INDEX idx_expiration_time ON state (expiration_time); ``` +> Dapr does not automatically [vacuum](https://www.sqlite.org/lang_vacuum.html) SQLite databases. + +### Sharing a SQLite database and using networked filesystems + +Although you can have multiple Dapr instances accessing the same SQLite database (for example, because your application is scaled horizontally or because you have multiple apps accessing the same state store), there are some caveats you should keep in mind. + +SQLite works best when all clients access a database file on the same, locally-mounted disk. Using virtual disks that are mounted from a SAN (Storage Area Network), as is common practice in virtualized or cloud environments, is fine. + +However, storing your SQLite database in a networked filesystem (for example via NFS or SMB, but these examples are not an exhaustive list) should be done with care. The official SQLite documentation has a page dedicated to [recommendations and caveats for running SQLite over a network](https://www.sqlite.org/useovernet.html). + +Given the risk of data corruption that running SQLite over a networked filesystem (such as via NFS or SMB) comes with, we do not recommend doing that with Dapr in production environment. However, if you do want to do that, you should configure your SQLite Dapr component with `disableWAL` set to `true`. + ## Related links - [Basic schema for a Dapr component]({{< ref component-schema >}}) diff --git a/daprdocs/content/en/reference/environment/_index.md b/daprdocs/content/en/reference/environment/_index.md index 3aba469ee2c..0f49b7510a5 100644 --- a/daprdocs/content/en/reference/environment/_index.md +++ b/daprdocs/content/en/reference/environment/_index.md @@ -24,3 +24,6 @@ The following table lists the environment variables used by the Dapr runtime, CL | DAPR_HELM_REPO_URL | Your private Dapr Helm chart url | Specifies a private Dapr Helm chart url, which defaults to the official Helm chart URL: `https://dapr.github.io/helm-charts`| | DAPR_HELM_REPO_USERNAME | A username for a private Helm chart | The username required to access the private Dapr Helm chart. If it can be accessed publicly, this env variable does not need to be set| | DAPR_HELM_REPO_PASSWORD | A password for a private Helm chart |The password required to access the private Dapr helm chart. If it can be accessed publicly, this env variable does not need to be set| +| OTEL_EXPORTER_OTLP_ENDPOINT | OpenTelemetry Tracing | Sets the Open Telemetry (OTEL) server address, turns on tracing. (Example: `http://localhost:4318`) | +| OTEL_EXPORTER_OTLP_INSECURE | OpenTelemetry Tracing | Sets the connection to the endpoint as unencrypted. (`true`, `false`) | +| OTEL_EXPORTER_OTLP_PROTOCOL | OpenTelemetry Tracing | The OTLP protocol to use Transport protocol. (`grpc`, `http/protobuf`, `http/json`) | \ No newline at end of file diff --git a/daprdocs/content/en/reference/resource-specs/_index.md b/daprdocs/content/en/reference/resource-specs/_index.md new file mode 100644 index 00000000000..fa7ba8fc820 --- /dev/null +++ b/daprdocs/content/en/reference/resource-specs/_index.md @@ -0,0 +1,7 @@ +--- +type: docs +title: "Dapr resource specs" +linkTitle: "Resource specs" +description: "Detailed information and specifications on Dapr resources" +weight: 500 +--- diff --git a/daprdocs/content/en/reference/resource-specs/httpendpoints-reference.md b/daprdocs/content/en/reference/resource-specs/httpendpoints-reference.md new file mode 100644 index 00000000000..66d4f56060e --- /dev/null +++ b/daprdocs/content/en/reference/resource-specs/httpendpoints-reference.md @@ -0,0 +1,41 @@ +--- +type: docs +title: "HTTPEndpoint spec" +linkTitle: "HTTPEndpoint spec" +description: "The HTTPEndpoint resource spec" +weight: 300 +aliases: + - "/operations/httpEndpoints/" +--- + +The `HTTPEndpoint` is a Dapr resource that is used to enable the invocation of non-Dapr endpoints from a Dapr application. + +## HTTPEndpoint format + +```yaml +apiVersion: dapr.io/v1alpha1 +kind: HTTPEndpoint +metadata: + name: +spec: + version: v1alpha1 + baseUrl: # Required. Use "http://" or "https://" prefix. + headers: # Optional + - name: + value: + - name: + secretKeyRef: + name: + key: +scopes: # Optional + - +auth: # Optional + secretStore: +``` + +## Spec fields + +| Field | Required | Details | Example | +|--------------------|:--------:|---------|---------| +| baseUrl | Y | Base URL of the non-Dapr endpoint | `"https://api.github.com"`, `"http://api.github.com"` +| headers | N | HTTP request headers for service invocation | `name: "Accept-Language" value: "en-US"`
`name: "Authorization" secretKeyRef.name: "my-secret" secretKeyRef.key: "myGithubToken" ` \ No newline at end of file diff --git a/daprdocs/data/components/bindings/generic.yaml b/daprdocs/data/components/bindings/generic.yaml index 57ec9e19fd1..f6ce33e6ec2 100644 --- a/daprdocs/data/components/bindings/generic.yaml +++ b/daprdocs/data/components/bindings/generic.yaml @@ -70,8 +70,8 @@ features: input: false output: true -- component: PostgreSql - link: postgres +- component: PostgreSQL + link: postgresql state: Stable version: v1 since: "1.9" @@ -142,3 +142,11 @@ features: input: true output: true +- component: Wasm + link: wasm + state: Alpha + version: v1 + since: "1.11" + features: + input: false + output: true diff --git a/daprdocs/data/components/configuration_stores/generic.yaml b/daprdocs/data/components/configuration_stores/generic.yaml index 6377c08cdb2..e51482aa3db 100644 --- a/daprdocs/data/components/configuration_stores/generic.yaml +++ b/daprdocs/data/components/configuration_stores/generic.yaml @@ -3,8 +3,8 @@ state: Stable version: v1 since: "1.11" -- component: Postgres - link: postgres-configuration-store +- component: PostgreSQL + link: postgresql-configuration-store state: Stable version: v1 since: "1.11" diff --git a/daprdocs/data/components/middleware/http.yaml b/daprdocs/data/components/middleware/http.yaml index 5f06f016d69..fdbeaa6a345 100644 --- a/daprdocs/data/components/middleware/http.yaml +++ b/daprdocs/data/components/middleware/http.yaml @@ -2,6 +2,7 @@ link: middleware-rate-limit state: Stable version: v1 + since: "1.11" description: Restricts the maximum number of allowed HTTP requests per second - component: OAuth2 Authorization Grant flow link: /reference/components-reference/supported-middleware/middleware-oauth2 @@ -44,8 +45,8 @@ state: Stable version: v1 description: Converts the body of the request to uppercase letters (demo) -- component: WASM +- component: Wasm link: /reference/components-reference/supported-middleware/middleware-wasm state: Alpha version: v1 - description: Use WASM middleware in your HTTP pipeline + description: Use Wasm middleware in your HTTP pipeline diff --git a/daprdocs/data/components/state_stores/azure.yaml b/daprdocs/data/components/state_stores/azure.yaml index e05757dbed5..356d0861c24 100644 --- a/daprdocs/data/components/state_stores/azure.yaml +++ b/daprdocs/data/components/state_stores/azure.yaml @@ -9,7 +9,7 @@ etag: true ttl: false query: false -- component: Azure CosmosDB +- component: Azure Cosmos DB link: setup-azure-cosmosdb state: Stable version: v1 @@ -29,7 +29,7 @@ crud: true transactions: true etag: true - ttl: false + ttl: true query: false - component: Azure Table Storage link: setup-azure-tablestorage diff --git a/daprdocs/data/components/state_stores/generic.yaml b/daprdocs/data/components/state_stores/generic.yaml index 55d12e0a5cb..dd7cacd13ba 100644 --- a/daprdocs/data/components/state_stores/generic.yaml +++ b/daprdocs/data/components/state_stores/generic.yaml @@ -42,6 +42,17 @@ etag: true ttl: false query: false +- component: etcd + link: setup-etcd + state: Beta + version: v1 + since: "1.11" + features: + crud: true + transactions: true + etag: true + ttl: true + query: false - component: Hashicorp Consul link: setup-consul state: Alpha @@ -108,7 +119,7 @@ etag: true ttl: true query: true -- component: MySQL +- component: MySQL & MariaDB link: setup-mysql state: Stable version: v1 @@ -117,11 +128,11 @@ crud: true transactions: true etag: true - ttl: false + ttl: true query: false - component: Oracle Database link: setup-oracledatabase - state: Alpha + state: Beta version: v1 since: "1.7" features: diff --git a/daprdocs/static/images/service-invocation-overview-non-dapr-endpoint.png b/daprdocs/static/images/service-invocation-overview-non-dapr-endpoint.png new file mode 100644 index 00000000000..28873f84cd4 Binary files /dev/null and b/daprdocs/static/images/service-invocation-overview-non-dapr-endpoint.png differ diff --git a/daprdocs/static/presentations/Dapr-Diagrams.pptx.zip b/daprdocs/static/presentations/Dapr-Diagrams.pptx.zip index 21bc81f9306..947ddb476a3 100644 Binary files a/daprdocs/static/presentations/Dapr-Diagrams.pptx.zip and b/daprdocs/static/presentations/Dapr-Diagrams.pptx.zip differ