Add documentation for actor reentrancy (#1483)

* Add documentation for actor reentrancy

#1482

* Incoporate PR feedback

Co-authored-by: Aaron Crawfis <Aaron.Crawfis@microsoft.com>

* Add link from preview feature to reentrancy

* Update daprdocs/content/en/developing-applications/building-blocks/actors/actor-reentrancy.md

Co-authored-by: Aaron Crawfis <Aaron.Crawfis@microsoft.com>

daprdocs/content/en/developing-applications/building-blocks/actors/actor-reentrancy.md

@@ -0,0 +1,94 @@
+---
+type: docs
+title: "How-to: Enable and use actor reentrancy in Dapr"
+linkTitle: "How-To: Actor reentrancy"
+weight: 30
+description: Learn more about actor reentrancy
+---
+
+{{% alert title="Preview feature" color="warning" %}}
+Actor reentrancy is currently in [preview]({{< ref preview-features.md >}}).
+{{% /alert %}}
+
+## Actor reentrancy
+A core tenet of the virtual actor pattern is the single-threaded nature of actor execution. Before reentrancy, this caused the Dapr runtime to lock an actor on any given request. A second request could not start until the first had completed. This behavior means an actor cannot call itself, or have another actor call into it even if it is part of the same chain. Reentrancy solves this by allowing requests from the same chain or context to re-enter into an already locked actor. Examples of chains that reentrancy allows can be seen below:
+
+```
+Actor A -> Actor A
+ActorA -> Actor B -> Actor A
+```
+
+With reentrancy, there can be more complex actor calls without sacrificing the single-threaded behavior of virtual actors.
+
+## Enabling actor reentrancy
+Actor reentrancy is currently in preview, so enabling it is a two step process.
+
+### Preview feature configuration
+Before using reentrancy, the feature must be enabled in Dapr. For more information on preview configurations, see [the full guide on opting into preview features in Dapr]({{< ref preview-features.md >}}). Below is an example of the configuration for actor reentrancy:
+
+```yaml
+apiVersion: dapr.io/v1alpha1
+kind: Configuration
+metadata:
+  name: reentrantconfig
+spec:
+  features:
+    - name: Actor.Reentrancy
+      enabled: true
+```
+
+### Actor runtime configuration
+Once actor reentrancy is enabled as an opt-in preview feature, the actor that will be reentrant must also provide the appropriate configuration to use reentrancy. This is done by the actor's endpoint for `GET /dapr/config`, similar to other actor configuration elements. Here is a snipet of an actor written in Golang providing the configuration:
+
+```go
+type daprConfig struct {
+	Entities                []string                `json:"entities,omitempty"`
+	ActorIdleTimeout        string                  `json:"actorIdleTimeout,omitempty"`
+	ActorScanInterval       string                  `json:"actorScanInterval,omitempty"`
+	DrainOngoingCallTimeout string                  `json:"drainOngoingCallTimeout,omitempty"`
+	DrainRebalancedActors   bool                    `json:"drainRebalancedActors,omitempty"`
+	Reentrancy              config.ReentrancyConfig `json:"reentrancy,omitempty"`
+}
+
+var daprConfigResponse = daprConfig{
+	[]string{defaultActorType},
+	actorIdleTimeout,
+	actorScanInterval,
+	drainOngoingCallTimeout,
+	drainRebalancedActors,
+	config.ReentrancyConfig{Enabled: true, MaxStackDepth: &maxStackDepth},
+}
+
+func configHandler(w http.ResponseWriter, r *http.Request) {
+	w.Header().Set("Content-Type", "application/json")
+	w.WriteHeader(http.StatusOK)
+	json.NewEncoder(w).Encode(daprConfigResponse)
+}
+```
+
+### Handling reentrant requests
+The key to a reentrant request is the `Dapr-Reentrancy-Id` header. The value of this header is used to match requests to their call chain and allow them to bypass the actor's lock. 
+
+The header is generated by the Dapr runtime for any actor request that has a reentrant config specified. Once it is generated, it is used to lock the actor and must be passed to all future requests. Below is a snippet of code from an actor handling this is Golang:
+
+```go
+func reentrantCallHandler(w http.ResponseWriter, r *http.Request) {
+    /*
+     * Omitted.
+     */
+
+	req, _ := http.NewRequest("PUT", url, bytes.NewReader(nextBody))
+
+	reentrancyID := r.Header.Get("Dapr-Reentrancy-Id")
+	req.Header.Add("Dapr-Reentrancy-Id", reentrancyID)
+
+	client := http.Client{}
+	resp, err := client.Do(req)
+
+    /*
+     * Omitted.
+     */
+}
+```
+
+Currently, no SDK supports actor reentrancy. In the future, the method for handling the reentrancy id may be different based on the SDK that is being used.

daprdocs/content/en/developing-applications/building-blocks/actors/actors-overview.md

@@ -89,6 +89,8 @@ Actors can deadlock on each other if there is a circular request between two act
 
 <img src="/images/actors_background_communication.png" width=600>
 
+#### Reentrancy
+As an enhancement to the base actors in dapr, reentrancy can now be enabled as a preview feature. To learn more about it, see [actor reentrancy]({{<ref actor-reentrancy.md>}})
 
 ### Turn-based access
 

daprdocs/content/en/operations/configuration/preview-features.md

@@ -13,7 +13,7 @@ Currently, preview features are enabled on a per application basis when running
 
 ### Current preview features
 Below is a list of existing preview features:
-- Actor Reentrancy
+- [Actor Reentrancy]({{<ref actor-reentrancy.md>}})
 
 ## Configuration properties
 The `features` section under the `Configuration` spec contains the following properties: