Skip to content

Commit

Permalink
[Docs] Add telemetry section to chaos strategies documentation pages (#…
Browse files Browse the repository at this point in the history
  • Loading branch information
peter-csala authored Apr 27, 2024
1 parent 07f09a6 commit 5441dc9
Show file tree
Hide file tree
Showing 12 changed files with 188 additions and 65 deletions.
22 changes: 13 additions & 9 deletions docs/advanced/telemetry.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Telemetry

Starting with version 8, Polly provides telemetry for all built-in resilience strategies.
Starting with version 8, Polly provides telemetry for all built-in standard and chaos resilience strategies.

## Usage

Expand Down Expand Up @@ -140,14 +140,18 @@ Tags:

The `event.name` tag is reported by individual resilience strategies. The built-in strategies report the following events:

- `OnRetry`
- `OnFallback`
- `OnHedging`
- `OnTimeout`
- `OnCircuitClosed`
- `OnCircuitOpened`
- `OnCircuitHalfOpened`
- `OnRateLimiterRejected`
- [`OnRetry`](../strategies/retry.md#telemetry)
- [`OnFallback`](../strategies/fallback.md#telemetry)
- [`OnHedging`](../strategies/hedging.md#telemetry)
- [`OnTimeout`](../strategies/timeout.md#telemetry)
- [`OnCircuitClosed`](../strategies/circuit-breaker.md#telemetry)
- [`OnCircuitOpened`](../strategies/circuit-breaker.md#telemetry)
- [`OnCircuitHalfOpened`](../strategies/circuit-breaker.md#telemetry)
- [`OnRateLimiterRejected`](../strategies/rate-limiter.md#telemetry)
- [`Chaos.OnFault`](../chaos/fault.md#telemetry)
- [`Chaos.OnOutcome`](../chaos/outcome.md#telemetry)
- [`Chaos.OnLatency`](../chaos/latency.md#telemetry)
- [`Chaos.OnBehavior`](../chaos/behavior.md#telemetry)

### Instrument: `resilience.polly.strategy.attempt.duration`

Expand Down
43 changes: 35 additions & 8 deletions docs/chaos/behavior.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,13 +2,15 @@

## About

- **Options**: [`ChaosBehaviorStrategyOptions`](xref:Polly.Simmy.Behavior.ChaosBehaviorStrategyOptions)
- **Extensions**: `AddChaosBehavior`
- **Strategy Type**: Proactive
- **Option(s)**:
- [`ChaosBehaviorStrategyOptions`](xref:Polly.Simmy.Behavior.ChaosBehaviorStrategyOptions)
- **Extension(s)**:
- `AddChaosBehavior`
- **Exception(s)**: -

---

The behavior chaos strategy is designed to inject custom behaviors into system operations right before such an operation is invoked. This strategy is flexible, allowing users to define specific behaviors such as altering the input, simulating resource exhaustion, putting the system in a given state before the actual operation is called, or other operational variations to simulate real-world scenarios.
The behavior **proactive** chaos strategy is designed to inject custom behaviors into system operations right before such an operation is invoked. This strategy is flexible, allowing users to define specific behaviors such as altering the input, simulating resource exhaustion, putting the system in a given state before the actual operation is called, or other operational variations to simulate real-world scenarios.

## Usage

Expand Down Expand Up @@ -66,10 +68,35 @@ var pipeline = new ResiliencePipelineBuilder()

## Defaults

| Property | Default Value | Description |
|----------------------|---------------|------------------------------------------------|
| `OnBehaviorInjected` | `null` | Action executed when the behavior is injected. |
| `BehaviorGenerator` | `null` | Custom behavior to be injected. |
| Property | Default Value | Description |
|----------------------|---------------|------------------------------------------------------------------------------------------------------------------------------|
| `BehaviorGenerator` | `null` | This required delegate allows you to inject custom behavior by utilizing information that is only available at runtime. |
| `OnBehaviorInjected` | `null` | If provided then it will be invoked after the behavior injection occurred. |

## Telemetry

The behavior chaos strategy reports the following telemetry events:

| Event Name | Event Severity | When? |
|--------------------|----------------|------------------------------------------------------------------|
| `Chaos.OnBehavior` | `Information` | Just before the strategy calls the `OnBehaviorInjected` delegate |

Here are some sample events:

```none
Resilience event occurred. EventName: 'Chaos.OnBehavior', Source: '(null)/(null)/Chaos.Behavior', Operation Key: '', Result: ''
Resilience event occurred. EventName: 'Chaos.OnBehavior', Source: 'MyPipeline/MyPipelineInstance/MyChaosBehaviorStrategy', Operation Key: 'MyBehaviorInjectedOperation', Result: ''
```

> [!NOTE]
> Please note that the `Chaos.OnBehavior` telemetry event will be reported **only if** the behavior chaos strategy injects a custom behavior which does not throw exception.
>
> So, if the behavior is either not injected or injected and throws an exception then there will be no telemetry emitted.
>
> Also remember that the `Result` will be **always empty** for the `Chaos.OnBehavior` telemetry event.
For further information please check out the [telemetry page](../advanced/telemetry.md).

## Diagrams

Expand Down
43 changes: 35 additions & 8 deletions docs/chaos/fault.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,13 +2,15 @@

## About

- **Options**: [`ChaosFaultStrategyOptions`](xref:Polly.Simmy.Fault.ChaosFaultStrategyOptions)
- **Extensions**: `AddChaosFault`
- **Strategy Type**: Proactive
- **Option(s)**:
- [`ChaosFaultStrategyOptions`](xref:Polly.Simmy.Fault.ChaosFaultStrategyOptions)
- **Extensions**:
- `AddChaosFault`
- **Exception(s)**: -

---

The fault chaos strategy is designed to introduce faults (exceptions) into the system, simulating real-world scenarios where operations might fail unexpectedly. It is configurable to inject specific types of exceptions or use custom logic to generate faults dynamically.
The fault **proactive** chaos strategy is designed to introduce faults (exceptions) into the system, simulating real-world scenarios where operations might fail unexpectedly. It is configurable to inject specific types of exceptions or use custom logic to generate faults dynamically.

## Usage

Expand Down Expand Up @@ -87,10 +89,35 @@ var pipeline = new ResiliencePipelineBuilder()

## Defaults

| Property | Default Value | Description |
|-------------------|---------------|------------------------------------------------------|
| `OnFaultInjected` | `null` | Action executed when the fault is injected. |
| `FaultGenerator` | `null` | Generates the fault to inject for a given execution. |
| Property | Default Value | Description |
|-------------------|---------------|------------------------------------------------------------------------------------------------------------------------|
| `FaultGenerator` | `null` | This required delegate allows you to inject exception by utilizing information that is only available at runtime. |
| `OnFaultInjected` | `null` | If provided then it will be invoked after the fault injection occurred. |

## Telemetry

The fault chaos strategy reports the following telemetry events:

| Event Name | Event Severity | When? |
|-----------------|----------------|---------------------------------------------------------------|
| `Chaos.OnFault` | `Information` | Just before the strategy calls the `OnFaultInjected` delegate |

Here are some sample events:

```none
Resilience event occurred. EventName: 'Chaos.OnFault', Source: '(null)/(null)/Chaos.Fault', Operation Key: '', Result: ''
Resilience event occurred. EventName: 'Chaos.OnFault', Source: 'MyPipeline/MyPipelineInstance/MyChaosFaultStrategy', Operation Key: 'MyFaultInjectedOperation', Result: ''
```

> [!NOTE]
> Please note that the `Chaos.OnFault` telemetry event will be reported **only if** the fault chaos strategy injects an exception which is wrapped into a `ValueTask`.
>
> So, if the fault is either not injected or injected and throws an exception then there will be no telemetry emitted.
>
> Also remember that the `Result` will be **always empty** for the `Chaos.OnFault` telemetry event.
For further information please check out the [telemetry page](../advanced/telemetry.md).

## Diagrams

Expand Down
34 changes: 19 additions & 15 deletions docs/chaos/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,10 +52,14 @@ This section highlights the major differences compared to the [`Polly.Contrib.Si
- **Unified configuration options**: The `InjectOptionsBase` and `InjectOptionsAsyncBase` are now consolidated into `ChaosStrategyOptions`. This change brings Simmy in line with the Polly v8 API, offering built-in support for options-based configuration and seamless integration of synchronous and asynchronous executions.
- **Chaos strategies enabled by default**: Adding a chaos strategy (previously known as monkey policy) now means it's active right away. This is a departure from earlier versions, where the monkey policy had to be explicitly enabled.
- **API changes**: The new version of Simmy introduces several API updates. While this list isn't complete, it includes key changes like renaming `Inject` to `AddChaos` and switching from `Result` to `Outcome`. Here are some specific renames:
- `InjectException` is now `AddChaosFault`
- `InjectResult` is now `AddChaosOutcome`
- `InjectBehavior` is now `AddChaosBehavior`
- `InjectLatency` is now `AddChaosLatency`

| From | To |
|-------------------|--------------------|
| `InjectException` | `AddChaosFault` |
| `InjectResult` | `AddChaosOutcome` |
| `InjectBehavior` | `AddChaosBehavior` |
| `InjectLatency` | `AddChaosLatency` |

- **Sync and async unification**: Before, Simmy had various methods to set policies like `InjectLatency`, `InjectLatencyAsync`, `InjectLatency<T>`, and `InjectLatencyAsync<T>`. With the new version based on Polly v8, these methods have been combined into a single `AddChaosLatency` extension that works for both `ResiliencePipelineBuilder` and `ResiliencePipelineBuilder<T>`. These rules are covering all types of chaos strategies (Outcome, Fault, Latency, and Behavior).

## Motivation
Expand All @@ -82,12 +86,12 @@ Chaos strategies (formerly known as Monkey strategies) are in essence a [Resilie

### Built-in strategies

| Strategy | Reactive | What does the strategy do? |
|-------------------------|----------|----------------------------------------------------------------------|
| [Fault](fault.md) | No | Injects exceptions in your system. |
| [Outcome](outcome.md) | Yes | Injects fake outcomes (results or exceptions) in your system. |
| [Latency](latency.md) | No | Injects latency into executions before the calls are made. |
| [Behavior](behavior.md) | No | Allows you to inject *any* extra behavior, before a call is placed. |
| Strategy | Type | What does the strategy do? |
|-------------------------|-----------|---------------------------------------------------------------------|
| [Fault](fault.md) | Proactive | Injects exceptions in your system. |
| [Outcome](outcome.md) | Reactive | Injects fake outcomes (results or exceptions) in your system. |
| [Latency](latency.md) | Proactive | Injects latency into executions before the calls are made. |
| [Behavior](behavior.md) | Proactive | Allows you to inject *any* extra behavior, before a call is placed. |

## Common options across strategies

Expand All @@ -114,12 +118,12 @@ All the strategies' options implement the [`ChaosStrategyOptions`](xref:Polly.Si

## Telemetry

The telemetry of chaos strategies is seamlessly integrated with Polly [telemetry infrastructure](../advanced/telemetry.md). The chaos strategies produce the following events:
The telemetry of chaos strategies is seamlessly integrated with Polly [telemetry infrastructure](../advanced/telemetry.md). The chaos strategies produce the following information events:

- `Chaos.OnFault`
- `Chaos.OnOutcome`
- `Chaos.OnLatency`
- `Chaos.OnBehavior`
- [`Chaos.OnFault`](fault.md#telemetry)
- [`Chaos.OnOutcome`](outcome.md#telemetry)
- [`Chaos.OnLatency`](latency.md#telemetry)
- [`Chaos.OnBehavior`](behavior.md#telemetry)

## Patterns

Expand Down
50 changes: 41 additions & 9 deletions docs/chaos/latency.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,13 +2,15 @@

## About

- **Options**: [`ChaosLatencyStrategyOptions`](xref:Polly.Simmy.Latency.ChaosLatencyStrategyOptions)
- **Extensions**: `AddChaosLatency`
- **Strategy Type**: Proactive
- **Option(s)**:
- [`ChaosLatencyStrategyOptions`](xref:Polly.Simmy.Latency.ChaosLatencyStrategyOptions)
- **Extension(s)**:
- `AddChaosLatency`
- **Exception(s)**: -

---

The latency chaos strategy is designed to introduce controlled delays into system operations, simulating network latency or slow processing times. This strategy helps in assessing and improving the resilience of applications against increased response times.
The latency **proactive** chaos strategy is designed to introduce controlled delays into system operations, simulating network latency or slow processing times. This strategy helps in assessing and improving the resilience of applications against increased response times.

## Usage

Expand Down Expand Up @@ -90,15 +92,45 @@ var pipeline = new ResiliencePipelineBuilder()

## Defaults

| Property | Default Value | Description |
|---------------------|---------------|--------------------------------------------------------|
| `Latency` | `30 seconds` | A `TimeSpan` indicating the delay to be injected. |
| `LatencyGenerator` | `null` | Generates the latency to inject for a given execution. |
| `OnLatencyInjected` | `null` | Action executed when latency is injected. |
| Property | Default Value | Description |
|---------------------|---------------|----------------------------------------------------------------------------------------------------------------------|
| `Latency` | 30 seconds | Defines a **fixed** delay to be injected. |
| `LatencyGenerator` | `null` | This delegate allows you to **dynamically** inject delay by utilizing information that is only available at runtime. |
| `OnLatencyInjected` | `null` | If provided then it will be invoked after the latency injection occurred. |

> [!NOTE]
> If both `Latency` and `LatencyGenerator` are specified then `Latency` will be ignored.
---

> [!IMPORTANT]
> Please note that if the calculated latency is negative (regardless if it's fixed or dynamic) then it will not be injected.
## Telemetry

The latency chaos strategy reports the following telemetry events:

| Event Name | Event Severity | When? |
|-------------------|----------------|-----------------------------------------------------------------|
| `Chaos.OnLatency` | `Information` | Just before the strategy calls the `OnLatencyInjected` delegate |

Here are some sample events:

```none
Resilience event occurred. EventName: 'Chaos.OnLatency', Source: '(null)/(null)/Chaos.Latency', Operation Key: '', Result: ''
Resilience event occurred. EventName: 'Chaos.OnLatency', Source: 'MyPipeline/MyPipelineInstance/MyLatencyStrategy', Operation Key: 'MyLatencyInjectedOperation', Result: ''
```

> [!NOTE]
> Please note that the `Chaos.OnLatency` telemetry event will be reported **only if** the latency chaos strategy injects a positive delay.
>
> So, if the latency is not injected then there will be no telemetry emitted. Also if injected but the latency is negative or the `LatencyGenerator` throws an exception then there will be no telemetry emitted.
>
> Also remember that the `Result` will be **always empty** for the `Chaos.OnLatency` telemetry event.
For further information please check out the [telemetry page](../advanced/telemetry.md).

## Diagrams

### Normal 🐵 sequence diagram
Expand Down
45 changes: 37 additions & 8 deletions docs/chaos/outcome.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,14 +2,15 @@

## About

- **Options**:
- **Option(s)**:
- [`ChaosOutcomeStrategyOptions<T>`](xref:Polly.Simmy.Outcomes.ChaosOutcomeStrategyOptions`1)
- **Extensions**: `AddChaosOutcome`
- **Strategy Type**: Reactive
- **Extension(s)**:
- `AddChaosOutcome`
- **Exception(s)**: -

---

The outcome chaos strategy is designed to inject or substitute fake results into system operations. This allows testing how an application behaves when it receives different types of responses, like successful results, errors, or exceptions.
The outcome **reactive** chaos strategy is designed to inject or substitute fake results into system operations. This allows testing how an application behaves when it receives different types of responses, like successful results, errors, or exceptions.

## Usage

Expand Down Expand Up @@ -79,10 +80,38 @@ var pipeline = new ResiliencePipelineBuilder<HttpResponseMessage>()

## Defaults

| Property | Default Value | Description |
|---------------------|---------------|---------------------------------------------------------|
| `OutcomeGenerator` | `null` | Function to generate the outcome for a given execution. |
| `OnOutcomeInjected` | `null` | Action executed when the outcome is injected. |
| Property | Default Value | Description |
|---------------------|---------------|-----------------------------------------------------------------------------------------------------------------------------|
| `OutcomeGenerator` | `null` | This required delegate allows you to inject custom outcome by utilizing information that is only available at runtime. |
| `OnOutcomeInjected` | `null` | If provided then it will be invoked after the outcome injection occurred. |

> [!NOTE]
> Please note this strategy is a reactive chaos strategy, but it does not have a `ShouldHandle` delegate.
## Telemetry

The outcome chaos strategy reports the following telemetry events:

| Event Name | Event Severity | When? |
|-------------------|----------------|-----------------------------------------------------------------|
| `Chaos.OnOutcome` | `Information` | Just before the strategy calls the `OnOutcomeInjected` delegate |

Here are some sample events:

```none
Resilience event occurred. EventName: 'Chaos.OnOutcome', Source: '(null)/(null)/Chaos.Outcome', Operation Key: '', Result: ''
Resilience event occurred. EventName: 'Chaos.OnOutcome', Source: 'MyPipeline/MyPipelineInstance/MyOutcomeStrategy', Operation Key: 'MyOutcomeInjectedOperation', Result: ''
```

> [!NOTE]
> Please note that the `Chaos.OnOutcome` telemetry event will be reported **only if** the outcome chaos strategy injects an outcome object.
>
> So, if the outcome is not injected or injected but the generator delegate throws an exception then there will be no telemetry emitted.
>
> Also remember that the `Result` will be **always empty** for the `Chaos.OnOutcome` telemetry event.
For further information please check out the [telemetry page](../advanced/telemetry.md).

## Diagrams

Expand Down
Loading

0 comments on commit 5441dc9

Please sign in to comment.