notebooks/PhollyIntroduction.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Pholly\n",
    "\n",
    "Welcome to the Pholly interactive notebook - this is an introduction to using resilience patterns with F# and Pholly which is itself based on Polly.\n",
    "\n",
    "## What is Pholly\n",
    "\n",
    "Pholly provides a more natural way to use Polly from within F# and uses a number of F# language features to provide an expressive DSL type approach to using common resilience patterns. Currently the three Polly reactive resilience patterns are supported:\n",
    "\n",
    "|Pattern|Purpose|\n",
    "|-------|-------|\n",
    "|Retry|Many faults are transient and may self-correct after a short delay and the retry pattern will attempt to retry the operation until it succeeds or a failure condition is met|\n",
    "|Circuit Breaker|When a system is seriously struggling, failing fast is better than making users/callers wait. The circuit breaker can protect a faulting system from overloading and help it recover.|\n",
    "|Fallback|Things can and do fail and it can be useful to provide a fallback|\n",
    "\n",
    "## Pholly Principles\n",
    "\n",
    "Pholly uses the F# Result<> type for its error control flow rather than the exceptions you normally see in Polly examples (which are often based on C#). It also makes use of tasks rather than F# async construct - internally it uses [Ply](https://github.com/crowded/ply) but you should be able to use the library with any task builder.\n",
    "\n",
    "## Using Pholly\n",
    "\n",
    "### Installing Pholly\n",
    "\n",
    "You can add Pholly either as a single source file [Pholly.fs](https://github.com/JamesRandall/Pholly/blob/main/src/Pholly.fs) or as a [NuGet package Pholly](https://www.nuget.org/packages/Pholly/0.0.2) using the package manager of your choice.\n",
    "\n",
    "### Retry Policy\n",
    "\n",
    "Before getting started with the policies we'll just get started by setting up some common concerns: import the Pholly package and create some methods that will let us roll a six sided dice and test if its 5 or higher."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "source": [
    "#!fsharp\n",
    "#r \"nuget: Pholly\"\n",
    "#r \"nuget: Ply\"\n",
    "\n",
    "open Pholly\n",
    "open Retry\n",
    "open System\n",
    "open System.Threading.Tasks\n",
    "open FSharp.Control.Tasks\n",
    "\n",
    "let log msg = printf \"%s\" msg\n",
    "\n",
    "let random = Random((DateTime.UtcNow.Ticks % (Int32.MaxValue |> int64)) |> int32)\n",
    "let rollDice () = random.Next(1,7)\n",
    "let isSuccessfulDiceRoll diceRoll = if diceRoll >= 5 then diceRoll |> Ok else \"Out of range\" |> Error"
   ],
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "Installed package Pholly version 0.0.7"
     },
     "execution_count": 1,
     "metadata": {}
    },
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "Installed package Ply version 0.3.1"
     },
     "execution_count": 1,
     "metadata": {}
    }
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we're going to wrap our dice roll in a retry policy and keep on retrying it until we roll a 5 or 6:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "source": [
    "#!fsharp\n",
    "let retryForever = Policy.retryForever [ ]\n",
    "let result = (rollDice >> isSuccessfulDiceRoll) |> retryForever\n",
    "sprintf \"Success - returned %d\" result |> log"
   ],
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "Success - returned 5"
     },
     "execution_count": 1,
     "metadata": {}
    }
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Its not usually helpful to retry an operation forever and so the code below builds on our forever example shows a simple retry policy that will roll a dice and continue to retry until a 5 or higher is rolled or 10 errors occur. Before each retry it will print a message with the retry count. Note that now we are not retrying forever we have to still expect a possible Result.Error:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "source": [
    "#!fsharp\n",
    "let retryPolicy = Policy.retry [\n",
    "    retry (upto 10<times>)\n",
    "    beforeEachRetry (fun _ retryAttempt _ -> sprintf \"Retrying attempt %d\" retryAttempt |> log)\n",
    "]\n",
    "\n",
    "match (rollDice >> isSuccessfulDiceRoll) |> retryPolicy with\n",
    "| Ok value -> sprintf \"Success - returned %d\" value |> log\n",
    "| Error error -> sprintf \"ERROR: %s\" error |> log\n",
    ""
   ],
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "Retrying attempt 1"
     },
     "execution_count": 1,
     "metadata": {}
    },
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "Retrying attempt 2"
     },
     "execution_count": 1,
     "metadata": {}
    },
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "Retrying attempt 3"
     },
     "execution_count": 1,
     "metadata": {}
    },
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "Success - returned 6"
     },
     "execution_count": 1,
     "metadata": {}
    }
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The next example shows an asynchronous retry policy for an asynchronous workload that also includes a back off sequence - each retry attempt will result in the next delay in the series. If their are more retries than their are specified intervals then the out of range retries will use the final interval."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "source": [
    "#!fsharp\n",
    "let asyncDiceroll () = task { return (rollDice () |> isSuccessfulDiceRoll) }\n",
    "let retryPolicy = Policy.retryAsync [\n",
    "    retry (upto 10<times>)\n",
    "    withIntervalsOf [50<ms> ; 500<ms> ; 1000<ms>]\n",
    "    beforeEachRetry (fun _ retryAttempt _ -> sprintf \"Retrying attempt %d\" retryAttempt |> log)\n",
    "]\n",
    "\n",
    "match asyncDiceroll |> retryPolicy |> Async.AwaitTask |> Async.RunSynchronously with\n",
    "| Ok value -> sprintf \"Success - returned %d\" value |> log\n",
    "| Error error -> sprintf \"ERROR: %s\" error |> log\n",
    ""
   ],
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "Retrying attempt 1"
     },
     "execution_count": 1,
     "metadata": {}
    },
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "Success - returned 5"
     },
     "execution_count": 1,
     "metadata": {}
    }
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Circuit Breaker\n",
    "\n",
    "The circuit breaker policy is a good way to keep a failing system space to recover - for example their's no point pounding on an API endpoint that is already overloaded and returning errors as that will likely hinder its recovery.\n",
    "\n",
    "First we'll begin by opening the circuit breaker module and creating a little helper to output the result of our circuit breaker protected operations:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "source": [
    "#!fsharp\n",
    "open CircuitBreaker\n",
    "\n",
    "let outputBreakerResult result = match result with | Ok s -> log s | Error e -> log (sprintf \"Error: %s\\n\" e)"
   ],
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next we'll create a circuit breaker such that if it returns 3 consecutive errors it will trip into the open state. When a circuit breaker is open, like an electric circuit, nothing will pass through it. We then use the breaker to make a series of calls. We'll start with a successful call (return an Ok) and then we'll generate 4 errors. Our circuit breaker will run the first 3 erroring operations and return the error result to us. The 3rd call will trip the breaker and so our fourth operation will never be called:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "source": [
    "#!fsharp\n",
    "// NOTE: would love feedback on if people prefer these as a three part tuple or would prefer a interface type approach e.g.\n",
    "// let breaker = Policy.circuitBreaker [...]\n",
    "// breaker.execute (...)\n",
    "let execute,reset,isolate =\n",
    "    Policy.circuitBreaker [\n",
    "        breakOn 3<consecutiveErrors>\n",
    "        whenCircuitIsOpenReturn (\"Circuit is open\" |> Error)\n",
    "    ]\n",
    "\n",
    "execute (fun () -> \"Success\" |> Ok) |> outputBreakerResult\n",
    "execute (fun () -> \"Consecutive error 1\" |> Error) |> outputBreakerResult\n",
    "execute (fun () -> \"Consecutive error 2\" |> Error) |> outputBreakerResult\n",
    "execute (fun () -> \"Consecutive error 3 - should trip breaker\" |> Error) |> outputBreakerResult\n",
    "execute (fun () -> \"Consecutive error 4 - should not be called, instead we should get 'Circuit is open'\" |> Error) |> outputBreakerResult"
   ],
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "Success"
     },
     "execution_count": 1,
     "metadata": {}
    },
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "Error: Consecutive error 1\n"
     },
     "execution_count": 1,
     "metadata": {}
    },
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "Error: Consecutive error 2\n"
     },
     "execution_count": 1,
     "metadata": {}
    },
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "Error: Consecutive error 3 - should trip breaker\n"
     },
     "execution_count": 1,
     "metadata": {}
    },
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "Error: Circuit is open\n"
     },
     "execution_count": 1,
     "metadata": {}
    }
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The next example breaks up our 3 consecutive errors with another successful result. This means the circuit breaker never trips as their are never more than 2 consecutive calls:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "source": [
    "#!fsharp\n",
    "execute (fun () -> \"Success\" |> Ok) |> outputBreakerResult\n",
    "execute (fun () -> \"Consecutive error a.1\" |> Error) |> outputBreakerResult\n",
    "execute (fun () -> \"Consecutive error a.2\" |> Error) |> outputBreakerResult\n",
    "execute (fun () -> \"Success\" |> Ok) |> outputBreakerResult\n",
    "execute (fun () -> \"Consecutive error b.1\" |> Error) |> outputBreakerResult\n",
    "execute (fun () -> \"Consecutive error b.2\" |> Error) |> outputBreakerResult"
   ],
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "Error: Circuit is open\n"
     },
     "execution_count": 1,
     "metadata": {}
    },
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "Error: Circuit is open\n"
     },
     "execution_count": 1,
     "metadata": {}
    },
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "Error: Circuit is open\n"
     },
     "execution_count": 1,
     "metadata": {}
    },
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "Error: Circuit is open\n"
     },
     "execution_count": 1,
     "metadata": {}
    },
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "Error: Circuit is open\n"
     },
     "execution_count": 1,
     "metadata": {}
    },
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "Error: Circuit is open\n"
     },
     "execution_count": 1,
     "metadata": {}
    }
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the next example we'll reset the open breaker so we can continue to make successful function calls:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "source": [
    "#!fsharp\n",
    "reset() // we will still be open from the previous demo\n",
    "execute (fun () -> \"Success\" |> Ok) |> outputBreakerResult\n",
    "execute (fun () -> \"Consecutive error 1\" |> Error) |> outputBreakerResult\n",
    "execute (fun () -> \"Consecutive error 2\" |> Error) |> outputBreakerResult\n",
    "execute (fun () -> \"Consecutive error 3 - should trip breaker\" |> Error) |> outputBreakerResult\n",
    "execute (fun () -> \"Consecutive error 4 - should not be called, instead we should get 'Circuit is open'\" |> Error) |> outputBreakerResult\n",
    "reset ()\n",
    "execute (fun () -> \"Success\" |> Ok) |> outputBreakerResult"
   ],
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "Success"
     },
     "execution_count": 1,
     "metadata": {}
    },
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "Error: Consecutive error 1\n"
     },
     "execution_count": 1,
     "metadata": {}
    },
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "Error: Consecutive error 2\n"
     },
     "execution_count": 1,
     "metadata": {}
    },
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "Error: Consecutive error 3 - should trip breaker\n"
     },
     "execution_count": 1,
     "metadata": {}
    },
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "Error: Circuit is open\n"
     },
     "execution_count": 1,
     "metadata": {}
    },
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "Success"
     },
     "execution_count": 1,
     "metadata": {}
    }
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next we'll show how you can manually force a circuit breaker open by using the isolate method. Immediately after calling isolate we will find our calls blocked by the breaker:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "source": [
    "#!fsharp\n",
    "execute (fun () -> \"Success\" |> Ok) |> outputBreakerResult\n",
    "isolate ()\n",
    "execute (fun () -> \"Will not be called\" |> Ok) |> outputBreakerResult"
   ],
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "Success"
     },
     "execution_count": 1,
     "metadata": {}
    },
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "Error: Circuit is open\n"
     },
     "execution_count": 1,
     "metadata": {}
    }
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Circuit breaks will automatically reset after a given duration - by default that is 1 minute with Pholly but you can change it by constructing a circuit breaker with the resetAfter option - this takes &lt;ms&gt; but their are simple helper functions to convert to other units. In this case we'll set the breaker to after a reset duration of 30 seconds:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "source": [
    "#!fsharp\n",
    "let execute,_,_ =\n",
    "    Policy.circuitBreaker [\n",
    "        breakOn 3<consecutiveErrors>\n",
    "        resetAfter (30 |> seconds)\n",
    "        whenCircuitIsOpenReturn (\"Circuit is open\" |> Error)\n",
    "    ]\n",
    "\n",
    "execute (fun () -> \"Success\" |> Ok) |> outputBreakerResult"
   ],
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "Success"
     },
     "execution_count": 1,
     "metadata": {}
    }
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also use circuit breakers with async operations:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "source": [
    "#!fsharp\n",
    "let outputBreakerResultAsync (result:Task<Result<string,string>>) = task { match! result with | Ok s -> log s | Error e -> log (sprintf \"Error: %s\\n\" e) }\n",
    "\n",
    "let executeAsync,_,_ =\n",
    "    Policy.circuitBreakerAsync [\n",
    "        breakOn 3<consecutiveErrors>\n",
    "        whenCircuitIsOpenReturn (\"Circuit is open\" |> Error)\n",
    "    ]\n",
    "\n",
    "let asyncSuccess () = task { return \"Success\" |> Ok }\n",
    "let asyncError () = task { return \"Error\" |> Error }\n",
    "\n",
    "task {\n",
    "    do! asyncSuccess |> executeAsync |> outputBreakerResultAsync\n",
    "    do! asyncError |> executeAsync |> outputBreakerResultAsync\n",
    "    do! asyncError |> executeAsync |> outputBreakerResultAsync\n",
    "    do! asyncError |> executeAsync |> outputBreakerResultAsync\n",
    "    do! asyncSuccess |> executeAsync |> outputBreakerResultAsync    \n",
    "} |> Async.AwaitTask |> Async.RunSynchronously"
   ],
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "Success"
     },
     "execution_count": 1,
     "metadata": {}
    },
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "Error: Error\n"
     },
     "execution_count": 1,
     "metadata": {}
    },
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "Error: Error\n"
     },
     "execution_count": 1,
     "metadata": {}
    },
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "Error: Error\n"
     },
     "execution_count": 1,
     "metadata": {}
    },
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "Error: Circuit is open\n"
     },
     "execution_count": 1,
     "metadata": {}
    }
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Fallback\n",
    "\n",
    "When all else fails it can be useful to provide a fallback result and you can do this with the fallback policy. The example below will fallback to the meaning of life if an error occurs:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "source": [
    "#!fsharp\n",
    "let fallbackPolicy = Policy.fallbackWith 42\n",
    "(fun () -> \"Gone wrong\" |> Error) |> fallbackPolicy |> sprintf \"Fallback result: %d\" |> log\n",
    "(fun () -> 1701 |> Ok) |> fallbackPolicy |> sprintf \"Fallback result: %d\" |> log"
   ],
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "Fallback result: 42"
     },
     "execution_count": 1,
     "metadata": {}
    },
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "Fallback result: 1701"
     },
     "execution_count": 1,
     "metadata": {}
    }
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "source": [
    "#!fsharp\n",
    ""
   ],
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Policy Composition\n",
    "\n",
    "Its possible to combine policies to provide protection in depth by using the -|> and --|> operators (for none-async and async respectively). For example in the below we run code inside a circuit breaker that is wrapped in a retry block and ultimate provides a fallback value when all else fails:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "source": [
    "#!fsharp\n",
    "let failingWorkload = fun () -> \"Always going wrong to show error control flow\" |> Error\n",
    "let breakerPolicy,_,_ = Policy.circuitBreaker [\n",
    "    breakOn 3<consecutiveErrors>\n",
    "    whenCircuitIsOpenReturn (\"Circuit is open, execution blocked\" |> Error)\n",
    "    whenCircuitIsOpened (fun _ _ _ -> \"Circuit opening\\n\" |> log)\n",
    "  ]\n",
    "let retryPolicy = Policy.retry [\n",
    "    retry (upto 10<times>)\n",
    "    beforeEachRetry (fun r _ _ -> sprintf \"Retrying after error: %s \\n\" r |> log)\n",
    "]\n",
    "let fallbackPolicy = Policy.fallbackWith 42  \n",
    "  \n",
    "let result = failingWorkload |> (breakerPolicy -|> retryPolicy -|> fallbackPolicy)\n",
    "  \n",
    "sprintf \"Final result %d\" result |> log"
   ],
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "Retrying after error: Always going wrong to show error control flow \n"
     },
     "execution_count": 1,
     "metadata": {}
    },
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "Retrying after error: Always going wrong to show error control flow \n"
     },
     "execution_count": 1,
     "metadata": {}
    },
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "Circuit opening\n"
     },
     "execution_count": 1,
     "metadata": {}
    },
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "Retrying after error: Always going wrong to show error control flow \n"
     },
     "execution_count": 1,
     "metadata": {}
    },
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "Retrying after error: Circuit is open, execution blocked \n"
     },
     "execution_count": 1,
     "metadata": {}
    },
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "Retrying after error: Circuit is open, execution blocked \n"
     },
     "execution_count": 1,
     "metadata": {}
    },
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "Retrying after error: Circuit is open, execution blocked \n"
     },
     "execution_count": 1,
     "metadata": {}
    },
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "Retrying after error: Circuit is open, execution blocked \n"
     },
     "execution_count": 1,
     "metadata": {}
    },
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "Retrying after error: Circuit is open, execution blocked \n"
     },
     "execution_count": 1,
     "metadata": {}
    },
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "Retrying after error: Circuit is open, execution blocked \n"
     },
     "execution_count": 1,
     "metadata": {}
    },
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "Retrying after error: Circuit is open, execution blocked \n"
     },
     "execution_count": 1,
     "metadata": {}
    },
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "Final result 42"
     },
     "execution_count": 1,
     "metadata": {}
    }
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The async version is pretty much the same but note the use of the --|> operator for composition and in this case I've composed the policy into a reusable value:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "source": [
    "#!fsharp\n",
    "task {  \n",
    "    let failingAsyncWorkload () = task { return \"Always going wrong to show error control flow\" |> Error }\n",
    "    let asyncBreakerPolicy,_,_ = Policy.circuitBreakerAsync [\n",
    "        breakOn 3<consecutiveErrors>\n",
    "        whenCircuitIsOpenReturn (\"Circuit is open, execution blocked\" |> Error)\n",
    "        whenCircuitIsOpened (fun _ _ _ -> \"Circuit opening\\n\" |> log)\n",
    "    ]\n",
    "    let asyncRetryPolicy = Policy.retryAsync [\n",
    "        retry (upto 10<times>)\n",
    "        beforeEachRetry (fun r _ _ -> sprintf \"Retrying after error: %s \\n\" r |> log)\n",
    "    ]\n",
    "    let asyncFallbackPolicy = Policy.fallbackAsyncWith 42  \n",
    "    let composedPolicy = asyncBreakerPolicy --|> asyncRetryPolicy --|> asyncFallbackPolicy\n",
    "\n",
    "    let! result = failingAsyncWorkload |> composedPolicy\n",
    "    \n",
    "    sprintf \"Final result %d\" result |> log\n",
    "    return ()\n",
    "} |> Async.AwaitTask |> Async.RunSynchronously"
   ],
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "Retrying after error: Always going wrong to show error control flow \n"
     },
     "execution_count": 1,
     "metadata": {}
    },
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "Retrying after error: Always going wrong to show error control flow \n"
     },
     "execution_count": 1,
     "metadata": {}
    },
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "Circuit opening\n"
     },
     "execution_count": 1,
     "metadata": {}
    },
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "Retrying after error: Always going wrong to show error control flow \n"
     },
     "execution_count": 1,
     "metadata": {}
    },
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "Retrying after error: Circuit is open, execution blocked \n"
     },
     "execution_count": 1,
     "metadata": {}
    },
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "Retrying after error: Circuit is open, execution blocked \n"
     },
     "execution_count": 1,
     "metadata": {}
    },
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "Retrying after error: Circuit is open, execution blocked \n"
     },
     "execution_count": 1,
     "metadata": {}
    },
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "Retrying after error: Circuit is open, execution blocked \n"
     },
     "execution_count": 1,
     "metadata": {}
    },
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "Retrying after error: Circuit is open, execution blocked \n"
     },
     "execution_count": 1,
     "metadata": {}
    },
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "Retrying after error: Circuit is open, execution blocked \n"
     },
     "execution_count": 1,
     "metadata": {}
    },
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "Retrying after error: Circuit is open, execution blocked \n"
     },
     "execution_count": 1,
     "metadata": {}
    },
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "Final result 42"
     },
     "execution_count": 1,
     "metadata": {}
    }
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Thanks\n",
    "\n",
    "Many thanks to the creators of [Polly](https://github.com/App-vNext/Polly) itself - I've used it in many production systems and I've used some of their descriptions of the patterns here.\n",
    "\n",
    "And thanks to [Scott Wlaschin](https://twitter.com/ScottWlaschin) for the name. If I have a kryptonite its naming things."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": ".NET (C#)",
   "language": "C#",
   "name": ".net-csharp"
  },
  "language_info": {
   "file_extension": ".cs",
   "mimetype": "text/x-csharp",
   "name": "C#",
   "pygments_lexer": "csharp",
   "version": "8.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}