dynamic_programming/robustness.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "<a id='rob'></a>\n",
    "<div id=\"qe-notebook-header\" style=\"text-align:right;\">\n",
    "        <a href=\"https://quantecon.org/\" title=\"quantecon.org\">\n",
    "                <img style=\"width:250px;display:inline;\" src=\"https://assets.quantecon.org/img/qe-menubar-logo.svg\" alt=\"QuantEcon\">\n",
    "        </a>\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Robustness\n",
    "\n",
    "\n",
    "<a id='index-0'></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Contents\n",
    "\n",
    "- [Robustness](#Robustness)  \n",
    "  - [Overview](#Overview)  \n",
    "  - [The Model](#The-Model)  \n",
    "  - [Constructing More Robust Policies](#Constructing-More-Robust-Policies)  \n",
    "  - [Robustness as Outcome of a Two-Person Zero-Sum Game](#Robustness-as-Outcome-of-a-Two-Person-Zero-Sum-Game)  \n",
    "  - [The Stochastic Case](#The-Stochastic-Case)  \n",
    "  - [Implementation](#Implementation)  \n",
    "  - [Application](#Application)  \n",
    "  - [Appendix](#Appendix)  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Overview\n",
    "\n",
    "\n",
    "<a id='index-1'></a>\n",
    "This lecture modifies a Bellman equation to express a decision maker’s doubts about  transition dynamics\n",
    "\n",
    "His specification doubts make the decision maker want a *robust* decision rule\n",
    "\n",
    "*Robust* means insensitive to misspecification of transition dynamics\n",
    "\n",
    "The decision maker has a single *approximating model*\n",
    "\n",
    "He calls it *approximating* to acknowledge that he doesn’t completely trust it\n",
    "\n",
    "He fears that outcomes will actually be determined by another model that he cannot describe explicitly\n",
    "\n",
    "All that he knows is that the actual data-generating model is in some (uncountable) set of models that surrounds his approximating model\n",
    "\n",
    "He quantifies the discrepancy between his approximating model and the genuine data-generating model by using a quantity called *entropy*\n",
    "\n",
    "(We’ll explain what entropy means below)\n",
    "\n",
    "He wants a decision rule that will work well enough no matter which of those other models actually governs outcomes\n",
    "\n",
    "This is what it means for his decision rule to be “robust to misspecification of an approximating model”\n",
    "\n",
    "This may  sound like too much to ask for, but $ \\ldots $\n",
    "\n",
    "$ \\ldots $  a *secret weapon* is available to design robust decision rules\n",
    "\n",
    "The secret weapon is max-min control theory\n",
    "\n",
    "A value-maximizing decision maker enlists the aid of an (imaginary) value-minimizing model chooser to construct *bounds* on the value attained by a given decision rule under different models of the transition dynamics\n",
    "\n",
    "The original decision maker uses those bounds to construct a decision rule with an assured performance level, no matter which model actually governs outcomes\n",
    "\n",
    ">**Note**\n",
    ">\n",
    ">In reading this lecture, please don’t think that our decision maker is paranoid when he conducts a worst-case analysis. By designing a rule that works well against a worst-case, his intention is to construct a rule that will work well across a *set* of models.\n",
    "\n",
    "\n",
    "<a id='rb-vec'></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Sets of Models Imply Sets Of Values\n",
    "\n",
    "Our “robust” decision maker wants to know how well a given rule will work when he does not *know* a single transition law  $ \\ldots $\n",
    "\n",
    "$ \\ldots $ he wants to know  *sets* of values that will be attained by a given decision rule $ F $ under a *set* of transition laws\n",
    "\n",
    "Ultimately, he wants to design a decision rule $ F $ that shapes these *sets* of values in ways that he prefers\n",
    "\n",
    "With this in mind, consider the following graph, which relates to a particular decision problem to be explained below\n",
    "\n",
    "<img src=\"https://s3-ap-southeast-2.amazonaws.com/lectures.quantecon.org/jl/_static/figures/kg0.png\" style=\"width:70%;height:70%\">\n",
    "\n",
    "  \n",
    "The figure shows a *value-entropy correspondence* for a particular decision rule $ F $\n",
    "\n",
    "The shaded set is the graph of the correspondence, which maps entropy to a set of values associated with a set of models that surround the decision maker’s approximating model\n",
    "\n",
    "Here\n",
    "\n",
    "- *Value* refers to a sum of discounted rewards obtained by applying the decision rule $ F $ when the state starts at some fixed initial state $ x_0 $  \n",
    "- *Entropy* is a nonnegative number that measures the size of a set of models surrounding the decision maker’s approximating model  \n",
    "  \n",
    "  - Entropy is zero when the set includes only the approximating model, indicating that the decision maker completely trusts the approximating model  \n",
    "  - Entropy is bigger, and the set of surrounding models is bigger, the less the decision maker trusts the approximating model  \n",
    "  \n",
    "\n",
    "\n",
    "The shaded region indicates that for **all** models having entropy less than or equal to the number on the horizontal axis, the value obtained will be somewhere within the indicated set of values\n",
    "\n",
    "Now let’s compare sets of values associated with two different decision rules, $ F_r $ and $ F_b $\n",
    "\n",
    "In the next figure,\n",
    "\n",
    "- The red set shows the value-entropy correspondence for decision rule $ F_r $  \n",
    "- The blue set shows the value-entropy correspondence for decision rule $ F_b $  \n",
    "\n",
    "\n",
    "<img src=\"https://s3-ap-southeast-2.amazonaws.com/lectures.quantecon.org/jl/_static/figures/kg.png\" style=\"width:70%;height:70%\">\n",
    "\n",
    "  \n",
    "The blue correspondence is skinnier than the red correspondence\n",
    "\n",
    "This conveys the sense in which the decision rule $ F_b $ is *more robust* than the decision rule $ F_r $\n",
    "\n",
    "- *more robust* means that the set of values is less sensitive to *increasing misspecification* as measured by entropy  \n",
    "\n",
    "\n",
    "Notice that the less robust rule $ F_r $ promises higher values for small misspecifications (small entropy)\n",
    "\n",
    "(But it is more fragile in the sense that it is more sensitive to perturbations of the approximating model)\n",
    "\n",
    "Below we’ll explain in detail how to construct these sets of values for a given $ F $, but for now $ \\ldots $\n",
    "\n",
    "Here is a hint about the  *secret weapons* we’ll use to construct these sets\n",
    "\n",
    "> - We’ll use some min problems to construct the lower bounds  \n",
    "- We’ll use some max problems to construct the upper bounds  \n",
    "\n",
    "\n",
    "\n",
    "We will also describe how to choose $ F $ to shape the sets of values\n",
    "\n",
    "This will involve crafting a *skinnier* set at the cost of  a lower *level* (at least for low values of entropy)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Inspiring Video\n",
    "\n",
    "If you want to understand more about why one serious quantitative researcher is interested in this approach, we recommend [Lars Peter Hansen’s Nobel lecture](http://www.nobelprize.org/mediaplayer/index.php?id=1994)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Other References\n",
    "\n",
    "Our discussion in this lecture is based on\n",
    "\n",
    "- [[HS00]](https://lectures.quantecon.org/zreferences.html#hansensargent2000)  \n",
    "- [[HS08]](https://lectures.quantecon.org/zreferences.html#hansensargent2008)  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Setup"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "hide-output": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[32m\u001b[1mActivated\u001b[0m /home/qebuild/repos/lecture-source-jl/_build/website/jupyter/Project.toml\u001b[39m\n",
      "\u001b[36m\u001b[1mInfo\u001b[0m quantecon-notebooks-julia 0.1.0 activated, 0.2.0 requested\u001b[39m\n"
     ]
    }
   ],
   "source": [
    "using InstantiateFromURL\n",
    "github_project(\"QuantEcon/quantecon-notebooks-julia\", version = \"0.2.0\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "hide-output": false
   },
   "outputs": [],
   "source": [
    "using LinearAlgebra, Statistics"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## The Model\n",
    "\n",
    "For simplicity, we present ideas in the context of a class of problems with linear transition laws and quadratic objective functions\n",
    "\n",
    "To fit in with [our earlier lecture on LQ control](https://lectures.quantecon.org/lqcontrol.html), we will treat loss minimization rather than value maximization\n",
    "\n",
    "To begin, recall the [infinite horizon LQ problem](https://lectures.quantecon.org/lqcontrol.html#lq-ih), where an agent chooses a sequence of controls $ \\{u_t\\} $ to minimize\n",
    "\n",
    "\n",
    "<a id='equation-rob-sih'></a>\n",
    "$$\n",
    "\\sum_{t=0}^{\\infty} \\beta^t\n",
    "   \\left\\{\n",
    "       x_t' R x_t +  u_t' Q u_t\n",
    "   \\right\\} \\tag{1}\n",
    "$$\n",
    "\n",
    "subject to the linear law of motion\n",
    "\n",
    "\n",
    "<a id='equation-rob-lom'></a>\n",
    "$$\n",
    "x_{t+1} = A x_t + B u_t + C w_{t+1},\n",
    "\\qquad t = 0, 1, 2, \\ldots \\tag{2}\n",
    "$$\n",
    "\n",
    "As before,\n",
    "\n",
    "- $ x_t $ is $ n \\times 1 $, $ A $ is $ n \\times n $  \n",
    "- $ u_t $ is $ k \\times 1 $, $ B $ is $ n \\times k $  \n",
    "- $ w_t $ is $ j \\times 1 $, $ C $ is $ n \\times j $  \n",
    "- $ R $ is $ n \\times n $ and $ Q $ is $ k \\times k $  \n",
    "\n",
    "\n",
    "Here $ x_t $ is the state, $ u_t $ is the control,  and $ w_t $ is a shock vector.\n",
    "\n",
    "For now we take $ \\{ w_t \\} := \\{ w_t \\}_{t=1}^{\\infty} $ to be deterministic — a single fixed sequence\n",
    "\n",
    "We also allow for *model uncertainty* on the part of the agent solving this optimization problem\n",
    "\n",
    "In particular, the agent takes $ w_t = 0 $ for all $ t \\geq 0 $ as a benchmark model, but admits the possibility that this model might be wrong\n",
    "\n",
    "As a consequence, she also considers a set of alternative models expressed in terms of  sequences $ \\{ w_t \\} $ that are “close” to the zero sequence\n",
    "\n",
    "She seeks a policy that will do well enough for a set of alternative models whose members are pinned down by  sequences $ \\{ w_t \\} $\n",
    "\n",
    "Soon we’ll quantify the quality of a model specification in terms of the maximal size of the expression $ \\sum_{t=0}^{\\infty} \\beta^{t+1}w_{t+1}' w_{t+1} $"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Constructing More Robust Policies\n",
    "\n",
    "If our agent takes $ \\{ w_t \\} $ as a given deterministic sequence, then, drawing on intuition from earlier lectures on dynamic programming, we can anticipate Bellman equations such as\n",
    "\n",
    "$$\n",
    "J_{t-1} (x)\n",
    "= \\min_u\n",
    "\\{\n",
    "    x' R x + u' Q u + \\beta \\,\n",
    "    J_t (Ax + B u + C w_t)\n",
    "\\}\n",
    "$$\n",
    "\n",
    "(Here $ J $ depends on $ t $ because the sequence $ \\{w_t\\} $ is not recursive)\n",
    "\n",
    "Our tool for studying robustness is to construct a rule that works well even if an adverse sequence $ \\{ w_t \\} $ occurs\n",
    "\n",
    "In our framework, “adverse” means “loss increasing”\n",
    "\n",
    "As we’ll see, this will eventually lead us to construct the Bellman equation\n",
    "\n",
    "\n",
    "<a id='equation-rb-wcb0'></a>\n",
    "$$\n",
    "J (x) = \\min_u \\max_w \\{ x' R x + u' Q u + \\beta \\,[J (Ax + B u + C w) - \\theta w'w] \\} \\tag{3}\n",
    "$$\n",
    "\n",
    "Notice that we’ve added the penalty term $ - \\theta w'w $\n",
    "\n",
    "Since $ w'w = \\| w \\|^2 $, this term becomes influential when $ w $ moves away from the origin\n",
    "\n",
    "The penalty parameter $ \\theta $ controls how much we penalize the maximizing agent for “harming” the minimizing agent\n",
    "\n",
    "By raising $ \\theta $ more and more, we more and more limit the ability of maximizing agent to distort outcomes relative to the approximating model\n",
    "\n",
    "So bigger $ \\theta $ is implicitly associated with smaller distortion sequences $ \\{w_t \\} $"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Analyzing the Bellman equation\n",
    "\n",
    "So what does $ J $ in [(3)](#equation-rb-wcb0) look like?\n",
    "\n",
    "As with the [ordinary LQ control model](https://lectures.quantecon.org/lqcontrol.html), $ J $ takes the form $ J(x) = x' P x $ for some symmetric positive definite matrix $ P $\n",
    "\n",
    "One of our main tasks will be to analyze and compute the matrix $ P $\n",
    "\n",
    "Related tasks will be to study associated feedback rules for $ u_t $ and $ w_{t+1} $\n",
    "\n",
    "First, using [matrix calculus](https://lectures.quantecon.org/tools_and_techniques/linear_algebra.html#la-mcalc), you will be able to verify that\n",
    "\n",
    "\n",
    "<a id='equation-rb-mp0'></a>\n",
    "$$\n",
    "\\begin{aligned}\n",
    "\\max_w\n",
    "&\\{\n",
    "    (Ax + B u + C w)' P (Ax + B u + C w) - \\theta w'w\n",
    "\\}\n",
    "\\\\\n",
    "& \\hspace{20mm} = (Ax + Bu)' \\mathcal D(P) (Ax + Bu)\n",
    "\\end{aligned} \\tag{4}\n",
    "$$\n",
    "\n",
    "where\n",
    "\n",
    "\n",
    "<a id='equation-rb-d'></a>\n",
    "$$\n",
    "\\mathcal D(P) := P + PC (\\theta I - C' P C)^{-1} C' P \\tag{5}\n",
    "$$\n",
    "\n",
    "and $ I $ is a $ j \\times j $ identity matrix.  Substituting this expression for the maximum into [(3)](#equation-rb-wcb0) yields\n",
    "\n",
    "\n",
    "<a id='equation-rb-owb'></a>\n",
    "$$\n",
    "x'Px  =  \\min_u    \\{\n",
    "    x' R x + u' Q u + \\beta \\,\n",
    "   (Ax + Bu)' \\mathcal D(P) (Ax + Bu)    \\} \\tag{6}\n",
    "$$\n",
    "\n",
    "Using similar mathematics, the solution to this minimization problem is $ u  = - F x $ where $ F := (Q + \\beta B' \\mathcal D (P) B)^{-1} \\beta B' \\mathcal D(P) A $\n",
    "\n",
    "Substituting this minimizer back into [(6)](#equation-rb-owb) and working through the algebra gives\n",
    "$ x' P x = x' \\mathcal B ( \\mathcal D( P )) x $ for all $ x $, or, equivalently,\n",
    "\n",
    "$$\n",
    "P = \\mathcal B ( \\mathcal D( P ))\n",
    "$$\n",
    "\n",
    "where $ \\mathcal D $ is the operator defined in [(5)](#equation-rb-d) and\n",
    "\n",
    "$$\n",
    "\\mathcal B(P)\n",
    ":= R - \\beta^2 A' P B (Q + \\beta B' P B)^{-1} B' P A + \\beta A' P A\n",
    "$$\n",
    "\n",
    "The operator $ \\mathcal B $ is the standard (i.e., non-robust) LQ Bellman operator, and $ P = \\mathcal B(P) $ is the standard matrix Riccati equation coming from the\n",
    "Bellman equation — see [this discussion](https://lectures.quantecon.org/lqcontrol.html#lq-ih)\n",
    "\n",
    "Under some regularity conditions (see [[HS08]](https://lectures.quantecon.org/zreferences.html#hansensargent2008)), the operator $ \\mathcal B \\circ \\mathcal D $\n",
    "has a unique positive definite fixed point, which we denote below by $ \\hat P $\n",
    "\n",
    "A robust policy, indexed by $ \\theta $, is $ u  = - \\hat F x $ where\n",
    "\n",
    "\n",
    "<a id='equation-rb-oc-ih'></a>\n",
    "$$\n",
    "\\hat F\n",
    ":= (Q + \\beta B' \\mathcal D (\\hat P) B)^{-1} \\beta B' \\mathcal D(\\hat P) A \\tag{7}\n",
    "$$\n",
    "\n",
    "We also define\n",
    "\n",
    "\n",
    "<a id='equation-rb-kd'></a>\n",
    "$$\n",
    "\\hat K\n",
    ":= (\\theta I - C' \\hat PC)^{-1} C'  \\hat P (A - B \\hat F) \\tag{8}\n",
    "$$\n",
    "\n",
    "The interpretation of $ \\hat K $ is that $ w_{t+1} = \\hat K x_t $ on the worst-case path of $ \\{x_t \\} $, in the sense that this vector is the maximizer of [(4)](#equation-rb-mp0) evaluated at the fixed rule $ u  = - \\hat F x $\n",
    "\n",
    "Note that $ \\hat P, \\hat F, \\hat K $ are all determined by the\n",
    "primitives and $ \\theta $\n",
    "\n",
    "Note also that if $ \\theta $ is very large, then $ \\mathcal D $ is\n",
    "approximately equal to the identity mapping\n",
    "\n",
    "Hence, when $ \\theta $ is large,  $ \\hat P $ and $ \\hat F $ are approximately equal to their standard LQ values\n",
    "\n",
    "Furthermore, when $ \\theta $ is large, $ \\hat K $ is approximately equal to zero\n",
    "\n",
    "Conversely, smaller $ \\theta $ is associated with greater fear of model misspecification, and greater concern for robustness"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Robustness as Outcome of a Two-Person Zero-Sum Game\n",
    "\n",
    "What we have done above can be interpreted in terms of  a two-person zero-sum game in which $ \\hat F, \\hat K $ are Nash equilibrium objects\n",
    "\n",
    "Agent 1 is our original agent, who seeks to minimize loss in the LQ program while admitting the possibility of misspecification\n",
    "\n",
    "Agent 2 is an imaginary malevolent player\n",
    "\n",
    "Agent 2’s malevolence helps the original agent to compute bounds on his value function across a set of models\n",
    "\n",
    "We begin with agent 2’s problem\n",
    "\n",
    "\n",
    "<a id='rb-a2'></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Agent 2’s Problem\n",
    "\n",
    "Agent 2\n",
    "\n",
    "1. knows a fixed policy $ F $ specifying the behavior of agent 1, in the sense that $ u_t = - F x_t $ for all $ t $  \n",
    "1. responds by choosing a shock sequence $ \\{ w_t \\} $ from a set of paths sufficiently close to the benchmark sequence $ \\{0, 0, 0, \\ldots\\} $  \n",
    "\n",
    "\n",
    "A natural way to say “sufficiently close to the zero sequence” is to restrict the\n",
    "summed inner product $ \\sum_{t=1}^{\\infty} w_t' w_t $ to be small\n",
    "\n",
    "However, to obtain a time-invariant recursive formulation, it turns out to be convenient to\n",
    "restrict a  discounted inner product\n",
    "\n",
    "\n",
    "<a id='equation-rb-dec'></a>\n",
    "$$\n",
    "\\sum_{t=1}^{\\infty} \\beta^t w_t' w_t \\leq \\eta \\tag{9}\n",
    "$$\n",
    "\n",
    "Now let $ F $ be a fixed policy, and let $ J_F(x_0, \\mathbf w) $ be the\n",
    "present-value cost of that policy given sequence $ \\mathbf w := \\{w_t\\} $\n",
    "and initial condition $ x_0 \\in \\mathbb R^n $\n",
    "\n",
    "Substituting $ -F x_t $ for $ u_t $ in [(1)](#equation-rob-sih), this value can be written as\n",
    "\n",
    "\n",
    "<a id='equation-rob-fpv'></a>\n",
    "$$\n",
    "J_F(x_0, \\mathbf w) :=\n",
    "    \\sum_{t=0}^{\\infty} \\beta^t\n",
    "    x_t' (R + F' Q F) x_t \\tag{10}\n",
    "$$\n",
    "\n",
    "where\n",
    "\n",
    "\n",
    "<a id='equation-rob-lomf'></a>\n",
    "$$\n",
    "x_{t+1} = (A - B F) x_t + C w_{t+1} \\tag{11}\n",
    "$$\n",
    "\n",
    "and the initial condition $ x_0 $ is as specified in the left side of [(10)](#equation-rob-fpv)\n",
    "\n",
    "Agent 2 chooses $ {\\mathbf w} $ to maximize agent 1’s loss $ J_F(x_0, \\mathbf w) $ subject to [(9)](#equation-rb-dec)\n",
    "\n",
    "Using a Lagrangian formulation, we can express this problem as\n",
    "\n",
    "$$\n",
    "\\max_{\\mathbf w}\n",
    " \\sum_{t=0}^{\\infty} \\beta^t\n",
    "  \\left\\{\n",
    "    x_t' (R + F' Q F) x_t - \\beta \\theta (w_{t+1}' w_{t+1} - \\eta)\n",
    "  \\right\\}\n",
    "$$\n",
    "\n",
    "where $ \\{x_t\\} $ satisfied  [(11)](#equation-rob-lomf) and $ \\theta $ is a Lagrange multiplier on constraint [(9)](#equation-rb-dec)\n",
    "\n",
    "For the moment, let’s take $ \\theta $ as fixed, allowing us to drop the constant $ \\beta \\theta \\eta $ term in the objective function, and hence write the problem as\n",
    "\n",
    "$$\n",
    "\\max_{\\mathbf w}\n",
    " \\sum_{t=0}^{\\infty} \\beta^t\n",
    "  \\left\\{\n",
    "    x_t' (R + F' Q F) x_t - \\beta \\theta w_{t+1}' w_{t+1}\n",
    "  \\right\\}\n",
    "$$\n",
    "\n",
    "or, equivalently,\n",
    "\n",
    "\n",
    "<a id='equation-rb-a2o'></a>\n",
    "$$\n",
    "\\min_{\\mathbf w}\n",
    "\\sum_{t=0}^{\\infty} \\beta^t\n",
    "\\left\\{\n",
    "    -x_t' (R + F' Q F) x_t + \\beta \\theta w_{t+1}' w_{t+1}\n",
    "\\right\\} \\tag{12}\n",
    "$$\n",
    "\n",
    "subject to [(11)](#equation-rob-lomf)\n",
    "\n",
    "What’s striking about this optimization problem is that it is once again an LQ discounted dynamic programming problem, with $ \\mathbf w = \\{ w_t \\} $ as the sequence of controls\n",
    "\n",
    "The expression for the optimal policy can be found by applying the usual LQ formula ([see here](https://lectures.quantecon.org/lqcontrol.html#lq-ih))\n",
    "\n",
    "We denote it by $ K(F, \\theta) $, with the interpretation $ w_{t+1} = K(F, \\theta) x_t $\n",
    "\n",
    "The remaining step for agent 2’s problem is to set $ \\theta $ to enforce the constraint [(9)](#equation-rb-dec), which can be done by choosing $ \\theta = \\theta_{\\eta} $ such that\n",
    "\n",
    "\n",
    "<a id='equation-rb-pdt'></a>\n",
    "$$\n",
    "\\beta \\sum_{t=0}^{\\infty} \\beta^t x_t' K(F, \\theta_\\eta)' K(F, \\theta_\\eta) x_t = \\eta \\tag{13}\n",
    "$$\n",
    "\n",
    "Here $ x_t $ is given by [(11)](#equation-rob-lomf) — which in this case becomes $ x_{t+1} = (A - B F + CK(F, \\theta)) x_t $\n",
    "\n",
    "\n",
    "<a id='rb-a1'></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Using Agent 2’s Problem to Construct Bounds on the Value Sets"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### The Lower Bound\n",
    "\n",
    "Define the minimized object on the right side of problem [(12)](#equation-rb-a2o) as $ R_\\theta(x_0, F) $.\n",
    "\n",
    "Because “minimizers minimize” we have\n",
    "\n",
    "$$\n",
    "R_\\theta(x_0, F) \\leq \\sum_{t=0}^\\infty \\beta^t \\left\\{  - x_t' (R + F' Q F) x_t \\right\\} + \\beta \\theta \\sum_{t=0}^\\infty \\beta^t w_{t+1}' w_{t+1},\n",
    "$$\n",
    "\n",
    "where $ x_{t+1} = (A - B F + CK(F, \\theta)) x_t $ and $ x_0 $ is a given initial condition.\n",
    "\n",
    "This inequality in turn implies the inequality\n",
    "\n",
    "\n",
    "<a id='equation-rob-bound'></a>\n",
    "$$\n",
    "R_\\theta(x_0, F) - \\theta \\ {\\rm ent}\n",
    "\\leq \\sum_{t=0}^\\infty \\beta^t \\left\\{  - x_t' (R + F' Q F) x_t \\right\\} \\tag{14}\n",
    "$$\n",
    "\n",
    "where\n",
    "\n",
    "$$\n",
    "{\\rm ent} := \\beta \\sum_{t=0}^\\infty \\beta^t w_{t+1}' w_{t+1}\n",
    "$$\n",
    "\n",
    "The left side of inequality [(14)](#equation-rob-bound) is a straight line with slope $ -\\theta $\n",
    "\n",
    "Technically, it is  a “separating hyperplane”\n",
    "\n",
    "At a particular value of entropy, the line is tangent to the lower bound of values  as a function of entropy\n",
    "\n",
    "In particular, the lower bound on the left side of [(14)](#equation-rob-bound) is attained when\n",
    "\n",
    "\n",
    "<a id='equation-rb-pdt22'></a>\n",
    "$$\n",
    "{\\rm ent} =  \\beta \\sum_{t=0}^{\\infty} \\beta^t x_t' K(F, \\theta)' K(F, \\theta) x_t \\tag{15}\n",
    "$$\n",
    "\n",
    "To construct the *lower bound* on the set of values associated with all perturbations $ {\\mathbf w} $ satisfying the entropy constraint [(9)](#equation-rb-dec) at a  given entropy level, we proceed as follows:\n",
    "\n",
    "> - For a given $ \\theta $, solve the minimization problem [(12)](#equation-rb-a2o)  \n",
    "- Compute the minimizer $ R_\\theta(x_0, F) $ and the associated entropy using [(15)](#equation-rb-pdt22)  \n",
    "- Compute the lower bound on the value function $ R_\\theta(x_0, F) - \\theta \\ {\\rm  ent} $ and plot it against $ {\\rm ent} $  \n",
    "- Repeat the preceding three steps  for a range of values of $ \\theta $ to trace out the lower bound  \n",
    "\n",
    "\n",
    "\n",
    ">**Note**\n",
    ">\n",
    ">This procedure  sweeps  out a set of separating hyperplanes indexed by different values for  the Lagrange multiplier $ \\theta $"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### The Upper Bound\n",
    "\n",
    "To construct an *upper bound* we use a very similar procedure\n",
    "\n",
    "We simply replace the *minimization* problem [(12)](#equation-rb-a2o) with the *maximization* problem\n",
    "\n",
    "\n",
    "<a id='equation-rba2omax'></a>\n",
    "$$\n",
    "V_{\\tilde \\theta}(x_0, F) =  \\max_{\\mathbf w}\n",
    "  \\sum_{t=0}^{\\infty} \\beta^t\n",
    "  \\left\\{\n",
    "      -x_t' (R + F' Q F) x_t - \\beta \\tilde \\theta w_{t+1}' w_{t+1}\n",
    "  \\right\\} \\tag{16}\n",
    "$$\n",
    "\n",
    "where now $ \\tilde \\theta  >0 $  penalizes the choice of $ {\\mathbf w} $ with larger entropy.\n",
    "\n",
    "(Notice that $ \\tilde \\theta = - \\theta $ in problem [(12)](#equation-rb-a2o))\n",
    "\n",
    "Because “maximizers maximize” we have\n",
    "\n",
    "$$\n",
    "V_{\\tilde \\theta}(x_0, F) \\geq \\sum_{t=0}^\\infty \\beta^t \\left\\{  - x_t' (R + F' Q F) x_t \\right\\} - \\beta \\tilde \\theta \\sum_{t=0}^\\infty \\beta^t w_{t+1}' w_{t+1}\n",
    "$$\n",
    "\n",
    "which in turn implies the inequality\n",
    "\n",
    "\n",
    "<a id='equation-robboundmax'></a>\n",
    "$$\n",
    "V_{\\tilde \\theta}(x_0, F) + \\tilde \\theta \\ {\\rm ent} \\geq \\sum_{t=0}^\\infty \\beta^t \\left\\{  - x_t' (R + F' Q F) x_t \\right\\} \\tag{17}\n",
    "$$\n",
    "\n",
    "where\n",
    "\n",
    "$$\n",
    "{\\rm ent} \\equiv \\beta \\sum_{t=0}^\\infty \\beta^t w_{t+1}' w_{t+1}\n",
    "$$\n",
    "\n",
    "The left side of inequality [(17)](#equation-robboundmax) is a straight line with slope $ \\tilde \\theta $\n",
    "\n",
    "The upper bound on the left side of [(17)](#equation-robboundmax) is attained when\n",
    "\n",
    "\n",
    "<a id='equation-rbpdt223'></a>\n",
    "$$\n",
    "{\\rm ent} =  \\beta \\sum_{t=0}^{\\infty} \\beta^t x_t' K(F, \\tilde \\theta)' K(F, \\tilde \\theta) x_t \\tag{18}\n",
    "$$\n",
    "\n",
    "To construct the *upper bound* on the set of values associated all perturbations $ {\\mathbf w} $ with a given entropy we proceed much as we did for the lower bound\n",
    "\n",
    "> - For a given $ \\tilde \\theta $, solve the maximization problem [(16)](#equation-rba2omax)  \n",
    "- Compute the maximizer $ V_{\\tilde \\theta}(x_0, F) $ and the associated entropy using [(18)](#equation-rbpdt223)  \n",
    "- Compute the upper bound on the value function $ V_{\\tilde \\theta}(x_0, F) + \\tilde \\theta \\ {\\rm ent} $ and plot it against $ {\\rm ent} $  \n",
    "- Repeat the preceding three steps  for a range of values of $ \\tilde \\theta $ to trace out the upper bound  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Reshaping the set of values\n",
    "\n",
    "Now in the interest of *reshaping* these sets of values by choosing $ F $, we turn to agent 1’s problem"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Agent 1’s Problem\n",
    "\n",
    "Now we turn to agent 1, who solves\n",
    "\n",
    "\n",
    "<a id='equation-rob-sihk'></a>\n",
    "$$\n",
    "\\min_{\\{u_t\\}}\n",
    "    \\sum_{t=0}^{\\infty} \\beta^t\n",
    "    \\left\\{\n",
    "        x_t' R x_t + u_t' Q u_t - \\beta \\theta w_{t+1}' w_{t+1}\n",
    "    \\right\\} \\tag{19}\n",
    "$$\n",
    "\n",
    "where  $ \\{ w_{t+1} \\} $  satisfies $ w_{t+1} = K x_t $\n",
    "\n",
    "In other words, agent 1 minimizes\n",
    "\n",
    "\n",
    "<a id='equation-rob-sihk2'></a>\n",
    "$$\n",
    "\\sum_{t=0}^{\\infty} \\beta^t\n",
    "    \\left\\{\n",
    "        x_t' (R - \\beta \\theta K' K ) x_t + u_t' Q u_t\n",
    "    \\right\\} \\tag{20}\n",
    "$$\n",
    "\n",
    "subject to\n",
    "\n",
    "\n",
    "<a id='equation-rob-lomk'></a>\n",
    "$$\n",
    "x_{t+1} = (A + C K) x_t + B u_t \\tag{21}\n",
    "$$\n",
    "\n",
    "Once again, the expression for the optimal policy can be found [here](https://lectures.quantecon.org/lqcontrol.html#lq-ih) — we denote\n",
    "it by $ \\tilde F $\n",
    "\n",
    "\n",
    "<a id='rb-eq'></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Nash Equilibrium\n",
    "\n",
    "Clearly the $ \\tilde F $ we have obtained depends on $ K $, which, in agent 2’s problem,\n",
    "depended on an initial policy $ F $\n",
    "\n",
    "Holding all other parameters fixed, we can represent this relationship as a mapping $ \\Phi $, where\n",
    "\n",
    "$$\n",
    "\\tilde F = \\Phi(K(F, \\theta))\n",
    "$$\n",
    "\n",
    "The map $ F \\mapsto \\Phi (K(F, \\theta)) $ corresponds to a situation in which\n",
    "\n",
    "1. agent 1 uses an arbitrary initial policy $ F $  \n",
    "1. agent 2 best responds to agent 1 by choosing $ K(F, \\theta) $  \n",
    "1. agent 1 best responds to agent 2 by choosing $ \\tilde F = \\Phi (K(F, \\theta)) $  \n",
    "\n",
    "\n",
    "As you may have already guessed, the robust policy $ \\hat F $ defined in [(7)](#equation-rb-oc-ih) is a fixed point of the mapping $ \\Phi $\n",
    "\n",
    "In particular, for any given $ \\theta $,\n",
    "\n",
    "1. $ K(\\hat F, \\theta) = \\hat K $, where $ \\hat K $ is as given in [(8)](#equation-rb-kd)  \n",
    "1. $ \\Phi(\\hat K) = \\hat F $  \n",
    "\n",
    "\n",
    "A sketch of the proof is given in [the appendix](#rb-appendix)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## The Stochastic Case\n",
    "\n",
    "Now we turn to the stochastic case, where the sequence $ \\{w_t\\} $ is treated as an iid sequence of random vectors\n",
    "\n",
    "In this setting, we suppose that our agent is uncertain about the *conditional probability distribution* of $ w_{t+1} $\n",
    "\n",
    "The agent takes the standard normal distribution $ N(0, I) $ as the baseline conditional distribution, while admitting the possibility that other “nearby” distributions prevail\n",
    "\n",
    "These alternative conditional distributions of $ w_{t+1} $ might depend nonlinearly on the history  $ x_s, s \\leq t $\n",
    "\n",
    "To implement this idea, we need a notion of what it means for one distribution\n",
    "to be near another one\n",
    "\n",
    "Here we adopt a very useful measure of closeness for distributions known as the *relative entropy*, or [Kullback-Leibler divergence](https://en.wikipedia.org/wiki/Kullback%E2%80%93Leibler_divergence)\n",
    "\n",
    "For densities $ p, q $, the Kullback-Leibler divergence of $ q $ from $ p $ is defined as\n",
    "\n",
    "$$\n",
    "D_{KL} (p, q) := \\int \\ln \\left[ \\frac{p(x)}{q(x)} \\right] p(x) \\, dx\n",
    "$$\n",
    "\n",
    "Using this notation, we replace [(3)](#equation-rb-wcb0) with the stochastic analogue\n",
    "\n",
    "\n",
    "<a id='equation-rb-wcb1'></a>\n",
    "$$\n",
    "J (x) =\n",
    "\\min_u\n",
    "\\max_{\\psi \\in \\mathcal P}\n",
    "\\left\\{\n",
    "    x' R x + u' Q u + \\beta \\,\n",
    "    \\left[\n",
    "    \\int J (Ax + B u + C w) \\, \\psi(dw) - \\theta D_{KL}(\\psi, \\phi)\n",
    "    \\right]\n",
    "\\right\\} \\tag{22}\n",
    "$$\n",
    "\n",
    "Here $ \\mathcal P $ represents the set of all densities on $ \\mathbb R^n $ and $ \\phi $ is the benchmark distribution $ N(0, I) $\n",
    "\n",
    "The distribution $ \\phi $ is chosen as the least desirable conditional distribution in terms of next period outcomes, while taking into account the penalty term $ \\theta D_{KL}(\\psi, \\phi) $\n",
    "\n",
    "This penalty term plays a role analogous to the one played by the deterministic penalty $ \\theta w'w $ in [(3)](#equation-rb-wcb0), since it discourages large deviations from the benchmark"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Solving the Model\n",
    "\n",
    "The maximization problem in [(22)](#equation-rb-wcb1) appears highly nontrivial — after all,\n",
    "we are maximizing over an infinite dimensional space consisting of the entire set of densities\n",
    "\n",
    "However, it turns out that the solution is  tractable, and in fact also falls within the class of normal distributions\n",
    "\n",
    "First, we note that $ J $ has the form $ J(x) = x' P x + d $ for some positive definite matrix $ P $ and constant real number $ d $\n",
    "\n",
    "Moreover, it turns out that if $ (I - \\theta^{-1} C' P C)^{-1} $ is nonsingular, then\n",
    "\n",
    "\n",
    "<a id='equation-rb-mls'></a>\n",
    "$$\n",
    "\\begin{aligned}\n",
    "\\max_{\\psi \\in \\mathcal P}\n",
    "&\\left\\{\n",
    "    \\int (Ax + B u + C w)' P (Ax + B u + C w) \\, \\psi(dw) - \\theta D_{KL}(\\psi, \\phi)\n",
    "\\right\\}\n",
    "\\\\\n",
    "& \\hspace{20mm} = (Ax + Bu)' \\mathcal D (P) (Ax + Bu) + \\kappa(\\theta, P)\n",
    "\\end{aligned} \\tag{23}\n",
    "$$\n",
    "\n",
    "where\n",
    "\n",
    "$$\n",
    "\\kappa(\\theta, P) := \\theta \\ln [ \\det(I - \\theta^{-1} C' P C)^{-1} ]\n",
    "$$\n",
    "\n",
    "and the maximizer is the Gaussian distribution\n",
    "\n",
    "\n",
    "<a id='equation-rb-md'></a>\n",
    "$$\n",
    "\\psi = N\n",
    "\\left(\n",
    "    (\\theta I -  C' P C)^{-1} C' P (Ax + Bu), (I - \\theta^{-1} C' P C)^{-1}\n",
    "\\right) \\tag{24}\n",
    "$$\n",
    "\n",
    "Substituting the expression for the maximum into  Bellman equation\n",
    "[(22)](#equation-rb-wcb1) and using $ J(x) = x'Px + d $ gives\n",
    "\n",
    "\n",
    "<a id='equation-rb-wcb2'></a>\n",
    "$$\n",
    "x' P x + d =\n",
    "\\min_u\n",
    "\\left\\{\n",
    "    x' R x + u' Q u + \\beta \\,\n",
    "    (Ax + Bu)' \\mathcal D (P) (Ax + Bu) + \\beta \\, [d + \\kappa(\\theta, P)]\n",
    "\\right\\} \\tag{25}\n",
    "$$\n",
    "\n",
    "Since constant terms do not affect minimizers, the solution is the same as\n",
    "[(6)](#equation-rb-owb), leading to\n",
    "\n",
    "$$\n",
    "x' P x + d\n",
    "= x' \\mathcal B( \\mathcal D(P)) x + \\beta \\, [d + \\kappa(\\theta, P)]\n",
    "$$\n",
    "\n",
    "To solve this Bellman equation, we take $ \\hat P $ to be the positive definite fixed point of\n",
    "$ \\mathcal B \\circ \\mathcal D $\n",
    "\n",
    "In addition, we take $ \\hat d $ as the real number solving $ d = \\beta \\, [d + \\kappa(\\theta, P)] $, which is\n",
    "\n",
    "\n",
    "<a id='equation-rb-hatd'></a>\n",
    "$$\n",
    "\\hat d := \\frac{\\beta}{1 - \\beta} \\kappa(\\theta, P) \\tag{26}\n",
    "$$\n",
    "\n",
    "The robust policy in this stochastic case is the minimizer in\n",
    "[(25)](#equation-rb-wcb2), which is once again $ u  = - \\hat F x $ for $ \\hat F $ given by [(7)](#equation-rb-oc-ih)\n",
    "\n",
    "Substituting the robust policy into [(24)](#equation-rb-md) we obtain the worst case shock\n",
    "distribution:\n",
    "\n",
    "$$\n",
    "w_{t+1} \\sim\n",
    "N( \\hat K x_t, (I - \\theta^{-1} C' \\hat PC)^{-1} )\n",
    "$$\n",
    "\n",
    "where $ \\hat K $ is given by [(8)](#equation-rb-kd)\n",
    "\n",
    "Note that the mean of the worst-case shock distribution is equal to the same  worst-case $ w_{t+1} $ as in the earlier deterministic setting\n",
    "\n",
    "\n",
    "<a id='rb-coq'></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Computing Other Quantities\n",
    "\n",
    "Before turning to implementation, we briefly outline how to compute several other quantities of interest"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Worst-Case Value of a Policy\n",
    "\n",
    "One thing we will be interested in doing is holding a policy fixed and\n",
    "computing the discounted loss associated with that policy\n",
    "\n",
    "So let $ F $ be a given policy and let $ J_F(x) $ be\n",
    "the associated loss, which, by analogy with [(22)](#equation-rb-wcb1), satisfies\n",
    "\n",
    "$$\n",
    "J_F(x)\n",
    "=\n",
    "\\max_{\\psi \\in \\mathcal P}\n",
    "\\left\\{\n",
    "    x' (R + F'QF) x + \\beta \\,\n",
    "    \\left[\n",
    "    \\int J_F ( (A - BF)x + C w) \\, \\psi(dw) - \\theta D_{KL}(\\psi, \\phi)\n",
    "    \\right]\n",
    "\\right\\}\n",
    "$$\n",
    "\n",
    "Writing $ J_F(x) = x'P_Fx + d_F $ and applying the same argument used to\n",
    "derive [(23)](#equation-rb-mls) we get\n",
    "\n",
    "$$\n",
    "x' P_F x + d_F\n",
    "=\n",
    "x' (R + F'QF) x + \\beta \\,\n",
    "\\left[\n",
    " x' (A - BF)' \\mathcal D(P_F) (A - BF) x + d_F + \\kappa(\\theta, P_F)\n",
    "\\right]\n",
    "$$\n",
    "\n",
    "To solve this we take $ P_F $ to be the fixed point\n",
    "\n",
    "$$\n",
    "P_F = R + F'QF + \\beta (A - BF)' \\mathcal D(P_F) (A - BF)\n",
    "$$\n",
    "\n",
    "and\n",
    "\n",
    "\n",
    "<a id='equation-rb-hatd2'></a>\n",
    "$$\n",
    "d_F\n",
    ":= \\frac{\\beta}{1 - \\beta} \\kappa(\\theta, P_F)\n",
    "= \\frac{\\beta}{1 - \\beta}\n",
    " \\theta \\ln [ \\det(I - \\theta^{-1} C' P_F C)^{-1} ] \\tag{27}\n",
    "$$\n",
    "\n",
    "If you skip ahead to [the appendix](#rb-appendix), you will be able to\n",
    "verify that $ -P_F $ is the solution to the Bellman equation in\n",
    "agent 2’s problem [discussed above](#rb-a2)  — we use this in our\n",
    "computations"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Implementation\n",
    "\n",
    "The [QuantEcon.jl](http://quantecon.org/julia_index.html) package provides a type called `RBLQ` for implementation of robust LQ optimal control\n",
    "\n",
    "The code can be found [on GitHub](https://github.com/QuantEcon/QuantEcon.jl/blob/master/src/robustlq.jl)\n",
    "\n",
    "Here is a brief description of the methods of the type\n",
    "\n",
    "- `d_operator()` and `b_operator()` implement $ \\mathcal D $\n",
    "  and $ \\mathcal B $ respectively  \n",
    "- `robust_rule()` and `robust_rule_simple()` both solve for the\n",
    "  triple $ \\hat F, \\hat K, \\hat P $, as described in equations [(7)](#equation-rb-oc-ih) – [(8)](#equation-rb-kd) and the surrounding discussion  \n",
    "  \n",
    "  - `robust_rule()` is more efficient  \n",
    "  - `robust_rule_simple()` is more transparent and easier to follow  \n",
    "  \n",
    "- `K_to_F()` and `F_to_K()` solve the decision problems\n",
    "  of [agent 1](#rb-a1) and [agent 2](#rb-a2) respectively  \n",
    "- `compute_deterministic_entropy()` computes the left-hand side of [(13)](#equation-rb-pdt)  \n",
    "- `evaluate_F()` computes the loss and entropy associated with a given\n",
    "  policy — see [this discussion](#rb-coq)  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Application\n",
    "\n",
    "Let us consider a monopolist similar to [this one](https://lectures.quantecon.org/lqcontrol.html#lqc-mwac), but now facing model uncertainty\n",
    "\n",
    "The inverse demand function is $ p_t = a_0 - a_1 y_t + d_t $\n",
    "\n",
    "where\n",
    "\n",
    "$$\n",
    "d_{t+1} = \\rho d_t + \\sigma_d w_{t+1},\n",
    "\\quad \\{w_t\\} \\stackrel{\\textrm{iid}}{\\sim} N(0,1)\n",
    "$$\n",
    "\n",
    "and all parameters are strictly positive\n",
    "\n",
    "The period return function for the monopolist is\n",
    "\n",
    "$$\n",
    "r_t =  p_t y_t - \\gamma \\frac{(y_{t+1} - y_t)^2}{2} - c y_t\n",
    "$$\n",
    "\n",
    "Its objective is to maximize expected discounted profits, or, equivalently, to minimize $ \\mathbb E \\sum_{t=0}^\\infty \\beta^t (- r_t) $\n",
    "\n",
    "To form a linear regulator problem, we take the state and control to be\n",
    "\n",
    "$$\n",
    "x_t\n",
    "= \\begin{bmatrix}\n",
    "      1 \\\\ y_t \\\\ d_t\n",
    "  \\end{bmatrix}\n",
    "  \\quad \\text{and} \\quad\n",
    "  u_t = y_{t+1} - y_t\n",
    "$$\n",
    "\n",
    "Setting $ b := (a_0 - c) / 2 $ we define\n",
    "\n",
    "$$\n",
    "R\n",
    "= -\n",
    "\\begin{bmatrix}\n",
    "   0 &  b & 0 \\\\\n",
    "   b & -a_1 & 1/2 \\\\\n",
    "   0 & 1/2 & 0\n",
    "\\end{bmatrix}\n",
    "\\quad \\text{and} \\quad\n",
    "Q = \\gamma / 2\n",
    "$$\n",
    "\n",
    "For the transition matrices we set\n",
    "\n",
    "$$\n",
    "A\n",
    "= \\begin{bmatrix}\n",
    "   1 & 0 & 0 \\\\\n",
    "   0 & 1 & 0 \\\\\n",
    "   0 & 0 & \\rho\n",
    "  \\end{bmatrix},\n",
    "\\qquad\n",
    "B\n",
    "= \\begin{bmatrix}\n",
    "   0 \\\\\n",
    "   1 \\\\\n",
    "   0\n",
    "  \\end{bmatrix},\n",
    "\\qquad\n",
    "C\n",
    "= \\begin{bmatrix}\n",
    "   0 \\\\\n",
    "   0 \\\\\n",
    "   \\sigma_d\n",
    "  \\end{bmatrix}\n",
    "$$\n",
    "\n",
    "Our aim is to compute the value-entropy correspondences [shown above](#rb-vec)\n",
    "\n",
    "The parameters are\n",
    "\n",
    "$$\n",
    "a_0 = 100, a_1 = 0.5, \\rho = 0.9, \\sigma_d = 0.05, \\beta = 0.95, c = 2, \\gamma = 50.0\n",
    "$$\n",
    "\n",
    "The standard normal distribution for $ w_t $ is understood as the\n",
    "agent’s baseline, with uncertainty parameterized by $ \\theta $\n",
    "\n",
    "We compute value-entropy correspondences for two policies\n",
    "\n",
    "1. The no concern for robustness policy $ F_0 $, which is the ordinary LQ\n",
    "  loss minimizer  \n",
    "1. A “moderate” concern for robustness policy $ F_b $, with $ \\theta = 0.02 $  \n",
    "\n",
    "\n",
    "The code for producing the graph shown above, with blue being for the robust policy, is as follows"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "hide-output": false
   },
   "outputs": [
    {
     "data": {
      "image/svg+xml": [
       "<?xml version=\"1.0\" encoding=\"utf-8\"?>\n",
       "<svg xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\" width=\"600\" height=\"400\" viewBox=\"0 0 2400 1600\">\n",
       "<defs>\n",
       "  <clipPath id=\"clip2600\">\n",
       "    <rect x=\"0\" y=\"0\" width=\"2400\" height=\"1600\"/>\n",
       "  </clipPath>\n",
       "</defs>\n",
       "<polygon clip-path=\"url(#clip2600)\" points=\"\n",
       "0,1600 2400,1600 2400,0 0,0 \n",
       "  \" fill=\"#ffffff\" fill-rule=\"evenodd\" fill-opacity=\"1\"/>\n",
       "<defs>\n",
       "  <clipPath id=\"clip2601\">\n",
       "    <rect x=\"480\" y=\"0\" width=\"1681\" height=\"1600\"/>\n",
       "  </clipPath>\n",
       "</defs>\n",
       "<polygon clip-path=\"url(#clip2600)\" points=\"\n",
       "363.188,1425.62 2352.76,1425.62 2352.76,47.2441 363.188,47.2441 \n",
       "  \" fill=\"#ffffff\" fill-rule=\"evenodd\" fill-opacity=\"1\"/>\n",
       "<defs>\n",
       "  <clipPath id=\"clip2602\">\n",
       "    <rect x=\"363\" y=\"47\" width=\"1991\" height=\"1379\"/>\n",
       "  </clipPath>\n",
       "</defs>\n",
       "<polyline clip-path=\"url(#clip2602)\" style=\"stroke:#000000; stroke-width:2; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  419.496,1425.62 419.496,47.2441 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip2602)\" style=\"stroke:#000000; stroke-width:2; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  1006.04,1425.62 1006.04,47.2441 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip2602)\" style=\"stroke:#000000; stroke-width:2; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  1592.59,1425.62 1592.59,47.2441 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip2602)\" style=\"stroke:#000000; stroke-width:2; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  2179.14,1425.62 2179.14,47.2441 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip2602)\" style=\"stroke:#000000; stroke-width:2; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  363.188,1309.98 2352.76,1309.98 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip2602)\" style=\"stroke:#000000; stroke-width:2; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  363.188,1051.94 2352.76,1051.94 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip2602)\" style=\"stroke:#000000; stroke-width:2; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  363.188,793.907 2352.76,793.907 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip2602)\" style=\"stroke:#000000; stroke-width:2; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  363.188,535.872 2352.76,535.872 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip2602)\" style=\"stroke:#000000; stroke-width:2; stroke-opacity:0.1; fill:none\" points=\"\n",
       "  363.188,277.838 2352.76,277.838 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip2600)\" style=\"stroke:#000000; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  363.188,1425.62 2352.76,1425.62 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip2600)\" style=\"stroke:#000000; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  363.188,1425.62 363.188,47.2441 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip2600)\" style=\"stroke:#000000; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  419.496,1425.62 419.496,1404.94 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip2600)\" style=\"stroke:#000000; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  1006.04,1425.62 1006.04,1404.94 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip2600)\" style=\"stroke:#000000; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  1592.59,1425.62 1592.59,1404.94 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip2600)\" style=\"stroke:#000000; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  2179.14,1425.62 2179.14,1404.94 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip2600)\" style=\"stroke:#000000; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  363.188,1309.98 393.031,1309.98 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip2600)\" style=\"stroke:#000000; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  363.188,1051.94 393.031,1051.94 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip2600)\" style=\"stroke:#000000; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  363.188,793.907 393.031,793.907 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip2600)\" style=\"stroke:#000000; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  363.188,535.872 393.031,535.872 \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip2600)\" style=\"stroke:#000000; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  363.188,277.838 393.031,277.838 \n",
       "  \"/>\n",
       "<g clip-path=\"url(#clip2600)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:48px; text-anchor:middle;\" transform=\"rotate(0, 419.496, 1479.62)\" x=\"419.496\" y=\"1479.62\">0</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip2600)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:48px; text-anchor:start;\" transform=\"rotate(0, 920.901, 1500.63)\" x=\"920.901\" y=\"1500.63\">5.0×10</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip2600)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:38px; text-anchor:start;\" transform=\"rotate(0, 1069.44, 1473.22)\" x=\"1069.44\" y=\"1473.22\">5</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip2600)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:48px; text-anchor:start;\" transform=\"rotate(0, 1507.45, 1500.63)\" x=\"1507.45\" y=\"1500.63\">1.0×10</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip2600)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:38px; text-anchor:start;\" transform=\"rotate(0, 1655.99, 1473.22)\" x=\"1655.99\" y=\"1473.22\">6</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip2600)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:48px; text-anchor:start;\" transform=\"rotate(0, 2094, 1500.63)\" x=\"2094\" y=\"1500.63\">1.5×10</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip2600)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:38px; text-anchor:start;\" transform=\"rotate(0, 2242.54, 1473.22)\" x=\"2242.54\" y=\"1473.22\">6</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip2600)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:48px; text-anchor:end;\" transform=\"rotate(0, 339.188, 1327.48)\" x=\"339.188\" y=\"1327.48\">0</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip2600)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:48px; text-anchor:start;\" transform=\"rotate(0, 168.903, 1075.67)\" x=\"168.903\" y=\"1075.67\">3.0×10</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip2600)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:38px; text-anchor:start;\" transform=\"rotate(0, 317.443, 1048.26)\" x=\"317.443\" y=\"1048.26\">4</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip2600)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:48px; text-anchor:start;\" transform=\"rotate(0, 168.903, 817.634)\" x=\"168.903\" y=\"817.634\">6.0×10</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip2600)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:38px; text-anchor:start;\" transform=\"rotate(0, 317.443, 790.224)\" x=\"317.443\" y=\"790.224\">4</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip2600)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:48px; text-anchor:start;\" transform=\"rotate(0, 168.903, 559.6)\" x=\"168.903\" y=\"559.6\">9.0×10</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip2600)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:38px; text-anchor:start;\" transform=\"rotate(0, 317.443, 532.189)\" x=\"317.443\" y=\"532.189\">4</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip2600)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:48px; text-anchor:start;\" transform=\"rotate(0, 168.903, 301.565)\" x=\"168.903\" y=\"301.565\">1.2×10</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip2600)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:38px; text-anchor:start;\" transform=\"rotate(0, 317.443, 274.155)\" x=\"317.443\" y=\"274.155\">5</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip2600)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:66px; text-anchor:middle;\" transform=\"rotate(0, 1357.97, 1559.48)\" x=\"1357.97\" y=\"1559.48\">Entropy</text>\n",
       "</g>\n",
       "<g clip-path=\"url(#clip2600)\">\n",
       "<text style=\"fill:#000000; fill-opacity:1; font-family:Arial,Helvetica Neue,Helvetica,sans-serif; font-size:66px; text-anchor:middle;\" transform=\"rotate(-90, 89.2861, 736.431)\" x=\"89.2861\" y=\"736.431\">Value</text>\n",
       "</g>\n",
       "<polyline clip-path=\"url(#clip2602)\" style=\"stroke:#ff0000; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  419.496,751.757 438.455,685.672 457.414,658.359 476.373,634.585 495.333,618.273 514.292,601.961 533.251,586.07 552.21,574.161 571.169,562.252 590.128,550.343 \n",
       "  609.087,538.434 628.046,527.953 647.005,518.859 665.964,509.766 684.924,500.673 703.883,491.58 722.842,482.487 741.801,473.394 760.76,465.548 779.719,458.415 \n",
       "  798.678,451.283 817.637,444.15 836.596,437.017 855.556,429.885 874.515,422.752 893.474,415.619 912.433,408.487 931.392,401.354 950.351,394.96 969.31,389.274 \n",
       "  988.269,383.589 1007.23,377.904 1026.19,372.219 1045.15,366.534 1064.11,360.848 1083.06,355.163 1102.02,349.478 1120.98,343.793 1139.94,338.108 1158.9,332.422 \n",
       "  1177.86,326.737 1196.82,321.052 1215.78,315.367 1234.74,310.38 1253.7,305.811 1272.66,301.242 1291.61,296.673 1310.57,292.104 1329.53,287.535 1348.49,282.966 \n",
       "  1367.45,278.397 1386.41,273.828 1405.37,269.259 1424.33,264.69 1443.29,260.121 1462.25,255.552 1481.21,250.982 1500.17,246.413 1519.12,241.844 1538.08,237.275 \n",
       "  1557.04,232.706 1576,228.137 1594.96,223.568 1613.92,218.999 1632.88,214.43 1651.84,210.373 1670.8,206.695 1689.76,203.016 1708.72,199.337 1727.67,195.658 \n",
       "  1746.63,191.98 1765.59,188.301 1784.55,184.622 1803.51,180.944 1822.47,177.265 1841.43,173.586 1860.39,169.908 1879.35,166.229 1898.31,162.55 1917.27,158.872 \n",
       "  1936.22,155.193 1955.18,151.514 1974.14,147.836 1993.1,144.157 2012.06,140.478 2031.02,136.799 2049.98,133.121 2068.94,129.442 2087.9,125.763 2106.86,122.085 \n",
       "  2125.82,118.406 2144.77,114.727 2163.73,111.049 2182.69,107.37 2201.65,103.691 2220.61,100.013 2239.57,96.3339 2258.53,92.6552 2277.49,89.2032 2296.45,86.2547 \n",
       "  \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip2602)\" style=\"stroke:#ff0000; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  419.496,751.757 438.455,818.574 457.414,846.883 476.373,868.673 495.333,887.077 514.292,902.98 533.251,917.287 552.21,930.609 571.169,943.101 590.128,954.501 \n",
       "  609.087,965.439 628.046,975.94 647.005,985.734 665.964,995.234 684.924,1004.44 703.883,1013.15 722.842,1021.65 741.801,1029.93 760.76,1037.84 779.719,1045.59 \n",
       "  798.678,1053.17 817.637,1060.47 836.596,1067.62 855.556,1074.63 874.515,1081.45 893.474,1088.11 912.433,1094.65 931.392,1101.08 950.351,1107.33 969.31,1113.48 \n",
       "  988.269,1119.52 1007.23,1125.46 1026.19,1131.28 1045.15,1137 1064.11,1142.63 1083.06,1148.17 1102.02,1153.62 1120.98,1158.99 1139.94,1164.27 1158.9,1169.47 \n",
       "  1177.86,1174.6 1196.82,1179.65 1215.78,1184.63 1234.74,1189.54 1253.7,1194.39 1272.66,1199.17 1291.61,1203.89 1310.57,1208.54 1329.53,1213.14 1348.49,1217.68 \n",
       "  1367.45,1222.16 1386.41,1226.58 1405.37,1230.95 1424.33,1235.27 1443.29,1239.54 1462.25,1243.75 1481.21,1247.91 1500.17,1252.03 1519.12,1256.1 1538.08,1260.12 \n",
       "  1557.04,1264.09 1576,1268.02 1594.96,1271.9 1613.92,1275.74 1632.88,1279.53 1651.84,1283.28 1670.8,1286.99 1689.76,1290.65 1708.72,1294.28 1727.67,1297.86 \n",
       "  1746.63,1301.4 1765.59,1304.9 1784.55,1308.36 1803.51,1311.78 1822.47,1315.16 1841.43,1318.5 1860.39,1321.81 1879.35,1325.07 1898.31,1328.29 1917.27,1331.47 \n",
       "  1936.22,1334.62 1955.18,1337.72 1974.14,1340.78 1993.1,1343.81 2012.06,1346.79 2031.02,1349.74 2049.98,1352.65 2068.94,1355.51 2087.9,1358.33 2106.86,1361.12 \n",
       "  2125.82,1363.86 2144.77,1366.56 2163.73,1369.22 2182.69,1371.83 2201.65,1374.41 2220.61,1376.94 2239.57,1379.42 2258.53,1381.86 2277.49,1384.26 2296.45,1386.61 \n",
       "  \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip2602)\" style=\"stroke:#0000ff; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  419.496,1134.6 438.455,1116.87 457.414,1109.57 476.373,1103.93 495.333,1099.18 514.292,1094.99 533.251,1091.21 552.21,1087.76 571.169,1084.55 590.128,1081.53 \n",
       "  609.087,1078.65 628.046,1075.94 647.005,1073.35 665.964,1070.83 684.924,1068.47 703.883,1066.13 722.842,1063.93 741.801,1061.75 760.76,1059.66 779.719,1057.62 \n",
       "  798.678,1055.61 817.637,1053.71 836.596,1051.81 855.556,1049.94 874.515,1048.17 893.474,1046.39 912.433,1044.63 931.392,1042.97 950.351,1041.31 969.31,1039.64 \n",
       "  988.269,1038.04 1007.23,1036.49 1026.19,1034.93 1045.15,1033.37 1064.11,1031.87 1083.06,1030.42 1102.02,1028.96 1120.98,1027.5 1139.94,1026.05 1158.9,1024.68 \n",
       "  1177.86,1023.31 1196.82,1021.95 1215.78,1020.58 1234.74,1019.22 1253.7,1017.94 1272.66,1016.65 1291.61,1015.37 1310.57,1014.09 1329.53,1012.81 1348.49,1011.56 \n",
       "  1367.45,1010.35 1386.41,1009.15 1405.37,1007.95 1424.33,1006.75 1443.29,1005.55 1462.25,1004.34 1481.21,1003.21 1500.17,1002.09 1519.12,1000.96 1538.08,999.832 \n",
       "  1557.04,998.705 1576,997.578 1594.96,996.45 1613.92,995.374 1632.88,994.317 1651.84,993.26 1670.8,992.204 1689.76,991.147 1708.72,990.09 1727.67,989.034 \n",
       "  1746.63,987.977 1765.59,986.952 1784.55,985.962 1803.51,984.972 1822.47,983.982 1841.43,982.992 1860.39,982.002 1879.35,981.012 1898.31,980.023 1917.27,979.033 \n",
       "  1936.22,978.048 1955.18,977.121 1974.14,976.194 1993.1,975.267 2012.06,974.341 2031.02,973.414 2049.98,972.487 2068.94,971.56 2087.9,970.633 2106.86,969.706 \n",
       "  2125.82,968.78 2144.77,967.875 2163.73,967.008 2182.69,966.141 2201.65,965.274 2220.61,964.406 2239.57,963.539 2258.53,962.672 2277.49,961.805 2296.45,960.938 \n",
       "  \n",
       "  \"/>\n",
       "<polyline clip-path=\"url(#clip2602)\" style=\"stroke:#0000ff; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  419.496,1134.6 438.455,1152.32 457.414,1159.67 476.373,1165.29 495.333,1170.03 514.292,1174.2 533.251,1177.96 552.21,1181.42 571.169,1184.63 590.128,1187.64 \n",
       "  609.087,1190.48 628.046,1193.18 647.005,1195.75 665.964,1198.22 684.924,1200.58 703.883,1202.86 722.842,1205.06 741.801,1207.19 760.76,1209.25 779.719,1211.25 \n",
       "  798.678,1213.2 817.637,1215.09 836.596,1216.93 855.556,1218.73 874.515,1220.48 893.474,1222.2 912.433,1223.87 931.392,1225.51 950.351,1227.11 969.31,1228.68 \n",
       "  988.269,1230.22 1007.23,1231.73 1026.19,1233.21 1045.15,1234.66 1064.11,1236.08 1083.06,1237.48 1102.02,1238.86 1120.98,1240.21 1139.94,1241.53 1158.9,1242.84 \n",
       "  1177.86,1244.12 1196.82,1245.38 1215.78,1246.62 1234.74,1247.84 1253.7,1249.05 1272.66,1250.23 1291.61,1251.39 1310.57,1252.54 1329.53,1253.67 1348.49,1254.78 \n",
       "  1367.45,1255.87 1386.41,1256.95 1405.37,1258.01 1424.33,1259.05 1443.29,1260.08 1462.25,1261.09 1481.21,1262.09 1500.17,1263.07 1519.12,1264.04 1538.08,1265 \n",
       "  1557.04,1265.94 1576,1266.86 1594.96,1267.77 1613.92,1268.67 1632.88,1269.55 1651.84,1270.42 1670.8,1271.28 1689.76,1272.12 1708.72,1272.95 1727.67,1273.77 \n",
       "  1746.63,1274.58 1765.59,1275.37 1784.55,1276.15 1803.51,1276.92 1822.47,1277.67 1841.43,1278.41 1860.39,1279.14 1879.35,1279.86 1898.31,1280.57 1917.27,1281.27 \n",
       "  1936.22,1281.95 1955.18,1282.62 1974.14,1283.28 1993.1,1283.93 2012.06,1284.57 2031.02,1285.2 2049.98,1285.81 2068.94,1286.42 2087.9,1287.01 2106.86,1287.59 \n",
       "  2125.82,1288.17 2144.77,1288.73 2163.73,1289.28 2182.69,1289.82 2201.65,1290.35 2220.61,1290.87 2239.57,1291.38 2258.53,1291.88 2277.49,1292.37 2296.45,1292.85 \n",
       "  \n",
       "  \"/>\n",
       "<polygon clip-path=\"url(#clip2602)\" points=\"\n",
       "419.496,751.757 438.455,685.672 457.414,658.359 476.373,634.585 495.333,618.273 514.292,601.961 533.251,586.07 552.21,574.161 571.169,562.252 590.128,550.343 \n",
       "  609.087,538.434 628.046,527.953 647.005,518.859 665.964,509.766 684.924,500.673 703.883,491.58 722.842,482.487 741.801,473.394 760.76,465.548 779.719,458.415 \n",
       "  798.678,451.283 817.637,444.15 836.596,437.017 855.556,429.885 874.515,422.752 893.474,415.619 912.433,408.487 931.392,401.354 950.351,394.96 969.31,389.274 \n",
       "  988.269,383.589 1007.23,377.904 1026.19,372.219 1045.15,366.534 1064.11,360.848 1083.06,355.163 1102.02,349.478 1120.98,343.793 1139.94,338.108 1158.9,332.422 \n",
       "  1177.86,326.737 1196.82,321.052 1215.78,315.367 1234.74,310.38 1253.7,305.811 1272.66,301.242 1291.61,296.673 1310.57,292.104 1329.53,287.535 1348.49,282.966 \n",
       "  1367.45,278.397 1386.41,273.828 1405.37,269.259 1424.33,264.69 1443.29,260.121 1462.25,255.552 1481.21,250.982 1500.17,246.413 1519.12,241.844 1538.08,237.275 \n",
       "  1557.04,232.706 1576,228.137 1594.96,223.568 1613.92,218.999 1632.88,214.43 1651.84,210.373 1670.8,206.695 1689.76,203.016 1708.72,199.337 1727.67,195.658 \n",
       "  1746.63,191.98 1765.59,188.301 1784.55,184.622 1803.51,180.944 1822.47,177.265 1841.43,173.586 1860.39,169.908 1879.35,166.229 1898.31,162.55 1917.27,158.872 \n",
       "  1936.22,155.193 1955.18,151.514 1974.14,147.836 1993.1,144.157 2012.06,140.478 2031.02,136.799 2049.98,133.121 2068.94,129.442 2087.9,125.763 2106.86,122.085 \n",
       "  2125.82,118.406 2144.77,114.727 2163.73,111.049 2182.69,107.37 2201.65,103.691 2220.61,100.013 2239.57,96.3339 2258.53,92.6552 2277.49,89.2032 2296.45,86.2547 \n",
       "  2296.45,1386.61 2277.49,1384.26 2258.53,1381.86 2239.57,1379.42 2220.61,1376.94 2201.65,1374.41 2182.69,1371.83 2163.73,1369.22 2144.77,1366.56 2125.82,1363.86 \n",
       "  2106.86,1361.12 2087.9,1358.33 2068.94,1355.51 2049.98,1352.65 2031.02,1349.74 2012.06,1346.79 1993.1,1343.81 1974.14,1340.78 1955.18,1337.72 1936.22,1334.62 \n",
       "  1917.27,1331.47 1898.31,1328.29 1879.35,1325.07 1860.39,1321.81 1841.43,1318.5 1822.47,1315.16 1803.51,1311.78 1784.55,1308.36 1765.59,1304.9 1746.63,1301.4 \n",
       "  1727.67,1297.86 1708.72,1294.28 1689.76,1290.65 1670.8,1286.99 1651.84,1283.28 1632.88,1279.53 1613.92,1275.74 1594.96,1271.9 1576,1268.02 1557.04,1264.09 \n",
       "  1538.08,1260.12 1519.12,1256.1 1500.17,1252.03 1481.21,1247.91 1462.25,1243.75 1443.29,1239.54 1424.33,1235.27 1405.37,1230.95 1386.41,1226.58 1367.45,1222.16 \n",
       "  1348.49,1217.68 1329.53,1213.14 1310.57,1208.54 1291.61,1203.89 1272.66,1199.17 1253.7,1194.39 1234.74,1189.54 1215.78,1184.63 1196.82,1179.65 1177.86,1174.6 \n",
       "  1158.9,1169.47 1139.94,1164.27 1120.98,1158.99 1102.02,1153.62 1083.06,1148.17 1064.11,1142.63 1045.15,1137 1026.19,1131.28 1007.23,1125.46 988.269,1119.52 \n",
       "  969.31,1113.48 950.351,1107.33 931.392,1101.08 912.433,1094.65 893.474,1088.11 874.515,1081.45 855.556,1074.63 836.596,1067.62 817.637,1060.47 798.678,1053.17 \n",
       "  779.719,1045.59 760.76,1037.84 741.801,1029.93 722.842,1021.65 703.883,1013.15 684.924,1004.44 665.964,995.234 647.005,985.734 628.046,975.94 609.087,965.439 \n",
       "  590.128,954.501 571.169,943.101 552.21,930.609 533.251,917.287 514.292,902.98 495.333,887.077 476.373,868.673 457.414,846.883 438.455,818.574 419.496,751.757 \n",
       "  \n",
       "  \" fill=\"#ff0000\" fill-rule=\"evenodd\" fill-opacity=\"0.1\"/>\n",
       "<polyline clip-path=\"url(#clip2602)\" style=\"stroke:#ff0000; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  419.496,751.757 438.455,685.672 457.414,658.359 476.373,634.585 495.333,618.273 514.292,601.961 533.251,586.07 552.21,574.161 571.169,562.252 590.128,550.343 \n",
       "  609.087,538.434 628.046,527.953 647.005,518.859 665.964,509.766 684.924,500.673 703.883,491.58 722.842,482.487 741.801,473.394 760.76,465.548 779.719,458.415 \n",
       "  798.678,451.283 817.637,444.15 836.596,437.017 855.556,429.885 874.515,422.752 893.474,415.619 912.433,408.487 931.392,401.354 950.351,394.96 969.31,389.274 \n",
       "  988.269,383.589 1007.23,377.904 1026.19,372.219 1045.15,366.534 1064.11,360.848 1083.06,355.163 1102.02,349.478 1120.98,343.793 1139.94,338.108 1158.9,332.422 \n",
       "  1177.86,326.737 1196.82,321.052 1215.78,315.367 1234.74,310.38 1253.7,305.811 1272.66,301.242 1291.61,296.673 1310.57,292.104 1329.53,287.535 1348.49,282.966 \n",
       "  1367.45,278.397 1386.41,273.828 1405.37,269.259 1424.33,264.69 1443.29,260.121 1462.25,255.552 1481.21,250.982 1500.17,246.413 1519.12,241.844 1538.08,237.275 \n",
       "  1557.04,232.706 1576,228.137 1594.96,223.568 1613.92,218.999 1632.88,214.43 1651.84,210.373 1670.8,206.695 1689.76,203.016 1708.72,199.337 1727.67,195.658 \n",
       "  1746.63,191.98 1765.59,188.301 1784.55,184.622 1803.51,180.944 1822.47,177.265 1841.43,173.586 1860.39,169.908 1879.35,166.229 1898.31,162.55 1917.27,158.872 \n",
       "  1936.22,155.193 1955.18,151.514 1974.14,147.836 1993.1,144.157 2012.06,140.478 2031.02,136.799 2049.98,133.121 2068.94,129.442 2087.9,125.763 2106.86,122.085 \n",
       "  2125.82,118.406 2144.77,114.727 2163.73,111.049 2182.69,107.37 2201.65,103.691 2220.61,100.013 2239.57,96.3339 2258.53,92.6552 2277.49,89.2032 2296.45,86.2547 \n",
       "  \n",
       "  \"/>\n",
       "<polygon clip-path=\"url(#clip2602)\" points=\"\n",
       "419.496,1134.6 438.455,1116.87 457.414,1109.57 476.373,1103.93 495.333,1099.18 514.292,1094.99 533.251,1091.21 552.21,1087.76 571.169,1084.55 590.128,1081.53 \n",
       "  609.087,1078.65 628.046,1075.94 647.005,1073.35 665.964,1070.83 684.924,1068.47 703.883,1066.13 722.842,1063.93 741.801,1061.75 760.76,1059.66 779.719,1057.62 \n",
       "  798.678,1055.61 817.637,1053.71 836.596,1051.81 855.556,1049.94 874.515,1048.17 893.474,1046.39 912.433,1044.63 931.392,1042.97 950.351,1041.31 969.31,1039.64 \n",
       "  988.269,1038.04 1007.23,1036.49 1026.19,1034.93 1045.15,1033.37 1064.11,1031.87 1083.06,1030.42 1102.02,1028.96 1120.98,1027.5 1139.94,1026.05 1158.9,1024.68 \n",
       "  1177.86,1023.31 1196.82,1021.95 1215.78,1020.58 1234.74,1019.22 1253.7,1017.94 1272.66,1016.65 1291.61,1015.37 1310.57,1014.09 1329.53,1012.81 1348.49,1011.56 \n",
       "  1367.45,1010.35 1386.41,1009.15 1405.37,1007.95 1424.33,1006.75 1443.29,1005.55 1462.25,1004.34 1481.21,1003.21 1500.17,1002.09 1519.12,1000.96 1538.08,999.832 \n",
       "  1557.04,998.705 1576,997.578 1594.96,996.45 1613.92,995.374 1632.88,994.317 1651.84,993.26 1670.8,992.204 1689.76,991.147 1708.72,990.09 1727.67,989.034 \n",
       "  1746.63,987.977 1765.59,986.952 1784.55,985.962 1803.51,984.972 1822.47,983.982 1841.43,982.992 1860.39,982.002 1879.35,981.012 1898.31,980.023 1917.27,979.033 \n",
       "  1936.22,978.048 1955.18,977.121 1974.14,976.194 1993.1,975.267 2012.06,974.341 2031.02,973.414 2049.98,972.487 2068.94,971.56 2087.9,970.633 2106.86,969.706 \n",
       "  2125.82,968.78 2144.77,967.875 2163.73,967.008 2182.69,966.141 2201.65,965.274 2220.61,964.406 2239.57,963.539 2258.53,962.672 2277.49,961.805 2296.45,960.938 \n",
       "  2296.45,1292.85 2277.49,1292.37 2258.53,1291.88 2239.57,1291.38 2220.61,1290.87 2201.65,1290.35 2182.69,1289.82 2163.73,1289.28 2144.77,1288.73 2125.82,1288.17 \n",
       "  2106.86,1287.59 2087.9,1287.01 2068.94,1286.42 2049.98,1285.81 2031.02,1285.2 2012.06,1284.57 1993.1,1283.93 1974.14,1283.28 1955.18,1282.62 1936.22,1281.95 \n",
       "  1917.27,1281.27 1898.31,1280.57 1879.35,1279.86 1860.39,1279.14 1841.43,1278.41 1822.47,1277.67 1803.51,1276.92 1784.55,1276.15 1765.59,1275.37 1746.63,1274.58 \n",
       "  1727.67,1273.77 1708.72,1272.95 1689.76,1272.12 1670.8,1271.28 1651.84,1270.42 1632.88,1269.55 1613.92,1268.67 1594.96,1267.77 1576,1266.86 1557.04,1265.94 \n",
       "  1538.08,1265 1519.12,1264.04 1500.17,1263.07 1481.21,1262.09 1462.25,1261.09 1443.29,1260.08 1424.33,1259.05 1405.37,1258.01 1386.41,1256.95 1367.45,1255.87 \n",
       "  1348.49,1254.78 1329.53,1253.67 1310.57,1252.54 1291.61,1251.39 1272.66,1250.23 1253.7,1249.05 1234.74,1247.84 1215.78,1246.62 1196.82,1245.38 1177.86,1244.12 \n",
       "  1158.9,1242.84 1139.94,1241.53 1120.98,1240.21 1102.02,1238.86 1083.06,1237.48 1064.11,1236.08 1045.15,1234.66 1026.19,1233.21 1007.23,1231.73 988.269,1230.22 \n",
       "  969.31,1228.68 950.351,1227.11 931.392,1225.51 912.433,1223.87 893.474,1222.2 874.515,1220.48 855.556,1218.73 836.596,1216.93 817.637,1215.09 798.678,1213.2 \n",
       "  779.719,1211.25 760.76,1209.25 741.801,1207.19 722.842,1205.06 703.883,1202.86 684.924,1200.58 665.964,1198.22 647.005,1195.75 628.046,1193.18 609.087,1190.48 \n",
       "  590.128,1187.64 571.169,1184.63 552.21,1181.42 533.251,1177.96 514.292,1174.2 495.333,1170.03 476.373,1165.29 457.414,1159.67 438.455,1152.32 419.496,1134.6 \n",
       "  \n",
       "  \" fill=\"#0000ff\" fill-rule=\"evenodd\" fill-opacity=\"0.1\"/>\n",
       "<polyline clip-path=\"url(#clip2602)\" style=\"stroke:#0000ff; stroke-width:4; stroke-opacity:1; fill:none\" points=\"\n",
       "  419.496,1134.6 438.455,1116.87 457.414,1109.57 476.373,1103.93 495.333,1099.18 514.292,1094.99 533.251,1091.21 552.21,1087.76 571.169,1084.55 590.128,1081.53 \n",
       "  609.087,1078.65 628.046,1075.94 647.005,1073.35 665.964,1070.83 684.924,1068.47 703.883,1066.13 722.842,1063.93 741.801,1061.75 760.76,1059.66 779.719,1057.62 \n",
       "  798.678,1055.61 817.637,1053.71 836.596,1051.81 855.556,1049.94 874.515,1048.17 893.474,1046.39 912.433,1044.63 931.392,1042.97 950.351,1041.31 969.31,1039.64 \n",
       "  988.269,1038.04 1007.23,1036.49 1026.19,1034.93 1045.15,1033.37 1064.11,1031.87 1083.06,1030.42 1102.02,1028.96 1120.98,1027.5 1139.94,1026.05 1158.9,1024.68 \n",
       "  1177.86,1023.31 1196.82,1021.95 1215.78,1020.58 1234.74,1019.22 1253.7,1017.94 1272.66,1016.65 1291.61,1015.37 1310.57,1014.09 1329.53,1012.81 1348.49,1011.56 \n",
       "  1367.45,1010.35 1386.41,1009.15 1405.37,1007.95 1424.33,1006.75 1443.29,1005.55 1462.25,1004.34 1481.21,1003.21 1500.17,1002.09 1519.12,1000.96 1538.08,999.832 \n",
       "  1557.04,998.705 1576,997.578 1594.96,996.45 1613.92,995.374 1632.88,994.317 1651.84,993.26 1670.8,992.204 1689.76,991.147 1708.72,990.09 1727.67,989.034 \n",
       "  1746.63,987.977 1765.59,986.952 1784.55,985.962 1803.51,984.972 1822.47,983.982 1841.43,982.992 1860.39,982.002 1879.35,981.012 1898.31,980.023 1917.27,979.033 \n",
       "  1936.22,978.048 1955.18,977.121 1974.14,976.194 1993.1,975.267 2012.06,974.341 2031.02,973.414 2049.98,972.487 2068.94,971.56 2087.9,970.633 2106.86,969.706 \n",
       "  2125.82,968.78 2144.77,967.875 2163.73,967.008 2182.69,966.141 2201.65,965.274 2220.61,964.406 2239.57,963.539 2258.53,962.672 2277.49,961.805 2296.45,960.938 \n",
       "  \n",
       "  \"/>\n",
       "</svg>\n"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "using QuantEcon, Plots, LinearAlgebra, Interpolations\n",
    "\n",
    "# model parameters\n",
    "a_0 = 100\n",
    "a_1 = 0.5\n",
    "ρ = 0.9\n",
    "σ_d = 0.05\n",
    "β = 0.95\n",
    "c = 2\n",
    "γ = 50.0\n",
    "θ = 0.002\n",
    "ac = (a_0 - c) / 2.0\n",
    "\n",
    "# Define LQ matrices\n",
    "R = [ 0   ac    0;\n",
    "    ac -a_1  0.5;\n",
    "    0.  0.5    0]\n",
    "R = -R  # For minimization\n",
    "Q = Matrix([γ / 2.0]')\n",
    "A = [1. 0. 0.;\n",
    "    0. 1. 0.;\n",
    "    0. 0.  ρ]\n",
    "B = [0. 1. 0.]'\n",
    "C = [0. 0. σ_d]'\n",
    "\n",
    "## Functions\n",
    "\n",
    "function evaluate_policy(θ, F)\n",
    "    rlq = RBLQ(Q, R, A, B, C, β, θ)\n",
    "    K_F, P_F, d_F, O_F, o_F = evaluate_F(rlq, F)\n",
    "    x0 = [1.0 0.0 0.0]'\n",
    "    value = - x0' * P_F * x0 .- d_F\n",
    "    entropy = x0' * O_F * x0 .+ o_F\n",
    "    return value[1], entropy[1]    # return scalars\n",
    "end\n",
    "\n",
    "function value_and_entropy(emax, F, bw, grid_size = 1000)\n",
    "    if lowercase(bw) == \"worst\"\n",
    "        θs = 1 ./ range(1e-8,  1000, length = grid_size)\n",
    "    else\n",
    "        θs = -1 ./ range(1e-8,  1000, length = grid_size)\n",
    "    end\n",
    "\n",
    "    data = zeros(grid_size, 2)\n",
    "\n",
    "    for (i, θ) in enumerate(θs)\n",
    "        data[i, :] = collect(evaluate_policy(θ, F))\n",
    "        if data[i, 2] ≥ emax      # stop at this entropy level\n",
    "            data = data[1:i, :]\n",
    "            break\n",
    "        end\n",
    "    end\n",
    "    return data\n",
    "end\n",
    "\n",
    "## Main\n",
    "\n",
    "# compute optimal rule\n",
    "optimal_lq = QuantEcon.LQ(Q, R, A, B, C, zero(B'A), bet=β)\n",
    "Po, Fo, Do = stationary_values(optimal_lq)\n",
    "\n",
    "# compute robust rule for our θ\n",
    "baseline_robust = RBLQ(Q, R, A, B, C, β, θ)\n",
    "Fb, Kb, Pb = robust_rule(baseline_robust)\n",
    "\n",
    "# Check the positive definiteness of worst-case covariance matrix to\n",
    "# ensure that θ exceeds the breakdown point\n",
    "test_matrix = I - (C' * Pb * C ./ θ)[1]\n",
    "eigenvals, eigenvecs = eigen(test_matrix)\n",
    "@assert all(x -> x ≥ 0, eigenvals)\n",
    "\n",
    "emax = 1.6e6\n",
    "\n",
    "# compute values and entropies\n",
    "optimal_best_case = value_and_entropy(emax, Fo, \"best\")\n",
    "robust_best_case = value_and_entropy(emax, Fb, \"best\")\n",
    "optimal_worst_case = value_and_entropy(emax, Fo, \"worst\")\n",
    "robust_worst_case = value_and_entropy(emax, Fb, \"worst\")\n",
    "\n",
    "# we reverse order of \"worst_case\"s so values are ascending\n",
    "data_pairs = ((optimal_best_case, optimal_worst_case),\n",
    "              (robust_best_case, robust_worst_case))\n",
    "\n",
    "egrid = range(0,  emax, length = 100)\n",
    "egrid_data = []\n",
    "for data_pair in data_pairs\n",
    "    for data in data_pair\n",
    "        x, y = data[:, 2], data[:, 1]\n",
    "        curve = LinearInterpolation(x, y, extrapolation_bc = Line())\n",
    "        push!(egrid_data, curve.(egrid))\n",
    "    end\n",
    "end\n",
    "plot(egrid, egrid_data, color=[:red :red :blue :blue])\n",
    "plot!(egrid, egrid_data[1], fillrange=egrid_data[2],\n",
    "    fillcolor=:red, fillalpha=0.1, color=:red, legend=:none)\n",
    "plot!(egrid, egrid_data[3], fillrange=egrid_data[4],\n",
    "    fillcolor=:blue, fillalpha=0.1, color=:blue, legend=:none)\n",
    "plot!(xlabel=\"Entropy\", ylabel=\"Value\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here’s another such figure, with $ \\theta = 0.002 $ instead of $ 0.02 $\n",
    "\n",
    "<img src=\"https://s3-ap-southeast-2.amazonaws.com/lectures.quantecon.org/jl/_static/figures/kg_small_theta.png\" style=\"width:70%;height:70%\">\n",
    "\n",
    "  \n",
    "Can you explain the different shape of the value-entropy correspondence for the robust policy?\n",
    "\n",
    "\n",
    "<a id='rb-appendix'></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Appendix\n",
    "\n",
    "We sketch the proof only of the first claim in [this section](#rb-eq),\n",
    "which is that, for any given $ \\theta $, $ K(\\hat F, \\theta) = \\hat K $,\n",
    "where $ \\hat K $ is as given in [(8)](#equation-rb-kd)\n",
    "\n",
    "This is the content of the next lemma\n",
    "\n",
    "**Lemma.** If $ \\hat P $ is the fixed point of the map $ \\mathcal B \\circ \\mathcal D $ and $ \\hat F $ is the robust policy as given in [(7)](#equation-rb-oc-ih), then\n",
    "\n",
    "\n",
    "<a id='equation-rb-kft'></a>\n",
    "$$\n",
    "K(\\hat F, \\theta) = (\\theta I - C'\\hat P C)^{-1} C' \\hat P  (A - B \\hat F) \\tag{28}\n",
    "$$\n",
    "\n",
    "*Proof:* As a first step, observe that when $ F = \\hat F $, the Bellman equation associated with the LQ problem  [(11)](#equation-rob-lomf) – [(12)](#equation-rb-a2o) is\n",
    "\n",
    "\n",
    "<a id='equation-rb-a2be'></a>\n",
    "$$\n",
    "\\tilde P = -R - \\hat F' Q \\hat F -\n",
    "\\beta^2 (A - B \\hat F)' \\tilde P C\n",
    "(\\beta \\theta I + \\beta C' \\tilde P C)^{-1} C' \\tilde P (A - B \\hat F) +\n",
    "\\beta (A - B \\hat F)' \\tilde P (A - B \\hat F) \\tag{29}\n",
    "$$\n",
    "\n",
    "(revisit [this discussion](https://lectures.quantecon.org/lqcontrol.html#lq-ih) if you don’t know where [(29)](#equation-rb-a2be) comes from) and the optimal policy is\n",
    "\n",
    "$$\n",
    "w_{t+1} = - \\beta (\\beta \\theta I + \\beta C' \\tilde P C)^{-1}\n",
    "C' \\tilde P (A - B \\hat F) x_t\n",
    "$$\n",
    "\n",
    "Suppose for a moment that $ - \\hat P $ solves the Bellman equation [(29)](#equation-rb-a2be)\n",
    "\n",
    "In this case the policy becomes\n",
    "\n",
    "$$\n",
    "w_{t+1} = (\\theta I - C' \\hat P C)^{-1} C' \\hat P (A - B \\hat F) x_t\n",
    "$$\n",
    "\n",
    "which is exactly the claim in [(28)](#equation-rb-kft)\n",
    "\n",
    "Hence it remains only to show that $ - \\hat P $ solves [(29)](#equation-rb-a2be),\n",
    "or, in other words,\n",
    "\n",
    "$$\n",
    "\\hat  P\n",
    " =\n",
    " R + \\hat F' Q \\hat F\n",
    " + \\beta (A - B \\hat F)' \\hat P C\n",
    "   (\\theta I - C' \\hat P C)^{-1} C' \\hat P (A - B \\hat F)\n",
    "   + \\beta (A - B \\hat F)' \\hat P (A - B \\hat F)\n",
    "$$\n",
    "\n",
    "Using the definition of $ \\mathcal D $, we can rewrite the right-hand\n",
    "side more simply as\n",
    "\n",
    "$$\n",
    "R + \\hat F' Q \\hat F\n",
    "+ \\beta (A - B \\hat F)' \\mathcal D(\\hat P) (A - B \\hat F)\n",
    "$$\n",
    "\n",
    "Although it involves a substantial amount of algebra, it can be shown that the\n",
    "latter is just $ \\hat P $\n",
    "\n",
    "(Hint: Use the fact that $ \\hat P = \\mathcal B( \\mathcal D( \\hat P)) $)"
   ]
  }
 ],
 "metadata": {
  "filename": "robustness.rst",
  "kernelspec": {
   "display_name": "Julia 1.2",
   "language": "julia",
   "name": "julia-1.2"
  },
  "language_info": {
   "file_extension": ".jl",
   "mimetype": "application/julia",
   "name": "julia",
   "version": "1.2.0"
  },
  "title": "Robustness"
 },
 "nbformat": 4,
 "nbformat_minor": 2
}