Skip to content

Commit a606a21

Browse files
jstacclaude
andauthored
Improve cake eating time iteration lecture: fix alpha parameter handling and code style (#729)
This commit improves the cake_eating_time_iter.md lecture with several important fixes and enhancements: 1. **Fix alpha parameter inconsistency**: Changed f and f_prime to take α as an explicit parameter instead of capturing it from global scope. This ensures the α stored in the Model is actually used by the functions. 2. **Fix spelling and grammatical errors**: - Fixed "first order" to "first-order" (compound adjective) - Removed extra spaces in punctuation - Fixed "Use same" to "Use the same" - Removed inappropriate period after "that solves" 3. **Improve code readability**: Unpacked model attributes at the beginning of code blocks instead of repeatedly accessing them via model.attribute notation. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude <noreply@anthropic.com>
1 parent 10d154a commit a606a21

File tree

1 file changed

+91
-58
lines changed

1 file changed

+91
-58
lines changed

lectures/cake_eating_time_iter.md

Lines changed: 91 additions & 58 deletions
Original file line numberDiff line numberDiff line change
@@ -38,7 +38,7 @@ In this lecture, we introduce the core idea of **time iteration**: iterating on
3838
a guess of the optimal policy using the Euler equation.
3939

4040
This approach differs from the value function iteration we used in
41-
{doc}`Cake Eating III <cake_eating_stochastic>`, where we iterated on the value function itself.
41+
{doc}`cake_eating_stochastic`, where we iterated on the value function itself.
4242

4343
Time iteration exploits the structure of the Euler equation to find the optimal
4444
policy directly, rather than computing the value function as an intermediate step.
@@ -49,7 +49,7 @@ policy function, we can often solve problems faster than with value function ite
4949
However, time iteration is not the most efficient Euler equation-based method
5050
available.
5151

52-
In {doc}`Cake Eating V <cake_eating_egm>`, we'll introduce the **endogenous
52+
In {doc}`cake_eating_egm`, we'll introduce the **endogenous
5353
grid method** (EGM), which provides an even more efficient way to solve the
5454
problem.
5555

@@ -68,9 +68,9 @@ from typing import NamedTuple, Callable
6868
## The Euler Equation
6969

7070
Our first step is to derive the Euler equation, which is a generalization of
71-
the Euler equation we obtained in {doc}`Cake Eating I <cake_eating>`.
71+
the Euler equation we obtained in {doc}`cake_eating`.
7272

73-
We take the model set out in {doc}`Cake Eating III <cake_eating_stochastic>` and add the following assumptions:
73+
We take the model set out in {doc}`cake_eating_stochastic` and add the following assumptions:
7474

7575
1. $u$ and $f$ are continuously differentiable and strictly concave
7676
1. $f(0) = 0$
@@ -120,12 +120,12 @@ v^*(x) = \max_{0 \leq k \leq x}
120120
\right\},
121121
$$
122122

123-
Differentiating with respect to $x$, and then evaluating at the optimum yields {eq}`cpi_env`.
123+
Differentiating with respect to $x$, and then evaluating at the optimum yields {eq}`cpi_env`.
124124

125125
(Section 12.1 of [EDTC](https://johnstachurski.net/edtc.html) contains full proofs of these results, and closely related discussions can be found in many other texts.)
126126

127127
Differentiability of the value function and interiority of the optimal policy
128-
imply that optimal consumption satisfies the first order condition associated
128+
imply that optimal consumption satisfies the first-order condition associated
129129
with {eq}`cpi_fpb30`, which is
130130

131131
```{math}
@@ -179,7 +179,7 @@ that are continuous, strictly increasing and interior.
179179
Henceforth we denote this set of policies by $\mathscr P$
180180

181181
1. The operator $K$ takes as its argument a $\sigma \in \mathscr P$ and
182-
1. returns a new function $K\sigma$, where $K\sigma(x)$ is the $c \in (0, x)$ that solves.
182+
1. returns a new function $K\sigma$, where $K\sigma(x)$ is the $c \in (0, x)$ that solves
183183

184184
```{math}
185185
:label: cpi_coledef
@@ -194,7 +194,7 @@ We call this operator the **Coleman-Reffett operator** to acknowledge the work o
194194
In essence, $K\sigma$ is the consumption policy that the Euler equation tells
195195
you to choose today when your future consumption policy is $\sigma$.
196196

197-
The important thing to note about $K$ is that, by
197+
The important thing to note about $K$ is that, by
198198
construction, its fixed points coincide with solutions to the functional
199199
equation {eq}`cpi_euler_func`.
200200

@@ -237,28 +237,46 @@ whenever $\sigma \in \mathscr P$.
237237
It is possible to prove that there is a tight relationship between iterates of
238238
$K$ and iterates of the Bellman operator.
239239

240-
Mathematically, the two operators are **topologically conjugate**.
240+
Mathematically, $T$ and $K$ are **topologically conjugate** under a translation
241+
that involves differentiation in one direction and integration in the other.
241242

242-
Loosely speaking, this means that if iterates of one operator converge then
243+
This conjugacy implies that if iterates of one operator converge then
243244
so do iterates of the other, and vice versa.
244245

245-
Moreover, there is a sense in which they converge at the same rate, at least
246-
in theory.
246+
Moreover, there is a sense in which they converge *at the same rate*, at least in theory.
247247

248-
However, it turns out that the operator $K$ is more stable numerically
248+
However, it turns out that the operator $K$ is more stable *numerically*
249249
and hence more efficient in the applications we consider.
250250

251+
This is because
252+
253+
* $K$ exploits additional structure because it uses first-order conditions, and
254+
* policies near the optimal policy have less curvature and hence are easier to
255+
approximate than value functions near the optimal value function.
256+
251257
Examples are given below.
252258

259+
253260
## Implementation
254261

255-
As in {doc}`Cake Eating III <cake_eating_stochastic>`, we continue to assume that
262+
Let's turn to implementation.
263+
264+
```{note}
265+
In this lecture we mainly focus on the algorithm, favoring clarity over efficiency in the code.
266+
267+
In later lectures we will optimize both the algorithm and the code.
268+
```
269+
270+
271+
272+
As in {doc}`cake_eating_stochastic`, we assume that
256273

257274
* $u(c) = \ln c$
258-
* $f(k) = k^{\alpha}$
259-
* $\phi$ is the distribution of $\xi := \exp(\mu + s \zeta)$ when $\zeta$ is standard normal
275+
* $f(x-c) = (x-c)^{\alpha}$
276+
* $\phi$ is the distribution of $\xi := \exp(\mu + \nu \zeta)$ when $\zeta$ is standard normal
260277

261-
This will allow us to compare our results to the analytical solutions
278+
This allows us to compare our results to the analytical solutions we obtained in
279+
that lecture:
262280

263281
```{code-cell} python3
264282
def v_star(x, α, β, μ):
@@ -283,34 +301,36 @@ means iterating with the operator $K$.
283301

284302
For this we need access to the functions $u'$ and $f, f'$.
285303

286-
We use the same `Model` structure from {doc}`Cake Eating III <cake_eating_stochastic>`.
304+
We use the same `Model` structure from {doc}`cake_eating_stochastic`.
287305

288306
```{code-cell} python3
289307
class Model(NamedTuple):
290308
u: Callable # utility function
291309
f: Callable # production function
292310
β: float # discount factor
293311
μ: float # shock location parameter
294-
s: float # shock scale parameter
312+
ν: float # shock scale parameter
295313
grid: np.ndarray # state grid
296314
shocks: np.ndarray # shock draws
297315
α: float = 0.4 # production function parameter
298316
u_prime: Callable = None # derivative of utility
299317
f_prime: Callable = None # derivative of production
300318
301319
302-
def create_model(u: Callable,
303-
f: Callable,
304-
β: float = 0.96,
305-
μ: float = 0.0,
306-
s: float = 0.1,
307-
grid_max: float = 4.0,
308-
grid_size: int = 120,
309-
shock_size: int = 250,
310-
seed: int = 1234,
311-
α: float = 0.4,
312-
u_prime: Callable = None,
313-
f_prime: Callable = None) -> Model:
320+
def create_model(
321+
u: Callable,
322+
f: Callable,
323+
β: float = 0.96,
324+
μ: float = 0.0,
325+
ν: float = 0.1,
326+
grid_max: float = 4.0,
327+
grid_size: int = 120,
328+
shock_size: int = 250,
329+
seed: int = 1234,
330+
α: float = 0.4,
331+
u_prime: Callable = None,
332+
f_prime: Callable = None
333+
) -> Model:
314334
"""
315335
Creates an instance of the cake eating model.
316336
"""
@@ -319,10 +339,9 @@ def create_model(u: Callable,
319339
320340
# Store shocks (with a seed, so results are reproducible)
321341
np.random.seed(seed)
322-
shocks = np.exp(μ + s * np.random.randn(shock_size))
342+
shocks = np.exp(μ + ν * np.random.randn(shock_size))
323343
324-
return Model(u=u, f=f, β=β, μ=μ, s=s, grid=grid, shocks=shocks,
325-
α=α, u_prime=u_prime, f_prime=f_prime)
344+
return Model(u, f, β, μ, ν, grid, shocks, α, u_prime, f_prime)
326345
```
327346

328347
Now we implement a method called `euler_diff`, which returns
@@ -341,14 +360,14 @@ def euler_diff(c: float, σ: np.ndarray, x: float, model: Model) -> float:
341360
342361
"""
343362
344-
β, shocks, grid = model.β, model.shocks, model.grid
345-
f, f_prime, u_prime = model.f, model.f_prime, model.u_prime
363+
# Unpack
364+
u, f, β, μ, ν, grid, shocks, α, u_prime, f_prime = model
346365
347-
# First turn σ into a function via interpolation
366+
# Turn σ into a function via interpolation
348367
σ_func = lambda x: np.interp(x, grid, σ)
349368
350369
# Now set up the function we need to find the root of.
351-
vals = u_prime(σ_func(f(x - c) * shocks)) * f_prime(x - c) * shocks
370+
vals = u_prime(σ_func(f(x - c, α) * shocks)) * f_prime(x - c, α) * shocks
352371
return u_prime(c) - β * np.mean(vals)
353372
```
354373

@@ -365,12 +384,10 @@ def K(σ: np.ndarray, model: Model) -> np.ndarray:
365384
"""
366385
The Coleman-Reffett operator
367386
368-
Here model is an instance of Model.
369387
"""
370388
371-
β = model.β
372-
f, f_prime, u_prime = model.f, model.f_prime, model.u_prime
373-
grid, shocks = model.grid, model.shocks
389+
# Unpack
390+
u, f, β, μ, ν, grid, shocks, α, u_prime, f_prime = model
374391
375392
σ_new = np.empty_like(σ)
376393
for i, x in enumerate(grid):
@@ -390,8 +407,8 @@ Let's generate an instance and plot some iterates of $K$, starting from $σ(x) =
390407
α = 0.4
391408
u = lambda c: np.log(c)
392409
u_prime = lambda c: 1 / c
393-
f = lambda k: k**α
394-
f_prime = lambda k: α * k**(α - 1)
410+
f = lambda k, α: k**α
411+
f_prime = lambda k, α: α * k**(α - 1)
395412
396413
model = create_model(u=u, f=f, α=α, u_prime=u_prime, f_prime=f_prime)
397414
grid = model.grid
@@ -417,20 +434,24 @@ plt.show()
417434
```
418435

419436
We see that the iteration process converges quickly to a limit
420-
that resembles the solution we obtained in {doc}`Cake Eating III <cake_eating_stochastic>`.
437+
that resembles the solution we obtained in {doc}`cake_eating_stochastic`.
421438

422439
Here is a function called `solve_model_time_iter` that takes an instance of
423440
`Model` and returns an approximation to the optimal policy,
424441
using time iteration.
425442

443+
426444
```{code-cell} python3
427-
def solve_model_time_iter(model: Model,
428-
σ_init: np.ndarray,
429-
tol: float = 1e-5,
430-
max_iter: int = 1000,
431-
verbose: bool = True) -> np.ndarray:
445+
def solve_model_time_iter(
446+
model: Model,
447+
σ_init: np.ndarray,
448+
tol: float = 1e-5,
449+
max_iter: int = 1000,
450+
verbose: bool = True
451+
) -> np.ndarray:
432452
"""
433453
Solve the model using time iteration.
454+
434455
"""
435456
σ = σ_init
436457
error = tol + 1
@@ -453,19 +474,25 @@ def solve_model_time_iter(model: Model,
453474
Let's call it:
454475

455476
```{code-cell} python3
456-
σ_init = np.copy(model.grid)
477+
# Unpack
478+
grid = model.grid
479+
480+
σ_init = np.copy(grid)
457481
σ = solve_model_time_iter(model, σ_init)
458482
```
459483

460484
Here is a plot of the resulting policy, compared with the true policy:
461485

462486
```{code-cell} python3
487+
# Unpack
488+
grid, α, β = model.grid, model.α, model.β
489+
463490
fig, ax = plt.subplots()
464491
465-
ax.plot(model.grid, σ, lw=2,
492+
ax.plot(grid, σ, lw=2,
466493
alpha=0.8, label='approximate policy function')
467494
468-
ax.plot(model.grid, σ_star(model.grid, model.α, model.β), 'k--',
495+
ax.plot(grid, σ_star(grid, α, β), 'k--',
469496
lw=2, alpha=0.8, label='true policy function')
470497
471498
ax.legend()
@@ -477,12 +504,15 @@ Again, the fit is excellent.
477504
The maximal absolute deviation between the two policies is
478505

479506
```{code-cell} python3
480-
np.max(np.abs(σ - σ_star(model.grid, model.α, model.β)))
507+
# Unpack
508+
grid, α, β = model.grid, model.α, model.β
509+
510+
np.max(np.abs(σ - σ_star(grid, α, β)))
481511
```
482512

483513
Time iteration runs faster than value function iteration, as discussed in {doc}`cake_eating_stochastic`.
484514

485-
This is because time iteration exploits differentiability and the first order conditions, while value function iteration does not use this available structure.
515+
This is because time iteration exploits differentiability and the first-order conditions, while value function iteration does not use this available structure.
486516

487517
At the same time, there is a variation of time iteration that runs even faster.
488518

@@ -519,7 +549,7 @@ def u_crra(c):
519549
def u_prime_crra(c):
520550
return c**(-γ)
521551
522-
# Use same production function as before
552+
# Use the same production function as before
523553
model_crra = create_model(u=u_crra, f=f, α=α,
524554
u_prime=u_prime_crra, f_prime=f_prime)
525555
```
@@ -528,13 +558,16 @@ Now we solve and plot the policy:
528558

529559
```{code-cell} python3
530560
%%time
531-
σ_init = np.copy(model_crra.grid)
561+
# Unpack
562+
grid = model_crra.grid
563+
564+
σ_init = np.copy(grid)
532565
σ = solve_model_time_iter(model_crra, σ_init)
533566
534567
535568
fig, ax = plt.subplots()
536569
537-
ax.plot(model_crra.grid, σ, lw=2,
570+
ax.plot(grid, σ, lw=2,
538571
alpha=0.8, label='approximate policy function')
539572
540573
ax.legend()

0 commit comments

Comments
 (0)