-
Notifications
You must be signed in to change notification settings - Fork 3
Stochastic Dynamic Programming in Java
We provide an example-driven short introduction to Stochastic Dynamic Programming in Java.
A tutorial on Java lambda calculus can be accessed here.
Tutorials on Java collections, streams, and MapReduce operations can be accessed here.
(work in progress...)
We discuss two examples of how Stochastic Dynamic Programming can be implemented in Java, the relevant code can be found in package jsdp.app.standalone; please note that these examples do not rely on the jsdp library.
Consider a 3-period inventory control problem. At the beginning of each period the firm should decide how many units of a product should be produced. If production takes place for x units, where x > 0, we incur a production cost c(x). This cost comprises both a fix and a variable component: c(x) = 0, if x = 0; c(x) = 3+2x, otherwise.
Production in each period cannot exceed 4 units. Demand in each period takes two possible values: 1 or 2 units with equal probability (0.5). Demand is observed in each period only after production has occurred. After meeting current period's demand holding cost of $1 per unit is incurred for any item that is carried over from one period to the next. Because of limited capacity the inventory at the end of each period cannot exceed 3 units. All demand should be met on time (no backorders). If at the end of the planning horizon (i.e. period 3) the firm still has units in stock, these can be salvaged at $2 per unit. The initial inventory is 1 unit.
(work in progress...)
This problem is taken from W. L. Winston, Operations Research: Applications and Algorithms (7th Edition), Duxbury Press, 2003, chap. 19, example 3.
A gambler has initialWealth. She is allowed to play a game of chance over a given betHorizon, and her goal is to maximize her probability of ending up with a least targetWealth.
If the gambler bets $b on a play of the game, then with probability p, she wins the game and increases her capital position by $b; with probability (1-p), she loses the game and decreases her capital by $b.
On any play of the game, the gambler may not bet more money than she has available.
Determine a betting strategy that will maximize the gambler's probability of attaining a wealth of at least $targetWealth by the end of the betting horizon.
(work in progress...)