Question

Exercise 12.2 In Example 12.1b, find the optimal strategy and the op- timal value when the urn contains three red and four blue balls S. Example 12.1b An urn initially has n red and m blue balls. At each stage the player may randomly choose a ball from the urn; if the ball is red, then 1 is earned, and if it is blue, then 1 is lost. The chosen ball is discarded. At any time the player can decide to stop playing. To maxi- mize the player's total expected net return, we analyze this as a dynamic programming problem with the state equal to the current composition of the urn. We let V(r, b) denote the maximum expected additional return given that there are currently r red and b blue balls in the urn. Now, the expected immediate reward if a ball is chosen in state (r, b) is r+b r+b r+b V(r-1, b) if a red ball is chosen, or V(r, b -1) if a blue ball is chosen, we see that the optimality equation is - Because the best one can do after the initial draw is V(r, b - 1) The Stochastic Dynamic Programming Problem 233 Starting with V(r, 0)r and V(O, b)0, the optimality equation can be utilized to obtain the desired value V(n, m). In some problems a reward is only earned when the problem ends.

Answer 1

2 1 28 m 二 1.3m