Optimal Stopping - Take Two The following was implemented in Maple by Marcus Davidsson (2010) davidsson_marcus@hotmail.com and is based upon Hill, T (2001) Knowing When to Stop , Allen,S (2006) The Secretary Problem and Niederhoffer, V (2006) Probability, Markets and Decision Making
The first game that we are going to discuss is as follows:
You are allowed to roll a dice a maximum of 6 times.
After each roll you will get the number of points on the face of the dice. The objective is to determined when you should stop to ensure that
you maximize your expected value.
We will solve the above problem by using backward induction.The basic rules that govern the dynamic are as follows:
i) If the outcome of a coin toss is larger or equal to the expected value
in the next period then you should stop otherwise you should continue
ii) The expected value in period n is given by the sum of the probability of a certain payout multiplied by the specific payout for all payouts.
iii) The expected value in period i is given by the probability of not stopping in period i
multiplied by the expected value in period i+1 plus the probability of stopping in period i
multiplied by the expected value of stopping in period i
The expected value for the last dice roll in period 6 is given by:
This means that if you get 1,2 or 3 in the dice roll in period 5 then you should continue since your expected value in the next period is larger than the outcome in this period If you get 4,5 or 6 then you should stop. Hence, P[not stop]=(3/6) and P[stop]=(3/6).
The expected value in period 5 is given by:
This means that if you get 1,2,3 or 4 in the dice roll in period 4 then you should continue since your expected value in the next period is larger than the outcome in this period If you get 5 or 6 then you should stop. Hence, P[not stop]=(4/6) and P[stop]=(2/6).
The expected value in period 4 is given by:
This means that if you get 1,2,3 or 4 in the dice roll in period 3 then you should continue since your expected value in the next period is larger than the outcome in this period If you get 5 or 6 then you should stop. Hence, P[not stop]=(4/6) and P[stop]=(2/6).
The expected value in period 3 is given by:
This means that if you get 1,2,3 or 4 in the dice roll in period 2 then you should continue since your expected value in the next period is larger than the outcome in this period If you get 5 or 6 then you should stop. Hence, P[not stop]=(4/6) and P[stop]=(2/6).
The expected value in period 2 is given by:
This means that if you get 1,2,3,4 or 5 in the first dice roll then you should continue since your expected value in the next period is larger than the outcome in this period If you get 6 then you should stop. Hence, P[not stop]=(5/6) and P[stop]=(1/6).
The expected value in period 1 is given by:
We can now plot the expected value and the lower bound for the stopping rule for each time period as follows: