14
Decisions in Uncertain Contexts
This chapter explores how to m ake rational deci si ons regar di n g
actions to be t aken in a context where the state variables on which
a dec i si on is based are themselves uncer t ain . Basic contextual
notions will be presented through an illustrative example before
introducing the formal mathematical notation associated with
rational decisions and value of information.
14.1 Introductory Example
(a) Given an
m
3
of possibly con-
taminated soil.
Pr = 0.9
Pr = 0.1
$0
9
$100
9$10K
9
$100
E[$| ]=9$1K
E[$| ]=9$100
(b) State probabilities, values, and
expected values.
Figure 14.1: Decision context for a soil
contamination example.
In or de r to introdu ce the noti ons associat ed wit h utility theory,
we re vi s it the soil contamination example presented in §8.2.5,
where we have a cubic meter of potentially contaminated soil . The
two possi b l e states
x 2{, }
are e i t he r contaminated or not
contaminated, and the two possi bl e acti ons
a 2{, }
are e i t he r to
do nothing or t o send the soil to a recycling plant where it is going
to be decontaminated. A soi l sampl e is defined as contaminated
if the pollutant concentration
c
exceeds a threshold value
c>
adm.
. Th e issu e is that for most m
3
in an indu st r i al sit e, the re
are n o observations made in order to verify whether or not it is
contaminated. Here, we rely on a regre ssi on model buil t from a set
of d is cr et e obser vations to predi ct for any
x, y, z
coordinates what
our knowledge of t he contaminant conc entration is as descri bed
by it s expected value and variance. In this example, our current
knowledge indicates that, for a specific m
3
,
Pr(C
adm.
) ⌘ Pr(X = )=0.9
Pr(C>
adm.
) ⌘ Pr(X = )=0.1.
The optimal decision regarding the action to choose depends
on t he value incurr ed by taking an action given the state
x
.In
this example we have two states and two actions, so there are four
possibilities to be defined, as illustrate d in figure 14.1. The incurred
j.-a. goulet 230
value ($) for each p ai r of st at e s x and actions taken a are
$(a, x)=$
, ,
, ,
⌘
$0 $10K
$100 $100
,
where doing nothing when the soil is n ot contaminated incurs no
cost, decontaminating incurs a cost of $100/m
3
whether or not t h e
soil is contaminated, and omitting the decontamination when it is
actually contaminated (i.e.,
c>
adm.
) i n cu rs a cost of $10 K in
future legal fees and compensatory damages.
Actions
Do nothing
Recycle
States
Not contaminated
Contaminated
:$100
Pr=0.1
:$100
Pr=0.9
:$10K
Pr=0.1
:$0
Pr=0.9
E[$| ]= $100
E[$| ]=
⇠
⇠
⇠
⇠
$1 000
Figure 14.2: Decision tree for the soil
contamination example.
Figure 14.2 depicts a decision tree illustrating the actions along
with the probability of each out c ome. The opti mal acti on to be
taken must be sele ct e d based on the expect ed value condit i onal on
each action,
E[$| ] = $( , ) ⇥ Pr(X = ) + $( , ) ⇥ Pr(X = )
= ($0 ⇥ 0.9) + ($10K ⇥ 0.1)
= $1 000
E[$| ] = $( , ) ⇥ Pr(X = ) + $( , ) ⇥ Pr(X = )
=($100 ⇥ 0.9) + ($100 ⇥0.1)
= $100 .
Because
E
[$
|
]
> E
[$
|
], the optimal action is thus t o send the
soil to a recycl i ng plant where it will be decontaminated. As you
can e x pect , changing the probabi l i ty of each state or changing the
incurred relative value ($) for a pair of states
x
and actions taken
a
will influence the optimal action to be taken. The next section
presents the utility theory, which formalizes the method employed
in this introductory ex amp le .
14.2 Utility Theory
Utility theory defines how rational decisions are made. This section
presents the nomenclature associated with utility theory, it formal-
izes what a ration al deci si on is, and it presents its fund amental
axioms.
14.2.1 Nomenclature
A = {a
1
, ··· , a
A
}
Asetofpossible
actions
x 2X✓Z or ✓ R
An outcome from
asetofpossible
states
Pr(X = x) ⌘ Pr(x)
Probability of a
state x
U(a, x)
Utility given a
state
x
and an
action a
A decision consists in the task of cho os i ng an action a
i
from a set of
possible actions
A
=
{
a
1
,
a
2
, ··· ,
a
A
}
. Th i s decis i on is based on our
knowledge of on e or several state variables
x
, wh i ch can either be
discrete or continuous. Our knowledge of a s t ate variable is eithe r
described by its probability density
X ⇠ f
(
x
) or mass functi on
X ⇠ p
(
x
)=
Pr
(
X
=
x
). The utility
U
(
a, x
) q u antifies th e relat i ve
probabilistic machine learning for civil engineers 231
preferences we have f or the joint result of taking an action
a
while
being in a state x.
Soil contamination ex am pl e For the example presented in the in-
troduction of this chapter, the actions and s tat e s can be formulated
as bi n ary disc rete variables;
a 2{, }⌘{
0
,
1
}
and
x 2{ , }⌘
{
0
,
1
}
. Th e probabi li ty of each state is
Pr
(
X
=
x
)=
{
0
.
9
,
0
.
1
}
, an d
the utility of each pair of act i ons and states, is
U(a, x)=U
, ,
, ,
⌘
$0 $10K
$100 $100
.
14.2.2 Rational Decisions
In t h e context of the utility theory, a rational decision is defined
as a decisi on that maximi z es the expected utility. In an uncertain
context, the perceived benefit of an action a
i
is measured by t he
expected ut ility,
U
(
a
)
⌘ E
[
U
(
a, X
)]. When
X
is a disc re t e random
variable so that x 2X= {1, 2, ··· , X},theexpectedutilityis
E[U(a, X)] =
X
x2X
U(a, x) ·p(x).
Instead, when
X
is a continuous rand om variable, the expect e d
utility is
E[U(a, X)] =
Z
U(a, x) ·f(x)dx.
The opti m al action
a
⇤
is the on e that maxim i z es the expected
utility,
a
⇤
= arg max
a2A
E[U(a, X)],
U(a
⇤
) ⌘ E[U(a
⇤
,X)] = max
a2A
E[U(a, X)].
14.2.3 Axioms of Utility Theory
The axioms of uti l ity the or y are based on the concepts of lotteries.
A lot t er y
L
is defined by a set of possible outcomes
x 2X
=
{
1
,
2
, ··· , X}
, each having its own probability of occurre nc e
p
X
(
x
)=
Pr(X = x), so that
L =[{p
X
(1),x=1}; {p
X
(2),x=2}; ··· ; {p
X
(X),x=X}]. (14.1)
A dec i si on is a choice made between several lott er ie s. For the soil
contamination example, there are two lotteries, each one corre-
sponding to a possi bl e acti on. If we choose to send the soil to a
recycling facility, we are cert ain of the outcome, that is, the soil is
j.-a. goulet 232
not contaminated after its treatment. If we choose t o do nothing,
there are two possib l e outcome s; eit h er no significant contaminant
was pr es ent with a probabili ty of 0
.
9, or the soil was wrongly left
without treatment with a probability of 0
.
1. These two lotteries can
be summarized as
L =[{1.0, ( , )}; {0.0, ( , )}]
L =[{0.9, ( , )}; {0.1, ( , )}].
The nomenclature employed for ordering preferences is
L
i
L
j
if
we prefer
L
i
over
L
j
,
L
i
⇠ L
j
if we are indi↵e re nt about
L
i
and
L
j
,
and
L
i
⌫ L
j
if either we prefer
L
i
over
L
j
or ar e indi ↵e re nt. The
axioms of utility theory
1
define a rational behavior. Here, we review
1
Von Neumann, J. and O. Morgenstern
(1947). The theory of games and economic
behavior.PrincetonUniversityPress
these axioms following the nomenclature employed by Russell and
Norvig:
2
2
Russell, S. and P. Norvig (2009). Artificial
Intelligence: A Modern Approach (3rd ed.).
Prentice-Hall
Orderability
(Completeness)
A dec i si on maker has well-defined prefe re nc es
for lotteries so that one of
(L
i
L
j
), (L
j
L
i
), (L
i
⇠ L
j
) holds.
Transitivity
Preferences over lotteries are transitive, so that
if (L
i
L
j
) and (L
j
L
k
), then (L
i
L
k
).
Continuity
If the preferences for lotterie s are ordered follow-
ing (
L
i
L
j
L
k
), then a probab i l ity
p
exists
so that [{p, L
i
}; {1 p, L
k
}] ⇠ L
j
.
Substitutability
(Indep endence)
If two lotteries are equivalent (L
i
⇠ L
j
),
then the lottery
L
i
or
L
j
can be subs t it u t ed
by anot he r equi valent lott e ry
L
k
following
[{p, L
i
}; {1 p, L
k
}] ⇠ [{p, L
j
}; {1 p, L
k
}].
Monotonicity
If t h e lott er y
L
i
is preferred over
L
j
,
L
i
L
j
,
then for a probab il i ty
p
greater than
q
,
[{p, L
i
}; {1 p, L
j
}] [{q, L
i
}; {1 q,L
j
}].
Decomposabil it y
The decomposability property assumes that
there is no reward associat ed wit h the deci si on-
making process itself, that is, there is no fun in
gambling.
L
i
L
j
L
i
is preferred over L
j
L
i
⇠ L
j
L
i
and
L
j
are indi↵er-
ent
L
i
⌫ L
j
L
i
is preferred over
L
j
or are indi↵erent
If the preferences over lotteries can be defined following all these
axioms, then a r at ion al deci si on maker should choose the lotte ry
associated with the action that m axi mi zes the expected utility.
14.3 Utility Functions
The axioms of uti l ity the or y are used to define a utility function
so t hat if a lottery
L
i
is preferred over
L
j
, t h en the lotte ry
L
i
must have a greater utili ty than
L
j
. If we are indi↵erent about
two lott e ri e s, the ir ut il i t i es must be equal. These properti es are
probabilistic machine learning for civil engineers 233
summarized as
U(L
i
) > U(L
j
) , L
i
L
j
U(L
i
)=U(L
j
) , L
i
⇠ L
j
.
The expected utility of a lottery (see equation 14.1) is define d as the
sum of the utility of each possible outcome in the lottery multiplied
by its probability,
E[U(L)] =
X
x2X
p(x) · U(x).
Keep in mind that a utility function defines the relative preferences
between outcomes and not t h e absolut e one. It means that we can
transform a utility function through an affine function,
U
tr
(x)=wU(x)+b, w > 0,
without changing the preferences of a decision maker. In the case
where we multiply a ut i li ty funct i on by a negative constant
w<
0,
this is no longe r tru e; ins te ad of maximizin g the expect ed uti l i ty,
the problem would then c ons i st in minimizing the expected loss,
a
⇤
= arg min
a
E[L(a, X)].
Nonlinear utility functions We now look at the example of a
literal lottery as the term is commonly employed outside the field
of utility theory. This lottery is defined like so: you rece ive $200 if a
coin toss lands on heads , and you pay $100 if a coin toss lands on
tails. This situation is formalized by the two lotteries respectively
corresponding to taking the bet ( ) or passi n g ( ),
L =[{
1
2
, + }; {
1
2
, }]
L =[{1, $0}],
where you are only cert ai n of the outcome if you choose to pass.
The question is, Do you want to take the bet? In practice, when
people are asked, most would not accept the bet, which contradicts
the utility theory principle requiring that one chooses the lottery
with the highest expecte d utility. For this problem,
E[$(L )] =
1
2
⇥ +$200 +
1
2
⇥$100 = +$50
E[$(L )] = $0,
which i n dic at e s that a rational decisi on maker should take the
bet. Does this mean that t he uti l i ty theor y does not work or that
people are acting irrationally? The answer is no; the reason for this
behavior is that utility functions for quantities such as monetary
value are typically nonli n ear .
j.-a. goulet 234
Risk aver s e versus risk seeking Instead of direct l y definin g the
utility for being in a st at e
x
while having taken the action
a
,we
can d efi n e it for continuous quantities such as a monetary valu e.
We define a va lu e
v
(
a, x
) as soci at ed with th e outcome of being in
a st at e
x
while having taken the action
a
, an d
U
(
v
(
a, x
))
⌘ U
(
v
)is
the utility for the value associated with x and a.
Figure 14.3 presents examples of utility functions expressing dif-
ferent risk behaviors. When a utility function is linear with respect
to a value
v
, we say that it represents a risk-neutral behavior, that
is, doubling
v
also doubles
U
(
v
). In comm on cases, indi vi d ual s are
not displaying risk-neutral behavior because, for example, gaining
or l osi n g $1 will impact behavior di↵erently depen d in g on whether
a pers on has $1 or $1
,
000
,
000. A risk- aver s e behavior is charac-
terized by a uti l ity funct i on having a negative second derivative
so t hat th e change in utility for a change of value
v
decreases as
v
increases. The consequence is that given the choice between a
certain event of receiving $100 and a lottery for which t he expected
gain is
E
[
L
] = $100, a risk-averse decision maker would prefer the
certain event. The opposite is a risk-seeking behavior, where there
is a small change in the utility funct ion for small values and large
changes in t he uti l i ty funct ion for large values. When facing the
same previous choice, a risk -s ee k i ng decision maker would prefer the
lottery over the cer tai n event.
0 0.2 0.4 0.6 0.8 1
v
0
1
Utility, (v)
Risk averseRisk neutral
Risk seeking
Figure 14.3: Comparison between risk
-seeking, -neutral,and-averse behaviors for
utility functions.
0 0.5 1
0
1
E[v(a
1
,X)] = E[v(a
2
,X)]
v(a, x)
p(v|a, x)
Action a
1
Action a
2
0 1
0
1
v(a, x)
Utility, U(v)
0
1
E[U(v(a
1
,X))]=E[U(v(a
2
,X))]
p(U(v)|a, x)
(a) Risk neutral
0 0.5 1
0
1
E[v(a
1
,X)] = E[v(a
2
,X)]
v(a, x)
p(v|a, x)
Action a
1
Action a
2
0 1
0
1
v(a, x)
Utility, U(v)
0
1
E[U(v(a
1
,X)) ]
E[U(v(a
2
,X)) ]
p(U(v)|a, x)
(b) Risk averse
Figure 14.4: Discrete case: Comparison of
the e↵ect of a risk-neutral and a risk-averse
behavior on the expected utility.
Let us consider a gener i c utility function for v 2 (0, 1) so that,
U(v)=v
k
, where
8
<
:
0 <k<1Riskaverse
k = 1 Neutral
k>1Riskseeking.
Figure 14.4 compares the e↵ect of a risk -n eu t ral and a risk-seeki n g
behavior on t he condi t i onal expe ct e d util i t i es
E
[
U
(
v
(a
i
,X
))]. In
this example, there is a binary random variable
X
describing
the possible state
x 2{
1
,
2
}
, wh er e the probabi li ty of each out-
come d epe nd s on an action a
i
, an d where
v
(a
i
,x
) i s the value
of bei n g in a state
x
while the action a
i
was taken. Thi s illu s-
trative example is designed so that the expected value for both
actions are equal,
E
[
v
(a
1
,X
)] =
E
[
v
(a
2
,X
)], but not t h ei r vari-
ance,
var
[
v
(a
1
,X
)]
> var
[
v
(a
2
,X
)]. With a r is k -n eu tr al behav-
ior in (a) , the expec t ed uti l i ti e s remain equ al for both actions,
E
[
U
(
v
(a
1
,X
))] =
E
[
U
(
v
(a
2
,X
))]. For the risk-averse behavior dis-
played in ( b ) , the expect e d util i ty is higher for action 2 than for
action 1,
E
[
U
(
v
(a
2
,X
))]
> E
[
U
(
v
(a
1
,X
))], because the variability in
value is greater for action 1 than for action 2.
Figure 14.5 presents the same example, this time for a c ontinu-
ous r an dom variable. The trans form at i on of the probabili ty density
probabilistic machine learning for civil engineers 235
function (PDF) from the value space to the utility space is done
following the change of variable rul e prese nted in §3.4. Again, for
a ri sk -n eu t r al behavior (a), the expect ed uti l i ty is equal, and for
a ri sk -averse behavior (b) , the expec t ed util i ty is higher for action
2 th an for action 1,
E
[
U
(
v
(a
2
,X
))]
> E
[
U
(
v
(a
1
,X
))], because the
variability in the value is greater for action 1 than for acti on 2.
People and organizations typically display a risk-averse behavior,
which must be consider ed if we want to properl y model optimal
decisions. In such a case, an optimal decis i on a
i
is the on e that
maximizes
E
[
U
(
v
(a
i
,X
))]. Although such a d ec is i on is optimal
in the sense of uti l ity the or y, it i s not the one with the highest
expected monetary value. Only a neutral attitude toward risks
maximizes the expected value.
For the bet example presented at the beginning of the section,
we di sc us se d that the action chosen is typically to not take the bet,
even if tak i n g it would result in the highest expect e d gain. Part of
it has t o do with the repeatabil ity of the bet; it is a certainty that
the gain per play will tend to the expected value as the number
of t i mes played increas es . Nevertheless, $100 mi ght appear as an
important sum to gamble because if a person plays and loses, he
or sh e might not a↵ord to play again to make up for the loss. In
the civil engineering context, risk aversion can be seen through a
similar scenario, where if someone loses a large amount of mone y
as t he conseq u en ce of a decision that was rightfully maximizi ng
the expected utility, that person might not keep hi s or her job
long enough t o compen sat e for the current loss with subsequ ent
profitable decisions.
0 0.5 1
E[v(a
1
,X)]=E[v(a
2
,X)]
v(a, x)
f(v|a, x)
Action a
1
Action a
2
0 1
0
1
v(a, x)
Utility, U(v)
0
E[U(v (a
1
,X)) ] =E[U(v(a
2
,X)) ]
f(U(v)|a, x)
(a) Risk neutral
0 0.5 1
E[v(a
1
,X)]=E[v(a
2
,X)]
v(a, x)
f(v|a, x)
Action a
1
Action a
2
0 1
0
1
v(a, x)
Utility, U(v)
0
E[U(v(a
1
,X))]
E[U(v(a
2
,X))]
f(U(v)|a, x)
(b) Risk averse
Figure 14.5: Continuous case: Comparison
of the e↵ect of a linear (risk-neutral) and a
nonlinear (risk-averse) utility function on
the expected utility.
Risk-averse behavior is the reason of being for insurance com-
panies, who have a neutr al attit u de toward the risks they are
providing insurance for. What they do is cover the risks associated
with events that would incur small monetary losses in comparison
with the size of th e company. Insurers seek to diversify the events
covered in or d er to maximi ze the indepe ndence of the probabil i ty
of each event. The objective is to avoid being exposed to payments
so l arge that it would jeopardi ze the solvency of the company. In-
dividuals who buy the insurance are risk averse, so they are ready
to p ay a premium ove r the expect e d value in order not to be put
in a ris k- ne ut ral posit i on. In other words, they accept paying an
amount higher than the expected costs to an insurer, who itself has
to p ay the expected cost because its exposu re to losses is spread
over thousands if not mi ll i on s of cust ome rs .
j.-a. goulet 236
14.4 Value of Information
Value of information quantifies the value associat e d wit h t he action
of gat he r in g additi onal inf or mat i on about a state
X
in order t o
reduce the epistemic uncertainty associated with its knowledge. In
this section, we w il l cover two cases: perfect and imperfect informa-
tion.
14.4.1 Value of Perfect Information
In cas es wher e the value of a state
x 2X
=
{
1
,
2
, ··· , X}
is imper-
fectly known, one possible action is to collect additional information
about
X
. W i t h the current knowledge of
X ⇠ p
(
x
), the expected
utility of the optimal action a
⇤
is
U(a
⇤
) ⌘ E[U(a
⇤
,X)] = max
a
X
x2X
U(a, x) ·p(x)
=
X
x2X
U(a
⇤
,x) · p(x).
(14.2)
If we gather perfect inf orm ati on so that
y
=
x 2X
, we can
then directly observe the true state variable, and the utility of the
optimal action becomes
U(a
⇤
,y) = max
a
U(a, y ).
However, because
y
has not been observed yet, we must consider al l
possible observations
y
=
x
weighted by t h ei r probabi l i ty of occur-
rence so t h e expect ed ut il ity condi t i onal on perfect infor mat i on is
U(˜a
⇤
) ⌘ E[U(˜a
⇤
,X)] =
X
x2X
max
a
U(a, x) ·p(x). (14.3)
Notice how in equat i on 14.2, the max operation was outside of
the sum, whereas in e q uat i on 14.3, the max is inside. In equation
14.2, we comput e the expect e d util i ty wher e the state is a random
variable and for an optimal action that is common for all states.
In eq u at i on 14.3, we assume that we will know the true state once
the observation becomes available, so we will be able to take the
optimal action for each state. Consequently, the e xpected utility
is calculated by weighting the utility corresponding to the optimal
action for each st at e, tim es the probab i lity of occurren ce of that
state. The value of perfect information (VPI) is defin ed as the
di↵erence between the expected utility conditional on perfect
information and the expect ed utility for the optimal action,
VPI(y)=U(˜a
⇤
) U(a
⇤
) 0.
probabilistic machine learning for civil engineers 237
Because the expected utility estimated using perfect information
E
[
U
(
˜a
⇤
,X
)] is greater than
E
[
U
(
a
⇤
,X
)], the
VPI
(
y
) must be greater
or eq u al to zero. The
VPI
quantifies the amount of mon ey you
should be willing to pay t o obtain perfect inf or mat i on. Note that
the concepts presented for discrete random variables can be ex-
tended for continuous on es by r ep lac i ng the sums by integrals.
Soil contamination ex am pl e We apply the concept of the value of
perfect information to the soil contamination example. The current
expected utility conditional on the optimal action is
U(a, x) x = x =
a = $100 $100
a = $0 $10K
y = { , }
y =
:$100
Pr= 1
⇠
⇠
⇠
⇠
:$100
Pr= 0
:$10K
Pr= 1
⇢
⇢
:$0
Pr= 0
Pr = 0.1
y =
⇠
⇠
⇠
⇠
:$100
Pr= 0
:$100
Pr= 1
⇠
⇠
⇠
⇠
:$10K
Pr= 0
:$0
Pr= 1
Pr = 0.9
U = $10
U = $0
U =
⇠
⇠
⇠
$100
U =
⇠
⇠
⇠
⇠
$10K
U = $100
Figure 14.6: Decision tree illustrating the
value of perfect information.
E[U( ,X)] = ($0 ⇥0.9) + ($10K ⇥ 0.1) = $1K
E[U( ,X)] = ($100 ⇥ 0.9) + ($100 ⇥ 0.1) = $100 = U(a
⇤
),
so the current optimal action is to send the soil to a recycling plant
with an as soci at ed expec t ed uti l i ty of -$100. Figure 14.6 presents
the decision tree illustrating the calculation for the value of perfect
information. The expected uti l i ty conditional on perfect inform at ion
is
U(˜a
⇤
)=
X
x2X
max
a
U(a, x) ·p(x)
= $0 ⇥ 0.9
| {z }
y =x =
+ $100 ⇥0.1
| {z }
y =x =
= $10 .
Having access to perfec t inform at ion red uc es the expect e d cost
because there is now a probabil i ty of 0.9 that the observation will
indicate that no treatment is required and only a probability of
0.10 t h at tre atm ent will be requir ed . Moreover, the possibi l i ty of
the worst cas e associ at ed with a false negative where
{a
=
,x
=
}
is now removed from the possibil it ies. The value of perfect
information is thus
VPI(y)=U(˜a
⇤
) U(a
⇤
) = $90 .
It m ean s that we should be willing to pay up to $90 for perfect
information capable of indicating whether not the m
3
of soi l is
contaminated.
14.4.2 Value of Imperfect Information
It i s common that observations of state variables are imperf ec t,
so t hat we want to compute the value of imperfect infor mat i on.
For the case of a discrete state variab l e
x
that belongs to the se t
X
=
{x
1
,x
2
, ··· ,x
X
}
, t h e expect ed cost s condit i onal on imperfec t
j.-a. goulet 238
information is obtained by marginalizing over both the possible
states x 2Xand the possible observations y 2X, so that
U(˜a
⇤
)=
X
y 2X
max
a
⇣
X
x2X
U(a, x) ·p(y|x) · p(x)
| {z }
p(y,x)
⌘
.
p(x, y) y = y =
x = 0.9·1=0.90.9·0=0
x = 0.1·0.05= 0.005 0.1·0.95 =0.095
p(y) 0.905 0.095
p(x|y) y = y =
x =
0.9
0.905
0
0.095
x =
0.005
0.905
0.095
0.095
y = { , }
y =
:$100
Pr=
0.095
0.095
:$100
Pr=
0
0.905
:$10K
Pr=
0.095
0.095
:$0
Pr=
0
0.095
Pr = 0.095
y =
:$100
Pr=
0.005
0.905
:$100
Pr=
0.9
0.905
:$10K
Pr=
0.005
0.905
:$0
Pr=
0.9
0.905
Pr = 0.905
U = $59.5
U = $55.25
U =
⇠
⇠
⇠
$100
U =
⇠
⇠
⇠
⇠
$10K
U = $100
Figure 14.7: Decision tree illustrating the
value of imperfect information.
Soil contamination ex am pl e We apply the concept of the value
of i mper f ec t infor mat i on to the soil contamination exampl e . The
conditional probability of an observation
y
conditional on the state
x is
p(y|x)
⇢
Pr(y = |x = )=1
Pr(y = |x = )=0.95,
where the observation is perfect if the soil is not contaminated,
and the pr obab i l ity of a correct classificat i on is 0
.
95 if the soil is
contaminated.
Figure 14.7 depicts the decision tree for calculating the value
of i nf or mat i on for this example , where the probabi l ity of each
observation is obtained by marginalizing the joint probability for
both observations and st at e s,
p(y)=
X
x2X
p(y|x) · p(x).
The expected utility condit i on al on im pe rf ec t information is then
U(˜a
⇤
)=
X
y 2X
max
a
⇣
X
x2X
U(a, x) ·p(y|x) · p(x)
| {z }
p(x,y )
⌘
= $0 ⇥ 0.9
| {z }
x=
+ $10K ⇥ 0.005
| {z }
x=
| {z }
y =
+ $100 ⇥0
| {z }
x=
+ $100 ⇥0.095
| {z }
x=
| {z }
y =
= $59.5 .
The valu e of information is now defined by the di↵erence between
the expected utility conditional on imperfect information and the
expected utility for the optimal action, so that
VOI (y)=U(˜a
⇤
) U(a
⇤
) 0
= $59.5 ($100)
= $40.5 .
By c omp ari n g this result wit h the value of perfect informat i on
that was equ al to $90, we see that having observation uncer t ai nty
reduces the value of the i n for mat i on .
The reader interested in advanced notions related to rational
decisions and utility theory should consult specialized textbooks
3
3
Russell, S. and P. Norvig (2009). Artificial
Intelligence: A Modern Approach (3rd ed.).
Prentice-Hall; and Bertsekas, D. P. (2007).
Dynamic programming and optimal control
(4th ed.), Volume 1. Athena Scientific
such as those by Russell and Norvi g or Ber t se kas.
