14
Decisions in Uncertain Contexts
This chapter explores how to m ake rational deci si ons regar di n g
actions to be t aken in a context where the state variables on which
a dec i si on is based are themselves uncer t ain . Basic contextual
notions will be presented through an illustrative example before
introducing the formal mathematical notation associated with
rational decisions and value of information.
14.1 Introductory Example
(a) Given an
m
3
of possibly
contaminated soil.
Pr = 0.9
Pr = 0.1
$0
9$100
9$10K
9$100
E[$| ]=9$1K
E[$| ]=9$100
(b) State probabilities, values, and
expected values.
Figure 14.1: Decision context for a soil
contamination example.
In or de r to introdu ce the noti ons associat ed wit h utility theory,
we re vi s it the soil contamination example presented in
§
8.2.5,
where we have a cubic meter of potentially contaminated soil . The
two possi b l e states
x 2{, }
are e i t he r contaminated or not
contaminated, and the two possi bl e acti ons
a 2{, }
are e i t he r to
do nothing or t o send the soil to a recycling plant where it is going
to be decontaminated. A soi l sampl e is defined as contaminated
if the pollutant concentration
c
exceeds a threshold value
c>
adm.
. Th e issu e is that for most m
3
in an indu st r i al sit e, the re
are n o observations made in order to verify whether or not it is
contaminated. Here, we rely on a regre ssi on model buil t from a set
of d is cr et e obser vations to predi ct for any
x, y, z
coordinates what
our knowledge of t he contaminant conc entration is as descri bed
by it s expected value and variance. In this example, our current
knowledge indicates that, for a specific m
3
,
Pr(C
adm.
) Pr(X = )=0.9
Pr(C>
adm.
) Pr(X = )=0.1.
The optimal decision regarding the action to choose depends
on t he value incurr ed by taking an action given the state
x
.In
this example we have two states and two actions, so there are four
possibilities to be defined, as illustrate d in figure 14.1. The incurred
j.-a. goulet 230
value ($) for each p ai r of st at e s x and actions taken a are
$(a, x)=$
, ,
, ,
$0 $10K
$100 $100
,
where doing nothing when the soil is n ot contaminated incurs no
cost, decontaminating incurs a cost of
$
100/m
3
whether or not t h e
soil is contaminated, and omitting the decontamination when it is
actually contaminated (i.e.,
c>
adm.
) i n cu rs a cost of
$
10 K in
future legal fees and compensatory damages.
Actions
Do nothing
Recycle
States
Not contaminated
Contaminated
:$100
Pr=0.1
:$100
Pr=0.9
:$10K
Pr=0.1
:$0
Pr=0.9
E[$| ]= $100
E[$| ]=
$1 000
Figure 14.2: Decision tree for the soil
contamination example.
Figure 14.2 depicts a decision tree illustrating the actions along
with the probability of each out c ome. The opti mal acti on to be
taken must be sele ct e d based on the expect ed value condit i onal on
each action,
E[$| ] = $( , ) Pr(X = ) + $( , ) Pr(X = )
= ($0 0.9) + ($10K 0.1)
= $1 000
E[$| ] = $( , ) Pr(X = ) + $( , ) Pr(X = )
=($100 0.9) + ($100 0.1)
= $100 .
Because
E
[$
|
]
> E
[$
|
], the optimal action is thus t o send the
soil to a recycl i ng plant where it will be decontaminated. As you
can e x pect , changing the probabi l i ty of each state or changing the
incurred relative value (
$
) f or a pair of states
x
and actions taken
a
will influence the optimal action to be taken. The next section
presents the utility theory, which formalizes the method employed
in this introductory ex amp le .
14.2 Utility Theory
Utility theory defines how rational decisions are made. This section
presents the nomenclature associated with utility theory, it formal-
izes what a ration al deci si on is, and it presents its fund amental
axioms.
14.2.1 Nomenclature
A = {a
1
, ··· , a
A
}
Asetofpossible
actions
x 2XZ or R
An outcome from
asetofpossible
states
Pr(X = x) Pr(x)
Probability of a
state x
U(a, x)
Utility given a
state
x
and an
action a
A decision consists in the task of cho os i ng an action a
i
from a set of
possible actions
A
=
{
a
1
,
a
2
, ··· ,
a
A
}
. Th i s decis i on is based on our
knowledge of on e or several state variables
x
, wh i ch can either be
discrete or continuous. Our knowledge of a s t ate variable is eithe r
described by its probability density
X f
(
x
) or mass functi on
X p
(
x
)=
Pr
(
X
=
x
). The utility
U
(
a, x
) q u antifies th e relat i ve
probabilistic machine learning for civil engineers 231
preferences we have f or the joint result of taking an action
a
while
being in a state x.
Soil contamination ex am pl e For the example presented in the in-
troduction of this chapter, the actions and s tat e s can be formulated
as bi n ary disc rete variables;
a 2{, }{
0
,
1
}
and
x 2{ , }
{
0
,
1
}
. Th e probabi li ty of each state is
Pr
(
X
=
x
)=
{
0
.
9
,
0
.
1
}
, an d
the utility of each pair of act i ons , and states, is
U(a, x)=U
, ,
, ,
$0 $10K
$100 $100
.
14.2.2 Rational Decisions
In t h e context of the utility theory, a rational decision is defined
as a decisi on that maximi z es the expected utility. In an uncertain
context, the perceived benefit of an action a
i
is measured by t he
expected ut ility,
U
(
a
)
E
[
U
(
a, X
)]. When
X
is a disc re t e random
variable so that x 2X= {1, 2, ··· , X},theexpectedutilityis
E[U(a, X)] =
X
x2X
U(a, x) ·p(x).
Instead, when
X
is a continuous rand om variable, the expect e d
utility is
E[U(a, X)] =
Z
U(a, x) ·f(x)dx.
The opti m al action
a
is the on e that maxim i z es the expected
utility,
a
= arg max
a2A
E[U(a, X)],
U(a
) E[U(a
,X)] = max
a2A
E[U(a, X)].
14.2.3 Axioms of Utility Theory
The axioms of uti l ity the or y are based on the concepts of lotteries.
A lot t er y
L
is defined by a set of possible outcomes
x 2X
=
{
1
,
2
, ··· , X}
, each having its own probability of occurre nc e
p
X
(
x
)=
Pr(X = x), so that
L =[{p
X
(1),x=1}; {p
X
(2),x=2}; ··· ; {p
X
(X),x=X}]. (14.1)
A dec i si on is a choice made between several lott er ie s. For the soil
contamination example, there are two lotteries, each one corre-
sponding to a possi bl e acti on. If we choose to send the soil to a
recycling facility, we are cert ain of the outcome, that is, the soil is