This package lets you estimate the value of information (VOI) of receiving a signal, in a simplified model.
It also lets you do a cost-benefit analysis: weighing the costs of the information against the stakes of the decision that may be improved by the information.
A simplified interface to this package is available at valueofinfo.com.
See example.py
:
prior_mu, prior_sigma = 1, 1
prior = lognormal(prior_mu, prior_sigma)
params = SimulationParameters(
prior=prior,
sd_B=10,
bar=6)
simulation_run = SimulationExecutor(params).execute()
cb_params = CostBenefitParameters(
value_units="utils",
money_units="M$",
capital=100,
signal_cost=5,
)
CostBenefitsExecutor(inputs=cb_params, simulation_run=simulation_run).execute()
Clone:
git clone https://github.com/tadamcz/value-of-information
cd value-of-information
Set up virtual environment:
poetry install
Run example
poetry run python example.py
This package has a robust set of unit tests and end-to-end tests.
tadamcz/bayes-continuous
is a dependency of this package used to
compute the Bayesian updates by numerical integration. It has its own set of tests, based on closed-form expressions
that are available for conjugate priors.
When we gain information about a decision-relevant quantity, that information may improve the decision we ultimately make. The value of the (expected) improvement in the decision is the value of information.
Sometimes, the information we gain tells us that one action is certain to be better than another (for example, knowing that tails came up in a coin toss implies we should bet on tails). But often the information is imperfect, and can only pull our decision in the direction of optimality, in expectation.
Such imperfect information can be modelled as observing a random variable (or signal) B
that is informative about the
true state of the world T
but contains some noise. The expected value of information is the expected benefit from
observing this random variable.
The realised value of information is:
VOI(T,B) = U(decision(B), T) - U(decision_0, T)
where U
is the payoff function, decision
is the decision function when we have access to the signal, and decision_0
is the decision we make in the absence of the signal.
If a signal of B
fails to change our decision, the value we realised is zero (regardless of T
). This is intuitive.
When the signal does change our decision, the size of the benefit depends on the true state T
, and on our decision
function, which in turn depends on how the distribution of B
is related to T
.
For each T=t
, the expected value of information is
VOI(t) = E_B[VOI(T,B) | T=t] = E_B[U(decision(B), T) - U(decision_0, T) | T=t]
where E_B
indicates that we're taking expectations with respect to (i.e. over the distribution of) B
.
We can then find the entirely unconditional expected VOI V
by taking expectations of the above with respect to T
:
V = E_T[ E_B[VOI(t,b) | T=t]]
Of course we might also, by the law of iterated expectations, write V=E[VOI(t,b)]
, where the expectation sign without
a subscript means the expectation is taken with respect to the joint distribution of T
and B
. The more explicit
version is helpful for remembering that V
is a double integral, which will make it easier to understand this model
both conceptually and computationally.
In this package, we make some simplifying assumptions:
- We model the decision problem as a binary choice.
- This means
decision(B)
can take only two valuesd_1
andd_2
(anddecision_0
is equal to one of them). - This simplifies
VOI(t)
in the following way. For eacht
, instead of taking expectations ofVOI(t,B)
over infinitely many values ofB|T=t
, we can ask: what is the probability of each decision, i.e. what are the probabilitiesP(d_1|T=t)
andP(d_2|T=t)
?
- This means
- The binary choice is between:
- the bar (
d_1
): an option with an expected payoff ofbar
about which we cannot gain additional information. Expressed mathematically, the inability to gain additional information means thatU(d_1, T)
is independent ofT
. So we can writeE[U(d_1)]=bar
. (It's irrelevant whether or not there is uncertainty over the payoffU(d_1)
, what matters here is that this uncertainty is independent ofT
so we cannot gain additional information). - the object of study (
d_2
): an uncertain option whose payoff isT
, about which we can gain additional information. The simplification here is thatU(d_2, T)=T
, but a more complicated dependenceU(d_2, T)=f(T)
could easily be modeled.
- the bar (
- The decision-maker is rational, i.e. upon receiving a signal of
B=b
they update their priorP(T)
toP(T|B=b)
. They risk-neutrally maximise expectedU
, which means they choose the object of study if and only ifE[T|B=b]>bar
( orE[T]>bar
in the absence of the signal). - The problem is one-dimensional, i.e.
T
andB
follow one-dimensional distributions. - Currently, only one distribution family is supported for
B
:B
has a normal distribution with unknown meanT
and known standard deviation.
The prior over T
can be any one-dimensional SciPy continuous distribution.
To recapitulate, it may be helpful to think about how we might simulate this process. In each simulation iteration i
:
- We draw a true value
t_i
from the decision-maker's priorP(T)
. - We draw an estimate
b_i
fromNormal(t_i,sd(B))
. - We can then calculate the decision that would be made with and without access to the signal:
- With the signal. The decision-maker's subjective posterior expected value is
E[T|b_i]
. IfE[T|b_i]>bar
, the decision-maker chooses the object of study, otherwise they choose the bar. - Without the signal. If
E[T]>bar
, the decision-maker chooses the object of study, otherwise they choose the bar.
- With the signal. The decision-maker's subjective posterior expected value is
- We calculate the decision-maker's payoffs with and without access to the signal. If choosing the object of study,
they get a payoff of
T_i
; the payoff for the bar isbar
.
Drawing t_i
corresponds to the outer expectation E_T[]
discussed above, and drawing b_i
(dependent on t_i
)
corresponds to the inner expectation E_B[]
. As we noted, for a discrete choice (in our case a binary one) the inner
expectation does not require an integral, so explicitly drawing a b_i
like in step 2 is not necessary, but it will
give a correct estimate for V
. In addition, explicitly drawing a b_i
might be easier to think about and check the
correctness of: each simulation iteration considers a fully specified world + observation pair t_i,b_i
drawn from the
joint distribution of T
and B
.
Drawing b_i
can be disabled by setting force_explicit_b_draw=False
Astute readers will have noticed another simplification. In calculating V
, we take expectations over T
according to
the decision maker's prior P(T)
(this is because T_i
s are drawn from P(T)
in step 1). In a subjective bayesian
sense, this means that we compute the expected VOI by the lights of the decision-maker; a frequentist interpretation
might be that the decision situation is drawn from a larger reference class in which T
follows P(T)
, and we are
computing the average VOI in that class.
These concepts need not coincide in general. We could without difficulty model the decision-maker as acting according
to P(T)
, but nonetheless compute the value of information by the lights of another actor who believes Q(T)
(or the
VOI in a reference class following Q(T)
).
Analogously, V
is calculated according to the same values as the decision-maker's values, i.e. it is modeled from a
risk-neutral U
-maximisation perspective, but this need not be so. (Technically this assumption is already present in
the first section of this document).
For each value of T=t
that we consider (in an integral or a simulation), we have to calculate P(d_2|T=t)
, the
probability that the object of study will be chosen. Or, if considering pairs t,b
drawn from the joint distribution,
we have to calculate decision(b)
for each pair. Both of these would depend on the decision maker's subjective
posterior distribution P(T|B)
. Computing P(T|B)
is computationally expensive in general.
However, we can make use of the following fact to avoid explicitly computing the posterior each time:
When the signal
B
is normally distributed, with meanT
, then, for any prior distribution overT
,E[T|B=b]
is increasing inb
.
This was shown by Andrews et al. 1972 (Lemma 1). It was generalised
by Ma 1999 (Corollary 1.3) to any likelihood function arising from a B
that (i) has T
as a
location parameter, and (ii) is strongly unimodally distributed.
In these cases, instead of explicitly computing the posterior for every b
-value, we
- First run a numerical equation solver to find the threshold value
b_*
("b-star"), such thatE[T|B=b]>bar
if and only ifb>b_*
. - Then, simply compare subsequent
b
-values tob_*
.
This is hundreds of times faster than explicitly computing the posterior probability distribution P(T|B=b)
for
many b
-values.
The shortcut and can be applied whether we are calculating P(d_2|T=t)
(binary choice approach) or decision(b)
(t,b
pairs approach).
The shortcut can be disabled by passing force_explicit_bayes=True
Note: Currently B~Normal(T,sd(B))
is the only distribution supported for B
, so this computational shortcut is always
applicable (i.e. "these cases" are all cases).
We can now once again recapitulate and get the following expression:
VOI(t)
= E_B[VOI(t,B) | T=t]
= E_B[U(decision(b), t) - U(decision_0, t) | T=t]
= P(B>b_* | T=t)*U(d_2, t) + P(B<b_* | T=t)*U(d_1, t) - U(decision_0, t)
= P(B>b_* | T=t)*t + P(B<b_* | T=t)*bar - U(decision_0, t)
Given that B
follows a normal distribution with mean T
and CDF F
:
VOI(t,B)
= (1-F(b_*))*t + F(b_*)*bar - U(decision_0, t)
= F(b_*) * (bar-t) + t - U(decision_0, t)
If P(T|B)
can be calculated in analytically, it follows that b_*
can be calculated analytically. In the case of a
normal prior over T
with parameters μ
, σ
(conjugate to the normal
likelihood function for B
)
E[T|B=b]>bar
iff
b*sd(B)^2 + mu*σ^2
──────────────────── > bar
sd(B)^2 + σ^2
iff
sd(B)^2 * (bar - μ)
b > ─────────────────── + bar
σ^2
It may be possible to use this result to directly calculate V
in closed form as well, but I have not yet found such an
expression.
Tests of tadamcz/bayes-continuous
(a dependency of this package) use
closed form solutions as well.
The previous section is quite abstract. It may be helpful to walk through a concrete example where our simplifications are reasonable, and the model is therefore a suitable one.
These could be estimated either by Monte Carlo simulation, or by explicit numerical integration. The current version uses simulation.
The cost-benefit analysis assumes:
- "Choosing" the bar or the object of study means spending one's capital implementing that option. The amount of capital may vary.
T
andbar
are expressed in terms of value realised per unit of capital. For example, "deaths averted per million dollars" or "new clients per dollar".- The decision-maker can choose to spend
signal_cost
to acquire the signal. All other capital is spent implementing the option with the highest expected value.
This model is well-suited when choosing between different options that can absorb flexible amounts of capital (e.g. venture capital, ad spend, or philanthropy). However, it should be easy to model the costs and benefits differently, while leaving unchanged the functionality concerned with quantifying the value of information, which is more general and is the main contribution of this package.
The console output should be relatively self-explanatory. The calculations can be read
in signal_cost_benefit.py
.
# At the root
poetry run pytest -n auto
This work was done under contract for Open Philanthropy. Open Philanthropy plans to use this tool as one input into the decision of whether to fund randomized trials in global health or development. Because the concept is quite general, we hope that the tool can also be useful to others.