replicator-dynamics.rst

Replicator dynamics

Motivating example: The Hawk Dove Game

Consider a population of animals. These animals when they interact will always share their food. Due to a genetic mutation, some of these animals may act in an aggressive manner and not share their food. If two aggressive animals meet they both compete and end up with no food. If an aggressive animal meets a sharing one, the aggressive one will take most of the food.

These interactions can be represented using the matrix A:

$$\begin{aligned} A = \begin{pmatrix} 2 & 1\\\ 3 & 0 \end{pmatrix} \end{aligned}$$

In this scenario: what is the likely long term effect of the genetic mutation?

Over time will:

• The population resist the mutation and all the animals continue to share their food.
• The population get taken over by the mutation and all animals become aggressive.
• A mix of animals are present in the population some act aggressively and some share.

To answer this question we will assume a vector x represents the population. In this case:

• x₁ represents the proportion of the population that shares.
• x₂ represents the proportion of the population that acts aggressively.

Note that as the components of x are proportions of the population this implies:

∑_ix_i = 1

We will also assume that any given individual in the population is playing a strategy <strategies-discussion> χ, which:

• shares χ₁ proportion of the time.
• acts aggressively χ₂ proportion of the time.

The overall fitness of an individual in the population is then given by their expected utility (as given by A) as they interact with the population:

χAx

We can in fact write down the fitness corresponding to each action (sharing or being aggressive):

f = Ax

The average fitness in the population is then given by:

ϕ = x^Tf

To understand how the population will evolve relative to their fitness the following differential equation will be used:

$$\frac{dx_i}{dt} = x_i(f_i - \phi)\text{ for all }i$$

In our case the differential equations are:

$$\begin{aligned} \begin{align} \frac{dx_1}{dt} &= x_1(2x_1 + x_2 - \phi)\\\ \frac{dx_2}{dt} &= x_2(3x_1 - \phi) \end{align} \end{aligned}$$

where:

ϕ = x₁(2x₁ + x₂) + x₂(3x₁)

This differential equation can then be solved numerically <how-to-use-replicator-dynamics> to show the evolution of the population over time. We can see that it looks like in our particular situation the mutation stays within the population and a mix of both sharing and aggressive animals will coexist.

import matplotlib.pyplot as plt import nashpy as nash import numpy as np

A = np.array([[2, 1], [3, 0]]) game = nash.Game(A) y0 = np.array([0.95, 0.05]) timepoints = np.linspace(0, 10, 1500) sharing_population, aggressive_population = game.replicator_dynamics(y0=y0, timepoints=timepoints).T plt.plot(sharing_population, label="$x_1$") plt.plot(aggressive_population, label="$x_2$") plt.ylim(0, 1) plt.ylabel("Population proportion") plt.xlabel("Time") plt.legend()

The replicator dynamics equation

Given a population with N types of individuals. Where the fitness of an individual of type i when interacting with an individual of type j is given by A_ij where A ∈ ℝ^N × N. The replicator dynamics equation is given by:

$$\frac{dx_i}{dt} = x_i(f_i - \phi)\text{ for all }i$$

where:

$$\phi = \sum_{i=1} ^ N x_i f_i(x)$$

where f_i is the population dependent fitness of individuals of type i:

f_i(x) = (Ax)_i

Note that there are equivalent linear algebraic definitions to the above:

f = Ax  ϕ = x^TAx

Question

For Rock Paper Scissors <motivating-example-strategy-for-rps>, what is the replicator dynamics equation?

Answer

Recalling that rock paper scissors has a payoff matrix A given by:

$$\begin{aligned} A = \begin{pmatrix} 0 & -1 & 1 \\\ 1 & 0 & -1\\\ -1 & 1 & 0\\\ \end{pmatrix} \end{aligned}$$

For a general population vector x the population dependent fitness f is given by:

$$\begin{aligned} f = Ax = \begin{pmatrix} -x_2 + x_3\\\ x_1 - x_3\\\ -x_1 + x_2\\\ \end{pmatrix} \end{aligned}$$

The average fitness is given by:

ϕ = x^Tf = x₁(x₃ − x₂) + x₂(x₁ − x₃) + x₃(x₂ − x₁)

The replicator dynamics equation is then given by:

$$\begin{aligned} \begin{align} \frac{dx_1}{dt} &= x_1(x_3 - x_2 - \phi)\\\ \frac{dx_2}{dt} &= x_2(x_1 - x_3 - \phi)\\\ \frac{dx_3}{dt} &= x_3(x_2 - x_1 - \phi) \end{align} \end{aligned}$$

Closer inspection of ϕ gives: ϕ = 0 thus:

$$\begin{aligned} \begin{align} \frac{dx_1}{dt} &= x_1(x_3 - x_2)\\\ \frac{dx_2}{dt} &= x_2(x_1 - x_3)\\\ \frac{dx_3}{dt} &= x_3(x_2 - x_1) \end{align} \end{aligned}$$

Stability of the replicator dynamics equation

Stability of the replicator dynamics equation is achieved when $\frac{dx_i}{dt} = 0$ for all i.

For a population vector x^* for which $\frac{dx^*_i}{dt} = 0$ for all i the population will not change without some other effect. This is referred to as a stable population.

Question

For the following games, what are the stable populations?

1. Rock Paper Scissors <motivating-example-strategy-for-rps>

$$\begin{aligned} A = \begin{pmatrix} 0 & -1 & 1 \\\ 1 & 0 & -1\\\ -1 & 1 & 0\\\ \end{pmatrix} \end{aligned}$$

2. Hawk Dove Game <motivating-example-replicator-dynamics>

$$\begin{aligned} A = \begin{pmatrix} 2 & 1\\\ 3 & 0 \end{pmatrix} \end{aligned}$$

Answer

1. The replicator dynamics equation for this game are:

$$\begin{aligned} \begin{align} \frac{dx_1}{dt} &= x_1(x_3 - x_2)\\\ \frac{dx_2}{dt} &= x_2(x_1 - x_3)\\\ \frac{dx_3}{dt} &= x_3(x_2 - x_1) \end{align} \end{aligned}$$

For them all to be 0, this requires:

• x₁ = 0 or x₂ = x₃
• x₂ = 0 or x₁ = x₃
• x₃ = 0 or x₁ = x₂

Which, through inspection in turn requires:

• x₁ ≠ 0 and x₂ = x₃ = 0 or
• x₂ ≠ 0 and x₁ = x₃ = 0 or
• x₃ ≠ 0 and x₁ = x₂ = 0 or
• x₁ = x₂ = x₃.

Given that x₁ + x₂ + x₃ = 1 this leaves us with 4 possible stable populations:

1. x = (1, 0, 0)
2. x = (0, 1, 0)
3. x = (0, 0, 1)
4. x = (1/3, 1/3, 1/3)

The following plot shows each of the above populations which no longer change over time:

import matplotlib.pyplot as plt import nashpy as nash import numpy as np

A = np.array([[0, -1, 1], [1, 0, -1], [-1, 1, 0]]) game = nash.Game(A) timepoints = np.linspace(0, 10, 1500) fig, axarr = plt.subplots(nrows=2, ncols=2)

initial_populations = (

np.array((1, 0, 0)), np.array((0, 1, 0)), np.array((0, 0, 1)), np.array((1/3, 1/3, 1/3)),

) for i, y0 in enumerate(initial_populations): rock_populations, paper_populations, scissors_populations = game.replicator_dynamics(y0=y0, timepoints=timepoints).T

ax = axarr[i % 2, int(i / 2)] ax.plot(rock_populations, label="$x_1$") ax.plot(paper_populations, label="$x_2$") ax.plot(scissors_populations, label="$x_3$") ax.set_ylim(-.1, 1.1) ax.set_ylabel("Population proportion") ax.set_xlabel("Time") ax.legend()

plt.tight_layout()

2. The replicator dynamics equation for this game are:

$$\begin{aligned} \begin{align} \frac{dx_1}{dt} &= x_1(2x_1 + x_2 - \phi)\\\ \frac{dx_2}{dt} &= x_2(3x_1 - \phi) \end{align} \end{aligned}$$

where:

ϕ = x₁(2x₁ + x₂) + x₂(3x₁)

substituting x₂ = 1 − x₁ here gives:

$$\begin{aligned} \begin{align} \frac{dx_1}{dt} &= x_1(x_1 - 1)(2x_1-1)\\\ \frac{dx_2}{dt} &= -x_1(x_1 - 1)(2x_1-1) \end{align} \end{aligned}$$

For them both to be 0, this requires:

• x₁ = 0 or
• x₁ = 1 or
• x₁ = 1/2

Recalling the substition that x₂ = 1 − x₁ this leaves us with 3 possible stable populations:

1. x = (1, 0)
2. x = (0, 1)
3. x = (1/2, 1/2)

The following plot shows each of the above populations which no longer change over time:

import matplotlib.pyplot as plt import nashpy as nash import numpy as np

A = np.array([[2, 1], [3, 0]]) game = nash.Game(A) timepoints = np.linspace(0, 10, 1500) fig, axarr = plt.subplots(nrows=1, ncols=3, figsize=(8, 3))

initial_populations = (

np.array((1, 0)), np.array((0, 1)), np.array((1 / 2, 1 / 2)),

) for i, y0 in enumerate(initial_populations): sharing_populations, aggressive_populations = game.replicator_dynamics(y0=y0, timepoints=timepoints).T

ax = axarr[i] ax.plot(sharing_populations, label="$x_1$") ax.plot(aggressive_populations, label="$x_2$") ax.set_ylim(-.1, 1.1) ax.set_ylabel("Population proportion") ax.set_xlabel("Time") ax.legend()

plt.tight_layout()

Evolutionary stable strategies

Evolutionary stable strategies are strategies that when adopted by an entire population are resistant to an alternative strategy that is initially rare.

By definition an evolutionary stable strategy corresponds to a stable population.

For the hawk dove game <motivating-example-replicator-dynamics> there are 3 stable populations:

• x = (1, 0)
• x = (0, 1)
• x = (1/2, 1/2)

However, if a small deviation is made from the first two populations then the population does not "resist". For example, we consider the initial population x = (1, 0) and introduce a small population aggressive behaviours to have: x = (1 − ϵ, ϵ) where ϵ > 0. The plot below shows this with ϵ = 10⁻5:

import matplotlib.pyplot as plt import nashpy as nash import numpy as np

A = np.array([[2, 1], [3, 0]]) game = nash.Game(A) epsilon = 10 ** -5 y0 = np.array([1 - epsilon, epsilon]) timepoints = np.linspace(0, 10, 10_000) sharing_population, aggressive_population = game.replicator_dynamics(y0=y0, timepoints=timepoints).T plt.plot(sharing_population, label="$x_1$") plt.plot(aggressive_population, label="$x_2$") plt.ylim(0, 1) plt.ylabel("Population proportion") plt.xlabel("Time") plt.legend()

This is also what happens if we start with a population of aggressive animals: We consider the initial population x = (0, 1) and introduce a small population aggressive behaviours to have: x = (ϵ, 1 − ϵ) where ϵ > 0. The plot below shows this with ϵ = 10^− 5:

import matplotlib.pyplot as plt import nashpy as nash import numpy as np

A = np.array([[2, 1], [3, 0]]) game = nash.Game(A) epsilon = 10 ** -5 y0 = np.array([epsilon, 1 - epsilon]) timepoints = np.linspace(0, 10, 10_000) sharing_population, aggressive_population = game.replicator_dynamics(y0=y0, timepoints=timepoints).T plt.plot(sharing_population, label="$x_1$") plt.plot(aggressive_population, label="$x_2$") plt.ylim(0, 1) plt.ylabel("Population proportion") plt.xlabel("Time") plt.legend()

However, this is not the case with the third stable population: x = (1/2, 1/2). The plot below shows x = (1/2 − ϵ, 1/2 + ϵ) with ϵ = 10^− 2:

import matplotlib.pyplot as plt import nashpy as nash import numpy as np

A = np.array([[2, 1], [3, 0]]) game = nash.Game(A) epsilon = 10 ** -2 y0 = np.array([1 / 2 - epsilon, 1 / 2 + epsilon]) timepoints = np.linspace(0, 2, 15) sharing_population, aggressive_population = game.replicator_dynamics(y0=y0, timepoints=timepoints).T plt.plot(sharing_population, label="$x_1$") plt.plot(aggressive_population, label="$x_2$") plt.ylim(0, 1) plt.ylabel("Population proportion") plt.xlabel("Time") plt.legend()

These observations can be confirmed analytically. Information on this can be found in [Fudenberg1998], [Webb2007] and [Nowak2006].

The replicator equations were first presented in [Maynard1974].

The replicator-mutation dynamics equation

An extension of the replicator equation <definition-of-the-replicator-dynamics-equation> is to allow for mutation [Komarova2004]. In this case reproduction is imperfect and individuals of a given type can give individuals of another.

This is expressed using a matrix Q where Q_ij denotes the probability of an individual of type j is produced by an individual of type i.

In this case the replicator equation can be modified to give the replicator-mutation equation:

$$\frac{dx_i}{dt} = \sum_{j=1}^Nx_j f_j Q_{ji}- x_i\phi\text{ for all }i$$

where, as before:

f = Ax  ϕ = x^TAx

This can modify emergent behaviour. For the Hawk Dove game <motivating-example-replicator-dynamics> if there is a 10% change that aggressive individuals will produce sharing ones the matrix Q is given by:

$$\begin{aligned} Q = \begin{pmatrix} 1 & 0\\\ 1 / 10 & 9 / 10 \end{pmatrix} \end{aligned}$$

The plot below shows the evolution of the system:

import matplotlib.pyplot as plt import nashpy as nash import numpy as np

A = np.array([[2, 1], [3, 0]]) Q = np.array([[1, 0], [1 / 10, 9 / 10]]) game = nash.Game(A) y0 = np.array([0.95, 0.05]) timepoints = np.linspace(0, 10, 1500) sharing_population, aggressive_population = game.replicator_dynamics(y0=y0, timepoints=timepoints, mutation_matrix=Q).T plt.plot(sharing_population, label="$x_1$") plt.plot(aggressive_population, label="$x_2$") plt.ylim(0, 1) plt.ylabel("Population proportion") plt.xlabel("Time") plt.legend()

Question

Show that for Q = I_N (the identity matrix of size N) the replicator-mutation equation corresponds to the replicator equation.

Answer

The replicator-mutation equation is:

$$\frac{dx_i}{dt} = \sum_{j=1}^Nx_j f_j Q_{ji}- x_i\phi\text{ for all }i$$

As Q = I_N:

$$\begin{aligned} Q_{ij} = \begin{cases} 1 & \text{ if } i = j\\\ 0 & \text{ otherwise} \end{cases} \end{aligned}$$

This gives:

$$\begin{aligned} \begin{align} \frac{dx_i}{dt} &= x_i f_i Q_{ii}- x_i\phi\text{ for all }i && Q_{ij}=0\text{ for all } i\ne j\\\ \frac{dx_i}{dt} &= x_i f_i - x_i\phi\text{ for all }i && Q_{ii}=1\\\ \frac{dx_i}{dt} &= x_i (f_i - \phi)\text{ for all }i \end{align} \end{aligned}$$

As required.

Using Nashpy

See how-to-use-replicator-dynamics for guidance of how to use Nashpy to obtain numerical solutions of the replicator dynamics equation. See how-to-use-replicator-dynamics-with-mutation for guidance of how to use Nashpy to obtain numerical solutions of the replicator-mutation dynamics equation.This is what is used to obtain all the plots above.