Skip to content

Commit

Permalink
update : Examples.ipynb. (#2)
Browse files Browse the repository at this point in the history
  • Loading branch information
@sadrasabouri
sadrasabouri committed Jun 2, 2021
1 parent edc3774 commit 9bb1b47
Showing 1 changed file with 297 additions and 2 deletions.
299 changes: 297 additions & 2 deletions Document/Examples.ipynb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -10,9 +10,18 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": 3
"version": "3.8.5"
},
"orig_nbformat": 2
"orig_nbformat": 2,
"kernelspec": {
"name": "python385jvsc74a57bd0916dbcbb3f70747c44a77c7bcd40155683ae19c65e1c03b4aa3499c5328201f1",
"display_name": "Python 3.8.5 64-bit"
},
"metadata": {
"interpreter": {
"hash": "916dbcbb3f70747c44a77c7bcd40155683ae19c65e1c03b4aa3499c5328201f1"
}
}
},
"nbformat": 4,
"nbformat_minor": 2,
Expand Down Expand Up @@ -40,6 +49,292 @@
"### Version : 0.09\n",
"-----"
]
},
{
"source": [
"This example set contains bellow examples from the first reference (Introduction to Stochastic Processes):\n",
"<ul>\n",
" <ol>\n",
" <li><a href=\"#Finite_Markov_Chains\">Finite_Markov_Chains</a></li>\n",
" <ul>\n",
" <li><a href=\"#Exercise1_2\">#Exercise1_2</a></li>\n",
" <li><a href=\"#Exercise1_3\">#Exercise1_3</a></li>\n",
" <li><a href=\"#Exercise1_4\">#Exercise1_4</a></li>\n",
" </ul>\n",
" </ol>\n",
"</ul>"
],
"cell_type": "markdown",
"metadata": {}
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"Requirement already satisfied: pyrandwalk in /usr/local/lib/python3.8/dist-packages/pyrandwalk-0.9-py3.8.egg (0.9)\n",
"Requirement already satisfied: art>=1.8 in /home/sadra/.local/lib/python3.8/site-packages (from pyrandwalk) (5.1)\n",
"Requirement already satisfied: matplotlib>=3.0.0 in /home/sadra/.local/lib/python3.8/site-packages (from pyrandwalk) (3.3.4)\n",
"Requirement already satisfied: networkx>=2 in /home/sadra/.local/lib/python3.8/site-packages (from pyrandwalk) (2.5)\n",
"Requirement already satisfied: numpy>=1.9.0 in /home/sadra/.local/lib/python3.8/site-packages (from pyrandwalk) (1.20.1)\n",
"Requirement already satisfied: pillow>=6.2.0 in /usr/lib/python3/dist-packages (from matplotlib>=3.0.0->pyrandwalk) (7.0.0)\n",
"Requirement already satisfied: cycler>=0.10 in /home/sadra/.local/lib/python3.8/site-packages (from matplotlib>=3.0.0->pyrandwalk) (0.10.0)\n",
"Requirement already satisfied: python-dateutil>=2.1 in /usr/lib/python3/dist-packages (from matplotlib>=3.0.0->pyrandwalk) (2.7.3)\n",
"Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.3 in /home/sadra/.local/lib/python3.8/site-packages (from matplotlib>=3.0.0->pyrandwalk) (2.4.7)\n",
"Requirement already satisfied: kiwisolver>=1.0.1 in /home/sadra/.local/lib/python3.8/site-packages (from matplotlib>=3.0.0->pyrandwalk) (1.3.1)\n",
"Requirement already satisfied: decorator>=4.3.0 in /home/sadra/.local/lib/python3.8/site-packages (from networkx>=2->pyrandwalk) (4.4.2)\n",
"Requirement already satisfied: six in /usr/lib/python3/dist-packages (from cycler>=0.10->matplotlib>=3.0.0->pyrandwalk) (1.14.0)\n"
]
}
],
"source": [
"!pip install pyrandwalk\n",
"from pyrandwalk import *\n",
"import numpy as np"
]
},
{
"source": [
"## 1. Finite_Markov_Chains"
],
"cell_type": "markdown",
"metadata": {}
},
{
"source": [
"### Exercise1_2\n",
"\n",
"Consider a Markov chain with state space \\{0, 1\\} and transition matrix\n",
"\n",
"$$\n",
"P =\n",
"\\left(\\begin{array}{cc} \n",
"1/3 & 2/3\\\\\n",
"3/4 & 1/4\n",
"\\end{array}\\right)\n",
"$$"
],
"cell_type": "markdown",
"metadata": {}
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [],
"source": [
"states = [0, 1]\n",
"trans = np.array([[1/3, 2/3],\n",
" [3/4, 1/4]])\n",
"rw = RandomWalk(states, trans)"
]
},
{
"source": [
"Assuming that the chain starts in state 0 at time n = 0, what is the probability that it is in state 1 at time n = 3?"
],
"cell_type": "markdown",
"metadata": {}
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"[[0.49537037 0.50462963]\n [0.56770833 0.43229167]]\nANSWER: 0.5046296296296297\n"
]
}
],
"source": [
"third_step_trans = rw.trans_power(3)\n",
"print(third_step_trans)\n",
"print(\"ANSWER:\", third_step_trans[0, 1])"
]
},
{
"source": [
"### Exercise1_3\n",
"\n",
"Consider a Markov chain with state space \\{1, 2, 3\\} and transition matrix\n",
"\n",
"$$\n",
"P =\n",
"\\left(\\begin{array}{cc} \n",
"0.4 & 0.2 & 0.4\\\\\n",
"0.6 & 0 & 0.4 \\\\\n",
"0.2 & 0.5 & 0.3\n",
"\\end{array}\\right)\n",
"$$\n"
],
"cell_type": "markdown",
"metadata": {}
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [],
"source": [
"states = [1, 2, 3]\n",
"trans = np.array([[0.4, 0.2, 0.4],\n",
" [0.6, 0 , 0.4],\n",
" [0.2, 0.5, 0.3]])\n",
"rw = RandomWalk(states, trans)"
]
},
{
"source": [
"what is the probability in the long run that the chain is in state 1?\n",
"Solve this problem two different ways:\n",
"\n",
"1) by raising the matrix to a high power:"
],
"cell_type": "markdown",
"metadata": {}
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {},
"outputs": [
{
"output_type": "execute_result",
"data": {
"text/plain": [
"array([[0.37878788, 0.25757576, 0.36363636],\n",
" [0.37878788, 0.25757576, 0.36363636],\n",
" [0.37878788, 0.25757576, 0.36363636]])"
]
},
"metadata": {},
"execution_count": 5
}
],
"source": [
"rw.trans_power(1000)"
]
},
{
"source": [
"2) by directly computing the invariant probability vector as a left eigenvector:"
],
"cell_type": "markdown",
"metadata": {}
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {},
"outputs": [
{
"output_type": "execute_result",
"data": {
"text/plain": [
"array([0.37878788, 0.25757576, 0.36363636])"
]
},
"metadata": {},
"execution_count": 6
}
],
"source": [
"rw.final_dist()"
]
},
{
"source": [
"### Exercise1_4\n",
"\n",
"Do the same with\n",
"\n",
"$$\n",
"P =\n",
"\\left(\\begin{array}{cc} \n",
"0.2 & 0.4 & 0.4\\\\\n",
"0.1 & 0.5 & 0.4 \\\\\n",
"0.6 & 0.3 & 0.1\n",
"\\end{array}\\right)\n",
"$$\n"
],
"cell_type": "markdown",
"metadata": {}
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {},
"outputs": [],
"source": [
"states = [1, 2, 3]\n",
"trans = np.array([[0.2, 0.4, 0.4],\n",
" [0.1, 0.5, 0.4],\n",
" [0.6, 0.3, 0.1]])\n",
"rw = RandomWalk(states, trans)"
]
},
{
"source": [
"1) by raising the matrix to a high power:"
],
"cell_type": "markdown",
"metadata": {}
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {},
"outputs": [
{
"output_type": "execute_result",
"data": {
"text/plain": [
"array([[0.28205128, 0.41025641, 0.30769231],\n",
" [0.28205128, 0.41025641, 0.30769231],\n",
" [0.28205128, 0.41025641, 0.30769231]])"
]
},
"metadata": {},
"execution_count": 8
}
],
"source": [
"rw.trans_power(1000)"
]
},
{
"source": [
"2) by directly computing the invariant probability vector as a left eigenvector:"
],
"cell_type": "markdown",
"metadata": {}
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {},
"outputs": [
{
"output_type": "execute_result",
"data": {
"text/plain": [
"array([0.28205128, 0.41025641, 0.30769231])"
]
},
"metadata": {},
"execution_count": 9
}
],
"source": [
"rw.final_dist()"
]
}
]
}

0 comments on commit 9bb1b47

Please sign in to comment.