Skip to content

Commit 05e9758

Browse files
shudipto-aminkazi-shudipto-aminleiosAmaras
authored
New chapter "Metropolis" with Python Implementation (#929)
* Add new chapter: metropolis_hastings; in python * Add markdown and python files for metropolis * Add image for metropolis * Update .gitignore, SUMMARY.md, and metropolis * Untrack .ipynb_checkpoints * Untrack ipynb_checkpoints * Fix algorithm steps list * Really fix markdown list * Add plot of P to chapter * Add metropolis animation and update plot of P(x) * Add minor update to chapter text * Generate gif and mp4 for random walk * Complete first draft! * Final version before Pull Request * Add metropolis citation * Remove unnecessary lines from code and bib file * Fix display of code in md * Fix random walk capitalization. * Apply Amaras' suggestions from code review This commit makes minor edits which resolve conflicts with PEP8 and improve readability of code. It also makes chapter sub-chapter of Monte Carlo. Co-authored-by: James Schloss <jrs.schloss@gmail.com> Co-authored-by: Sammy Plat <amaras@vivaldi.net> * Fix the code import lines in md file. * Change to in metropolis.py * Add probability section to metropolis.md However, this is temporary, as this section needs to go into a separate chapter. * Move Probability section in metropolis to own chapter. * Fix SUMMARY.md spelling mistake from previous commit * Add figures for probability distribution chapter * Update image of normal distribution * Finish first draft of probability chapter * Minor edits to distributions.md. * "Minor changes to distributions.md" * Complete the Example/Application section of metropolis. * Address most issues in review of PR#929 * Application sections created. * g() definition moved up to a better place * More discussion on g. * Missing citations provided * Clarity provided (hopefully) * Add image of 1D_particles * Update citations in md file * Add testing function to metropolis python code. Also modifies markdown to describe test. Also adds citation for RMSD. * Numpyfy metropolis f function and fix errors. * Implement generator to iterate. * Reformat output of test and nrmsd error reporting. * Add description of video and fix code display. * Update contents/probability/distributions/distributions.md Co-authored-by: James Schloss <jrs.schloss@gmail.com> * Update contents/probability/distributions/distributions.md Co-authored-by: James Schloss <jrs.schloss@gmail.com> * Update contents/probability/distributions/distributions.md Co-authored-by: James Schloss <jrs.schloss@gmail.com> * Update contents/probability/distributions/distributions.md Co-authored-by: James Schloss <jrs.schloss@gmail.com> * Update contents/probability/distributions/distributions.md Co-authored-by: James Schloss <jrs.schloss@gmail.com> * Update contents/probability/distributions/distributions.md Co-authored-by: James Schloss <jrs.schloss@gmail.com> * Update contents/probability/distributions/distributions.md Co-authored-by: James Schloss <jrs.schloss@gmail.com> * Update contents/probability/distributions/distributions.md Co-authored-by: James Schloss <jrs.schloss@gmail.com> * Update contents/probability/distributions/distributions.md Co-authored-by: James Schloss <jrs.schloss@gmail.com> * Put sentences in `metropolis.md` on separate lines. * Put sentences in distributions.md chapter on separate lines. * Addresses issues raised by Leios' 1st review of probability chapter. * Adds intro to beginning. * Fixes plurality of die. * Clarifies some notation in discrete section. * Removes explanation of calculus. * Other minor edits. * Add minor edits. * Update contents/metropolis/metropolis.md title Co-authored-by: James Schloss <jrs.schloss@gmail.com> * Simplify intro to contents/metropolis/metropolis.md Co-authored-by: James Schloss <jrs.schloss@gmail.com> * Minor formatting to contents/metropolis/metropolis.md Co-authored-by: James Schloss <jrs.schloss@gmail.com> * Fix spelling contents/metropolis/metropolis.md Co-authored-by: James Schloss <jrs.schloss@gmail.com> * Simplify line 71 of contents/metropolis/metropolis.md Co-authored-by: James Schloss <jrs.schloss@gmail.com> * Update contents/metropolis/metropolis.md Co-authored-by: James Schloss <jrs.schloss@gmail.com> * Update example application in metropolis according to Leios comments. * Minor edit: contents/probability/distributions/distributions.md Co-authored-by: James Schloss <jrs.schloss@gmail.com> * Minor edit: contents/probability/distributions/distributions.md Co-authored-by: James Schloss <jrs.schloss@gmail.com> * Minor edit: contents/probability/distributions/distributions.md Co-authored-by: James Schloss <jrs.schloss@gmail.com> * Minor edit: contents/probability/distributions/distributions.md Co-authored-by: James Schloss <jrs.schloss@gmail.com> * Minor edit: contents/probability/distributions/distributions.md Co-authored-by: James Schloss <jrs.schloss@gmail.com> * Apply minor suggestions from Leios' code review Mostly typos, spelling, punctuation, and grammar. Co-authored-by: James Schloss <jrs.schloss@gmail.com> * Add minor edits from code Leios' review * Add minor edit * Apply minor edit suggestions from Leios Mostly punctuation and grammar. Co-authored-by: James Schloss <jrs.schloss@gmail.com> * Add minor formatting, such as periods after equations. * Fix integer interval notation using hack. * Add minor edits missed previously in probability chapter. * Add minor edits to metropolis, mostly punctuation. * Apply minor edits from Leios Mostly grammar and sentence structure. Co-authored-by: James Schloss <jrs.schloss@gmail.com> * Add intro line to Algorithm section of metropolis. * Add name to contributor.md * Apply suggestions from Leios * Minor edits for grammar and clarity. * Applies to both metropolis and probablity chapters. Co-authored-by: James Schloss <jrs.schloss@gmail.com> * Apply suggestions from code review Co-authored-by: Shudipto <32040453+shudipto-amin@users.noreply.github.com> Co-authored-by: Kazi Shudipto Amin <kazi.amin@ucalgary.ca> Co-authored-by: James Schloss <jrs.schloss@gmail.com> Co-authored-by: Sammy Plat <amaras@vivaldi.net>
1 parent ffaa890 commit 05e9758

16 files changed

+692
-1
lines changed

.gitignore

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -478,7 +478,7 @@ paket-files/
478478
# Python Tools for Visual Studio (PTVS)
479479
__pycache__/
480480
*.pyc
481-
481+
*.ipynb_checkpoints*
482482
# Cake - Uncomment if you are using it
483483
# tools/**
484484
# !tools/packages.config

CONTRIBUTORS.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -61,4 +61,5 @@ This file lists everyone, who contributed to this repo and wanted to show up her
6161
- Hugo Salou
6262
- Dimitri Belopopsky
6363
- Henrik Abel Christensen
64+
- K. Shudipto Amin
6465
- Peanutbutter_Warrior

SUMMARY.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -20,9 +20,11 @@
2020
* [Multiplication as a Convolution](contents/convolutions/multiplication/multiplication.md)
2121
* [Convolutions of Images (2D)](contents/convolutions/2d/2d.md)
2222
* [Convolutional Theorem](contents/convolutions/convolutional_theorem/convolutional_theorem.md)
23+
* [Probability Distributions](contents/probability/distributions/distributions.md)
2324
* [Tree Traversal](contents/tree_traversal/tree_traversal.md)
2425
* [Euclidean Algorithm](contents/euclidean_algorithm/euclidean_algorithm.md)
2526
* [Monte Carlo](contents/monte_carlo_integration/monte_carlo_integration.md)
27+
* [Metropolis](contents/metropolis/metropolis.md)
2628
* [Matrix Methods](contents/matrix_methods/matrix_methods.md)
2729
* [Gaussian Elimination](contents/gaussian_elimination/gaussian_elimination.md)
2830
* [Thomas Algorithm](contents/thomas_algorithm/thomas_algorithm.md)
Lines changed: 90 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,90 @@
1+
import numpy as np
2+
3+
4+
def f(x, normalize=False):
5+
'''
6+
Function proportional to target distribution, a sum of Gaussians.
7+
For testing, set normalize to True, to get target distribution exactly.
8+
'''
9+
# Gaussian heights, width parameters, and mean positions respectively:
10+
a = np.array([10., 3., 1.]).reshape(3, 1)
11+
b = np.array([ 4., 0.2, 2.]).reshape(3, 1)
12+
xs = np.array([-4., -1., 5.]).reshape(3, 1)
13+
14+
if normalize:
15+
norm = (np.sqrt(np.pi) * (a / np.sqrt(b))).sum()
16+
a /= norm
17+
18+
return (a * np.exp(-b * (x - xs)**2)).sum(axis=0)
19+
20+
def g():
21+
'''Random step vector.'''
22+
return np.random.uniform(-1,1)
23+
24+
def metropolis_step(x, f=f, g=g):
25+
'''Perform one full iteration and return new position.'''
26+
27+
x_proposed = x + g()
28+
a = min(1, (f(x_proposed) / f(x)).item())
29+
30+
x_new = np.random.choice([x_proposed, x], p=[a, 1-a])
31+
32+
return x_new
33+
34+
def metropolis_iterate(x0, num_steps):
35+
'''Iterate metropolis algorithm for num_steps using iniital position x_0'''
36+
37+
for n in range(num_steps):
38+
if n == 0:
39+
x = x0
40+
else:
41+
x = metropolis_step(x)
42+
yield x
43+
44+
45+
def test_metropolis_iterate(num_steps, xmin, xmax, x0):
46+
'''
47+
Calculate error in normalized density histogram of data
48+
generated by metropolis_iterate() by using
49+
normalized-root-mean-square-deviation metric.
50+
'''
51+
52+
bin_width = 0.25
53+
bins = np.arange(xmin, xmax + bin_width/2, bin_width)
54+
centers = np.arange(xmin + bin_width/2, xmax, bin_width)
55+
56+
true_values = f(centers, normalize=True)
57+
mean_value = np.mean(true_values - min(true_values))
58+
59+
x_dat = list(metropolis_iterate(x0, num_steps))
60+
heights, _ = np.histogram(x_dat, bins=bins, density=True)
61+
62+
nmsd = np.average((heights - true_values)**2 / mean_value)
63+
nrmsd = np.sqrt(nmsd)
64+
65+
return nrmsd
66+
67+
68+
69+
if __name__ == "__main__":
70+
xmin, xmax = -10, 10
71+
x0 = np.random.uniform(xmin, xmax)
72+
73+
num_steps = 50_000
74+
75+
x_dat = list(metropolis_iterate(x0, 50_000))
76+
77+
# Write data to file
78+
output_string = "\n".join(str(x) for x in x_dat)
79+
80+
with open("output.dat", "w") as out:
81+
out.write(output_string)
82+
out.write("\n")
83+
84+
85+
# Testing
86+
print(f"Testing with x0 = {x0:5.2f}")
87+
print(f"{'num_steps':>10s} {'NRMSD':10s}")
88+
for num_steps in (500, 5_000, 50_000):
89+
nrmsd = test_metropolis_iterate(num_steps, xmin, xmax, x0)
90+
print(f"{num_steps:10d} {nrmsd:5.1%}")

contents/metropolis/metropolis.md

Lines changed: 283 additions & 0 deletions
Large diffs are not rendered by default.
23 KB
Loading
1.57 MB
Loading
564 KB
Binary file not shown.
41.5 KB
Loading
20.2 KB
Binary file not shown.

0 commit comments

Comments
 (0)