Lecture Notes of Probability Theory

This is a repository for materials written for the course on probability theory.

Our Progress

• Week 0

  • Introduction

• Week 1

  • Cardinality of sets
  • Ordinals
  • Axiom of choice
  • Tons of exercises

• Week 2

  • Cantor set, normal number
  • How to define a measures on $\mathbb{R}$?
  • Baire Category Theorem
  • Existence of unmeasurable sets

• Week 3

  • $\sigma$-algebra, $\pi$-system, Borel sets
  • $\sigma$-additivity and its consequence
  • Lebesgue measure, measurable sets,completion of measure space
  • Probability space, random variable
  • Carathéodory’s Extension Theorem

• Week 4

  • Definitions: Event, $(E_n, \text{i.o.}), (E_n, \text{ev})$, almost sure(a.s.)
  • Borel-Cantelli lemma (BC1)
  • Poincaré’s Recurrence Theorem
  • Continuity of measure, Fatou’s Lemma and Reverse Fatou Lemma
  • Measurable function, $\sigma$-algebra generated by r.v.

• Week 5

  • Doob–Dynkin Lemma
  • Independence

• Week 6

  • Lovasz Local Lemma
  • Product space
  • Kolmogorov Existence Theorem (This guarantees the existence of random process.)
  • Borel's 0-1 Law
  • Kolmogorov's 0-1 Law
  • Hewitt-Savage 0-1 Law

• Week 7

  • Definition: Lebesgue integration for $(m\Sigma)^+$
  • Monotone-Convergence Theorem(MON)
  • The Fatou Lemmas for functions
  • $\mu$-integrable functions: $\mathcal{L}^1(S,\Sigma, \mu)$
  • Linearity of integration

• Week 8

  • Standard machine
  • Radon-Nikodym Theorem
  • Dominance-Convergence Theorem
  • Modes of convergence
  • Scheffe's Lemma

• Week 9 (International Labor Day)

• Week 10

  • Supporting hyperplane theorem and Jensen's inequality
    • Fatou lemmas from (infinite-dimensional) Jensen's inequality.
  • Monotonicity of $\mathcal{L}^p$: Lyapounov's inequality and truncation method
  • Hölder's inequality
  • $\mathcal{L}^p / \sim$ is a Banach space: Minkowski's inequality (triangle inequality) and completeness
  • $\mathcal{L}^2$ is a Hilbert space: orthogonal projection
  • Covariance matrix as a Gram matrix of an inner product space

• Week 11

  • Calculate expectation from law
  • Probability density function(pdf)
  • Gaussian distribution
  • An introduction of Central Limit Theorem(CLT) and Berry-Esseen theorem
  • Bernstein polynomials and Weierstrass approximation theorem

• Week 12

  • Definition and existence of conditional expectation: Fundamental Theorem (Kolmogorov)
    • proof from Radon-Nikodym theorem
    • proof as least-squares-best predicator
  • Properties of conditional expectation
  • Definitions: filtration, adapted process, martingale
    • e.g. Doob martingale
  • Previsible process, martingale transform and (discrete) stochastic integral

• Week 13

  • Simple examples of martingale
    • Simple random walk: sum (or product) of I.I.D random variables
    • "ABRACAADABRA" (c.f. E10.6. of the textbook)
    • Make a submartingale from a martingale with Jensen's inequality
  • Stopping time (or Optional time)
  • A characterization of martingale using stopping time

• Week 14

  • More examples: Polya's Urn, branching process(c.f. Chapter 0), quadratic variation
  • Doob's maximal inequalities:
    • Doob's submartingale inequality
    • Doob's $\mathcal{L}^p$ inequality
  • Upcrossings and Doob's upcrossing lemma
  • Doob's 'forward' convergence theorem
    • e.g. 1D drunkard's walk
  • Doob decomposition
  • Uniform integrability
    • UI property: bridge between different modes of convergence
    • Sufficient conditions for UI property
    • UI property of conditional expectations

• Week 15

  • Bounded Convergence Theorem
  • Convergence in probability + UI property $\iff \mathcal{L}^1$ convergence
  • Lévy's 'Upward' Theorem
  • Lévy's 0-1 law
  • Martingale proof of Kolmogorov's 0-1 law

• Week 16

  • Radon-Nikodym derivative revisited
    • ${\frac{dP_n}{dQ_n}}$ is a martingale
    • Conditional expectation of $\frac{dP}{dQ}$
    • Calculate Radon-Nikodym derivative from pdf
    • Martingale proof of Radon-Nikodym Theorem (Meyer 1966)
  • Back to starting point: various interpretations of probability
    • Linear expectation
    • Sublinear expectation and its connection with linear expectation
    • Quantum expectation
    • An example from high-dimensional probability
  • Subadditive functions and rooted trees