1. Probability

From Data to Uncertainty: The Idea of Probability

Probability quantifies how plausible outcomes are before seeing data.

Probability is about describing uncertainty consistently.

$$0 \le P(A) \le 1,\quad P(\Omega)=1,\quad P(A\cup B)=P(A)+P(B)-P(A\cap B)$$

View	Example	Mental Model
Frequentist	40 defective parts in 1000 → 0.04	Long‑run relative frequency
Bayesian	Before testing, defect rate believed 4%	Degree of belief, updated with evidence
Geometric/Measure	Dart hits within region of area ratio	Probability as normalized measure

Probability isn’t “randomness,” it’s a language for incomplete information.

That is why we can’t say a single coin toss has a 50% “chance” of heads after seeing it. Because probability quantifies uncertainty before the outcome.

Events → Conditional Probability → Bayes (minimal toolkit)

Purpose: compute uncertainty correctly before any inference.

A) Events and sample space

An event E is a proposition, a statement, which we do not yet know whether it is true or false.

• Finetti

• Sample space $(\Omega)$: all possible outcomes.
• Event $(A \subseteq \Omega)$: a set of outcomes whose probability we track.
• Intersection $(A \cap B)$: both happen; union $(A \cup B)$: at least one happens; complement $(A^c)$: does not happen.

B) Conditional probability and independence

• Definition: $(P(A,|,B) = \dfrac{P(A \cap B)}{P(B)})$, for $(P(B)>0)$.
• Independence: $(A \perp B \iff P(A \cap B) = P(A),P(B) \iff P(A,|,B)=P(A))$.

C) Law of Total Probability (partition ({B_i}))

$$ P(A)=\sum_i P(A,|,B_i),P(B_i)\quad\text{with disjoint }B_i,\ \bigcup_i B_i=\Omega $$

D) Bayes’ theorem

$$ P(B,|,A)=\frac{P(A,|,B),P(B)}{P(A)}\quad\text{and with a partition }{B_i}:\quad P(B_k,|,A)=\frac{P(A,|,B_k),P(B_k)}{\sum_i P(A,|,B_i),P(B_i)} $$

E) Practical handles (with pandas)

Use counts → proportions; avoid means on nominal codes (see Scales page).

import pandas as pd

# Intention: estimate P(Disease | Test = "pos") from a table of outcomes.
df = pd.DataFrame({
    "test":    ["pos","pos","neg","pos","neg","neg","pos","neg"],
    "disease": [ True, False, False, True, False, False, True, False]
})

# Joint counts:
tab = pd.crosstab(df["disease"], df["test"])     # rows: disease; cols: test

# P(B) marginals (column proportions): P(test)
p_test = tab.sum(axis=0) / tab.values.sum()

# P(A|B): column-wise conditional P(disease | test)
p_dis_given_test = tab.div(tab.sum(axis=0), axis=1)

p_pos = p_test["pos"]
p_d_given_pos = p_dis_given_test.loc[True, "pos"]

print(f"P(test=pos) ≈ {p_pos:.3f}")
print(f"P(disease | test=pos) ≈ {p_d_given_pos:.3f}")

Checklist before computing:

1. Name events precisely (membership in a category, threshold on a ratio variable).
2. Pick the right denominator (condition first, then divide).
3. For group comparisons, report both rates and base rates.

Pitfalls (and fixes):

• Collapsing categories arbitrarily → define a proper partition before using total probability.

• Treating ordinal codes as interval → only order-based summaries should inform events (e.g., “≥ 4”).

• Ignoring class imbalance → always show $𝑃(𝐵)$ when reporting $P(A∣B)$.

Micro-exercises (with brief answers):

• Q1: A test has $(P(\text{pos}|\text{disease})=0.9), (P(\text{pos}|\text{healthy})=0.1)$. Disease prevalence $(P(\text{disease})=0.02). Compute (P(\text{disease}|\text{pos}))$.
  A1: $( \frac{0.9\cdot0.02}{0.9\cdot0.02 + 0.1\cdot0.98} \approx 0.155 )$ (15.5%).
• Q2: If $(P(A)=0.4), (P(B)=0.5), (P(A\cap B)=0.2)$, are $(A)$ and $(B)$ independent?
  A2: Yes, $(0.4\cdot0.5=0.2\Rightarrow)$ independent.
• Q3: You report “80% of severe cases among seniors.” What denominator is correct for $(P(\text{severe}|\text{senior}))$?
  A3: Seniors only (condition first, then divide by seniors).