2.2 Bayesian Statistics

Bayes' Theorem is a fundamental concept in probability theory and statistics, and it offers a powerful way to update our belief or certainty about a hypothesis when we are presented with new evidence or information. It's particularly useful in situations where we need to revise our assumptions based on new data, which is often referred to as "knowledge actualization."

The formula is usually stated as:

$$ P(A|B) = \frac{P(B|A) \times P(A)}{P(B)} $$

Here’s what each term represents:

• $( P(A|B) )$ is the probability of event $A$ occurring given that $B$ is true. This is known as the posterior probability.
• $( P(B|A) )$ is the probability of event $B$ occurring given that $A$ is true. This is known as the likelihood.
• $( P(A) )$ is the probability of event $A$. This is known as the prior probability.
• $( P(B) )$ is the probability of event $B$. This is known as the marginal likelihood.

In the context of knowledge actualization and certainty:

1. Prior Probability ( $( P(A) )$ ): This represents our initial belief about the probability of an event (or the truth of a hypothesis) before we have any additional information.

2. Likelihood ( $( P(B|A) )$ ): This is the probability of observing the evidence $(B)$ given that our hypothesis $(A)$ is true. It reflects how we update our beliefs in light of new data.

3. Marginal Likelihood ( $( P(B) )$ ): This term normalizes the probability, ensuring that the probabilities sum up to $1$. It's essentially the probability of observing the evidence under all possible hypotheses.

4. Posterior Probability ( $( P(A|B) )$ ): This is what we are often trying to find out. It represents the updated belief about our hypothesis after taking into account the new evidence.

For example, in a medical context, Bayes' Theorem can be used to update the probability of a patient having a certain disease $(A)$ after getting a positive result on a diagnostic test $(B)$. Here, the prior probability would be the prevalence of the disease in the general population, the likelihood would be the accuracy of the test, and the posterior probability would be the updated probability of the patient having the disease after considering the test result.

In summary, Bayes' Theorem allows us to adjust our certainty or belief in a hypothesis (or the occurrence of an event) based on new evidence, making it a powerful tool for knowledge actualization in various fields.

Proof

The proof of Bayes' Theorem is rooted in the concepts of conditional probability and the definition of independence in probability theory. Let's go through it step by step.

Background Concepts:

1. Conditional Probability: The probability of an event A given that event B has occurred is denoted as ( $P(A|B)$ ) and is given by the formula: $$ P(A|B) = \frac{P(A \cap B)}{P(B)} $$ where ( P(A \cap B) ) is the probability that both A and B occur.

2. Multiplication Rule: The probability of both A and B occurring (the joint probability) is given by: $$ P(A \cap B) = P(A|B) \times P(B) $$ This can also be expressed as: $$ P(A \cap B) = P(B|A) \times P(A) $$ since the event of A and B occurring together doesn't depend on which event is considered first.

Proof of Bayes' Theorem:

Now, using these concepts, we can derive Bayes' Theorem. We know from the multiplication rule that:

$$ P(A \cap B) = P(A|B) \times P(B) $$

$$ P(A \cap B) = P(B|A) \times P(A) $$

Since ( $P(A \cap B)$ ) is the same regardless of how it's calculated, we can equate these two expressions:

$$ P(A|B) \times P(B) = P(B|A) \times P(A) $$

We are interested in finding ( $P(A|B)$ ), the probability of $A$ given $B$. So, rearranging the terms, we get:

$$ P(A|B) = \frac{P(B|A) \times P(A)}{P(B)} $$

And that's Bayes' Theorem! It shows how the probability of event A given B is related to the probability of event $B$ given $A$, the individual probabilities of A and B, and how these probabilities are adjusted based on the occurrence of the other event.

This theorem is a powerful tool in statistics and probability, as it allows for the updating of probabilities with new information, which is a cornerstone of inferential statistics and many applications in data science and machine learning.