Skip to content
The basic distribution probability Tutorial for Deep Learning Researchers
Python
Branch: master
Clone or download
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
graph complete Sep 7, 2019
LICENSE complete Sep 7, 2019
README.md Update README.md Sep 7, 2019
bernoulli.py init commit Sep 7, 2019
beta.py
binomial.py
categorical.py init commit Sep 7, 2019
chi-squared.py
dirichlet.py init commit Sep 7, 2019
exponential.py init commit Sep 7, 2019
gamma.py init commit Sep 7, 2019
gaussian.py init commit Sep 7, 2019
gmm.py complete Sep 7, 2019
multinomial.py init commit Sep 7, 2019
normal.py
overview.png edit #1 Squared sigma Sep 7, 2019
overview.pptx edit #1 Squared sigma Sep 7, 2019
student-t.py complete Sep 7, 2019
uniform.py init commit Sep 7, 2019

README.md

distribution-is-all-you-need

distribution-is-all-you-need is the basic distribution probability tutorial for most common distribution focused on Deep learning using python library.

Overview of distribution probability

distribution probabilities and features

  1. Uniform distribution(continuous), code
    • Uniform distribution has same probaility value on [a, b], easy probability.

  1. Bernoulli distribution(discrete), code
    • Bernoulli distribution is not considered about prior probability P(X). Therefore, if we optimize to the maximum likelihood, we will be vulnerable to overfitting.
    • We use binary cross entropy to classify binary classification. It has same form like taking a negative log of the bernoulli distribution.

  1. Binomial distribution(discrete), code
    • Binomial distribution with parameters n and p is the discrete probability distribution of the number of successes in a sequence of n independent experiments.
    • Binomial distribution is distribution considered prior probaility by specifying the number to be picked in advance.

  1. Multi-Bernoulli distribution, Categorical distribution(discrete), code
    • Multi-bernoulli called categorical distribution, is a probability expanded more than 2.
    • cross entopy has same form like taking a negative log of the Multi-Bernoulli distribution.

  1. Multinomial distribution(discrete), code
    • The multinomial distribution has the same relationship with the categorical distribution as the relationship between Bernoull and Binomial.

  1. Beta distribution(continuous), code
    • Beta distribution is conjugate to the binomial and Bernoulli distributions.
    • Using conjucation, we can get the posterior distribution more easily using the prior distribution we know.
    • Uniform distiribution is same when beta distribution met special case(alpha=1, beta=1).

  1. Dirichlet distribution(continuous), code
    • Dirichlet distribution is conjugate to the MultiNomial distributions.
    • If k=2, it will be Beta distribution.

  1. Gamma distribution(continuous), code
    • Gamma distribution will be beta distribution, if Gamma(a,1) / Gamma(a,1) + Gamma(b,1) is same with Beta(a,b).
    • The exponential distribution and chi-squared distribution are special cases of the gamma distribution.

  1. Exponential distribution(continuous), code
    • Exponential distribution is special cases of the gamma distribution when alpha is 1.

  1. Gaussian distribution(continuous), code
    • Gaussian distribution is a very common continuous probability distribution

  1. Normal distribution(continuous), code
    • Normal distribution is standarzed Gaussian distribution, it has 0 mean and 1 std.

  1. Chi-squared distribution(continuous), code
    • Chi-square distribution with k degrees of freedom is the distribution of a sum of the squares of k independent standard normal random variables.
    • Chi-square distribution is special case of Beta distribution

  1. Student-t distribution(continuous), code
    • The t-distribution is symmetric and bell-shaped, like the normal distribution, but has heavier tails, meaning that it is more prone to producing values that fall far from its mean.

Author

If you would like to see the details about relationship of distribution probability, please refer to this.

  • Tae Hwan Jung @graykode, Kyung Hee Univ CE(Undergraduate).
  • Author Email : nlkey2022@gmail.com
  • If you leave the source, you can use it freely.
You can’t perform that action at this time.