Probability Distributions Explained: Intuition, Math, and Python Examples (Complete Guide)
Probability distributions are at the core of data science and machine learning. Whether you’re analyzing data, building models, or answering technical interviews, a thorough understanding of distributions empowers you to make sense of the randomness in the real world. In this comprehensive guide, you’ll learn the intuition, mathematics, and Python tools for the 12 most important probability distributions. Let’s demystify distributions together—from the basics to practical code.
1. Why Distributions Matter in Data Science & Machine Learning
To model uncertainty and make informed predictions, data scientists rely on probability distributions. Here’s why mastering distributions is essential:
- Everything Is a Random Variable: Every dataset contains elements subject to chance—from customer behavior to server response times.
- Modeling Uncertainty: Distributions let you quantify unknowns, estimate probabilities, and model rare events.
- Interview Relevance: Distributions are a staple of data science interviews because they test both conceptual understanding and practical skills.
By mastering distributions, you unlock a solid analytical toolkit that’s vital for statistics, machine learning, and real-world problem-solving.
2. What Is a Random Variable?
Discrete vs Continuous
A random variable is a variable that can take on different values, each with a certain probability. Random variables come in two types:
- Discrete: Can only take specific, separate values (e.g., number of coin tosses until first heads).
- Continuous: Can take any value within a range (e.g., height in centimeters).
PMF vs PDF vs CDF
- PMF (Probability Mass Function): Applies to discrete variables. Gives the probability that the variable equals a certain value:
\( P(X = k) \) - PDF (Probability Density Function): Applies to continuous variables. Represents the density of probability at each point:
\( f(x) \) (probability within an interval is the area under the curve) - CDF (Cumulative Distribution Function): Gives the probability that the variable is less than or equal to a value:
\( F(x) = P(X \leq x) \)
Numeric Example
Suppose you toss a fair die. The number shown is a discrete variable: 1, 2, 3, 4, 5, or 6.
- PMF: \( P(X = k) = \frac{1}{6} \) for \( k = 1, ..., 6 \)
- CDF: \( F(3) = P(X \leq 3) = P(X=1) + P(X=2) + P(X=3) = \frac{3}{6} = 0.5 \)
For height (a continuous variable), the PDF tells you the relative likelihood at any given height, and probabilities are found by integrating the PDF over intervals.
3. Deep Dive Into the 12 Most Important Distributions
Let’s explore the world’s foundational probability distributions—from intuition to math and Python code.
1. Bernoulli Distribution
- Intuition: Simple “yes/no”, “success/failure” experiment.
- Formula:
\( P(X = k) = p^k (1-p)^{1-k} \) where \( k=0 \) or \( 1 \) - Parameters:
- p = probability of success
- Real-life Example: Flipping a coin (heads = 1, tails = 0).
- Python Example:
import numpy as np
p = 0.7
bernoulli_sample = np.random.binomial(1, p, 10)
print(bernoulli_sample)
- Graph: Two spikes at 0 and 1, with heights \( 1-p \) and \( p \) respectively.
2. Binomial Distribution
- Intuition: Number of successes in a fixed number of independent Bernoulli trials.
- Formula:
\( P(X = k) = \binom{n}{k} p^k (1-p)^{n-k} \) for \( k=0,1,\ldots,n \) - Parameters:
- n = number of trials, p = probability of success
- Real-life Example: Number of heads in 10 coin tosses.
- Python Example:
from scipy.stats import binom
import matplotlib.pyplot as plt
n, p = 10, 0.5
x = np.arange(0, n+1)
prob = binom.pmf(x, n, p)
plt.stem(x, prob)
plt.xlabel('Number of Successes')
plt.ylabel('Probability')
plt.title('Binomial Distribution')
plt.show()
- Graph: Discrete spikes, usually bell-shaped for large n and p ≈ 0.5.
3. Poisson Distribution
- Intuition: Counts number of events in a fixed interval when events happen independently at a constant average rate.
- Formula:
\( P(X = k) = \frac{\lambda^k e^{-\lambda}}{k!} \) - Parameters:
- \(\lambda\) = expected number of events per interval
- Real-life Example: Number of cars passing a checkpoint in an hour.
- Python Example:
from scipy.stats import poisson
x = np.arange(0, 15)
lam = 4
prob = poisson.pmf(x, lam)
plt.stem(x, prob)
plt.title('Poisson Distribution')
plt.xlabel('Events')
plt.ylabel('Probability')
plt.show()
- Graph: Discrete, right-skewed if \(\lambda\) small, approaches bell shape as \(\lambda\) increases.
4. Geometric Distribution
- Intuition: Number of trials until the first success.
- Formula:
\( P(X=k) = (1-p)^{k-1}p \), for \( k=1,2,\ldots \) - Parameters:
- p = probability of success each trial
- Real-life Example: Rolling a die until you get a six.
- Python Example:
from scipy.stats import geom
p = 0.2
x = np.arange(1, 11)
prob = geom.pmf(x, p)
plt.stem(x, prob)
plt.title('Geometric Distribution')
plt.xlabel('Trial')
plt.ylabel('Probability')
plt.show()
- Graph: Decreases exponentially with increasing trial number.
5. Uniform Distribution
- Intuition: All outcomes in an interval are equally likely.
- Formula:
For continuous uniform: \( f(x) = \frac{1}{b-a} \) for \( a \leq x \leq b \) - Parameters:
- a = lower bound, b = upper bound
- Real-life Example: Picking a random point on a stick of length 1.
- Python Example:
from scipy.stats import uniform
a, b = 0, 1
x = np.linspace(a, b, 100)
pdf = uniform.pdf(x, a, b-a)
plt.plot(x, pdf)
plt.title('Uniform Distribution')
plt.xlabel('Value')
plt.ylabel('Density')
plt.show()
- Graph: Constant horizontal line between a and b.
6. Normal (Gaussian) Distribution
- Intuition: Bell curve; many real phenomena approximate normality by central limit theorem.
- Formula:
\( f(x) = \frac{1}{\sqrt{2\pi\sigma^2}} e^{-\frac{(x-\mu)^2}{2\sigma^2}} \) - Parameters:
- \(\mu\) = mean, \(\sigma^2\) = variance
- Real-life Example: Heights, test scores, errors.
- Python Example:
from scipy.stats import norm
mu, sigma = 0, 1
x = np.linspace(-4, 4, 100)
pdf = norm.pdf(x, mu, sigma)
plt.plot(x, pdf)
plt.title('Normal Distribution')
plt.xlabel('Value')
plt.ylabel('Density')
plt.show()
- Graph: Symmetrical bell-shaped curve.
7. Exponential Distribution
- Intuition: Time between events in a Poisson process.
- Formula:
\( f(x) = \lambda e^{-\lambda x} \) for \( x \geq 0 \) - Parameters:
- \(\lambda\) = rate parameter
- Real-life Example: Time between incoming calls at a call center.
- Python Example:
from scipy.stats import expon
lam = 2
x = np.linspace(0, 3, 100)
pdf = expon.pdf(x, scale=1/lam)
plt.plot(x, pdf)
plt.title('Exponential Distribution')
plt.xlabel('Time')
plt.ylabel('Density')
plt.show()
- Graph: Sharp descent from the y-axis, decreases rapidly as x increases.
8. Gamma Distribution
- Intuition: Sum of multiple exponential random variables.
- Formula:
\( f(x) = \frac{\lambda^k x^{k-1} e^{-\lambda x}}{\Gamma(k)} \) for \( x\geq 0 \) - Parameters:
- k = shape, \(\lambda\) = rate
- Real-life Example: Modeling total waiting times for multiple Poisson events.
- Python Example:
from scipy.stats import gamma
k, lam = 2, 1.5
x = np.linspace(0, 10, 100)
pdf = gamma.pdf(x, k, scale=1/lam)
plt.plot(x, pdf)
plt.title('Gamma Distribution')
plt.xlabel('Value')
plt.ylabel('Density')
plt.show()
- Graph: Right-skewed curve, shape depends on k and λ.
9. Beta Distribution
- Intuition: Distribution of probabilities; good for modeling beliefs.
- Formula:
\( f(x) = \frac{x^{\alpha - 1} (1-x)^{\beta - 1}}{B(\alpha, \beta)} \) for \( 0 \leq x \leq 1 \) - Parameters:
- \(\alpha\), \(\beta\) = shape parameters
- Real-life Example: Probability an email is spam, after observing some data.
- Python Example:
from scipy.stats import beta
a, b = 2, 5
x = np.linspace(0, 1, 100)
pdf = beta.pdf(x, a, b)
plt.plot(x, pdf)
plt.title('Beta Distribution')
plt.xlabel('Probability')
plt.ylabel('Density')
plt.show()
- Graph: Flexible: can be U-shaped, uniform, or bell-shaped depending on α, β.
