Standard discrete and continuous distributions notes — Unit 3

In Unit 2, we learned how to analyze arbitrary, custom probability functions. However, the real world is highly structured. Specific types of random events (like tossing coins, customer arrivals, or measurement errors) consistently follow specific, predictable mathematical patterns.

Statisticians have formalized these patterns into Standard Distributions. Instead of deriving integrals from scratch every time, we identify the physical scenario, map it to a standard distribution, and instantly use its pre-calculated formulas for Mean and Variance.

The Binomial distribution models the number of "Successes" in a fixed sequence of independent Yes/No trials.

⇒2.1 The Bernoulli Trial Requirements

To use the Binomial distribution, the experiment MUST strictly satisfy four conditions:

• 1. The number of trials ($n$) is fixed in advance.
• 2. Each trial has exactly two possible outcomes: "Success" or "Failure".
• 3. The trials are completely Independent. (e.g., drawing cards with replacement).
• 4. The probability of Success ($p$) remains perfectly constant from trial to trial.

⇒2.2 The Mathematics

Let $X$ be the number of successes in $n$ trials.
Parameters: $n$ (trials), $p$ (probability of success), $q = 1-p$ (probability of failure).

PMF Formula: $P(X = x) = \binom{n}{x} p^x q^{n-x}$ (for $x = 0, 1, ..., n$)

Mean ($E[X]$): $np$
Variance ($Var(X)$): $npq$

Example: A factory produces CPUs with a 5% defect rate ($p=0.05$). If we test 100 CPUs ($n=100$), the probability of finding exactly 2 defects is $P(X=2) = \binom{100}{2} (0.05)^2 (0.95)^{98}$.

The Poisson distribution models the number of independent events occurring within a fixed interval of Time or Space.

It is the backbone of queueing theory and network traffic analysis.

⇒3.1 Requirements and Applications

Used when events happen continuously and independently at a known constant average rate, but the exact timing is totally random.

Examples:
- Number of HTTP requests hitting a server per minute.
- Number of typos on a printed page.
- Number of alpha particles emitted by radioactive material per second.

⇒3.2 The Mathematics

Let $X$ be the number of occurrences.
Parameter: $\lambda$ (Lambda), the average rate of occurrence per interval.

PMF Formula: $P(X = x) = \frac{e^{-\lambda} \lambda^x}{x!}$ (for $x = 0, 1, 2, ... \infty$)

Mean ($E[X]$): $\lambda$
Variance ($Var(X)$): $\lambda$

(A unique property of Poisson is that its Mean and Variance are always mathematically identical).

The Binomial formula requires calculating massive factorials like $\binom{10000}{3}$. For huge numbers, CPUs suffer from floating-point overflow.

⇒4.1 The Limiting Case

Siméon Denis Poisson proved that if a Binomial experiment has a massive number of trials ($n \rightarrow \infty$) and the probability of success is extremely tiny ($p \rightarrow 0$), the Binomial distribution mathematically transforms into the Poisson distribution.

We can use Poisson to approximate Binomial when:
$n \ge 50$ and $p \le 0.1$.

To approximate, we simply set the Poisson rate $\lambda$ equal to the Binomial mean:
$\lambda = np$

Then use the easy Poisson formula $P(X=x) = \frac{e^{-\lambda} \lambda^x}{x!}$ instead of the heavy Binomial combinatorics.

The simplest continuous distribution. It models scenarios where every single real number in a specific interval $[a, b]$ is equally likely to occur.

It is the mathematical definition of "pure, unbiased randomness". All computer `random()` functions generate numbers following a Uniform Distribution between $[0.0, 1.0]$.

⇒5.1 The Mathematics

Parameters: $a$ (minimum limit) and $b$ (maximum limit).

PDF Formula:
$f(x) = \begin{cases} \frac{1}{b-a} & \text{for } a \le x \le b \\ 0 & \text{otherwise} \end{cases}$

Because the probability is spread perfectly evenly, the PDF graphs as a flat rectangle. The height must be $1/(b-a)$ to ensure the total area equals 1.

Mean ($E[X]$): $\frac{a+b}{2}$
Variance ($Var(X)$): $\frac{(b-a)^2}{12}$

The Exponential distribution is the continuous twin of the discrete Poisson distribution. While Poisson counts the number of events per minute, Exponential models the exact continuous amount of time you must wait between those events.

It is the standard model for Reliability Engineering (e.g., how long until a hard drive permanently fails) and Queueing Theory (wait time for a packet).

⇒6.1 The Mathematics

Parameter: $\lambda$ (the constant rate of occurrence, exact same $\lambda$ as Poisson).

PDF Formula:
$f(x) = \begin{cases} \lambda e^{-\lambda x} & \text{for } x \ge 0 \\ 0 & \text{for } x < 0 \end{cases}$

Because it requires integration to find probabilities, the pre-integrated CDF is highly useful:
CDF Formula: $P(X \le x) = 1 - e^{-\lambda x}$

Mean ($E[X]$): $\frac{1}{\lambda}$
Variance ($Var(X)$): $\frac{1}{\lambda^2}$

Example: If hard drives fail at an average rate of 2 per year ($\lambda=2$), the average time until failure (Mean) is $1/2$ year (6 months).

The Exponential distribution possesses a profound, counter-intuitive mathematical trait: It has absolutely no memory of the past.

⇒7.1 The Definition

Mathematically: $P(X > s+t \mid X > s) = P(X > t)$

In English: Suppose a lightbulb's lifespan is Exponentially distributed. You have already used the lightbulb for 500 hours, and it hasn't broken yet ($X > s$). What is the probability it will survive for another 100 hours ($t$)?

The Memoryless property dictates that the probability is exactly the same as a brand new, straight-out-of-the-box lightbulb surviving its first 100 hours. The used lightbulb does not mathematically "age" or "wear out". The past 500 hours are completely irrelevant to its future.

The Exponential distribution is the ONLY continuous distribution in the universe that possesses this property.

The King of all distributions. The famous "Bell Curve".

It is the most important distribution because of the Central Limit Theorem: virtually any naturally occurring data in the universe (human heights, IQ scores, measurement errors, noise in electronic circuits) naturally shapes itself into a Normal distribution if the sample size is large enough.

⇒8.1 The Mathematics

Parameters: $\mu$ (the exact center Mean) and $\sigma^2$ (the Variance/spread).

The PDF is the most complex formula we use:
$f(x) = \frac{1}{\sigma \sqrt{2\pi}} e^{-\frac{1}{2}(\frac{x-\mu}{\sigma})^2}$

The graph is perfectly symmetric around $\mu$. The total area under the bell curve is 1. Mean = Median = Mode.

There is a massive mathematical problem with the Normal PDF formula: It is mathematically impossible to integrate algebraically. To find the area under the bell curve (the probability), you cannot use calculus; you must use numerical approximation.

Since there are infinite Normal distributions (every combination of $\mu$ and $\sigma$), statisticians cannot print infinite tables of integrals. They solved this via Standardization.

⇒9.1 Z-Scores

We mathematically map ANY arbitrary Normal distribution into ONE single, universal distribution called the Standard Normal Distribution (denoted as $Z$).

The Standard Normal $Z$ is defined as having exactly $\mu = 0$ and exactly $\sigma = 1$.

To convert any raw data point $X$ into a Standard $Z$-score, use the Z-transform:
$Z = \frac{X - \mu}{\sigma}$

The Z-score tells you exactly how many standard deviations the data point is located above or below the mean.

Once you have converted your problem into $Z$-scores, you look up the probability (the pre-calculated integral area) in a standard textbook Z-Table.

⇒10.1 The 68-95-99.7 Empirical Rule

For any normal distribution, the Z-Table guarantees these strict boundaries:

• $P(-1 \le Z \le 1) \approx 68.27\%$: Roughly 68% of all data falls within 1 standard deviation of the mean.
• $P(-2 \le Z \le 2) \approx 95.45\%$: 95% of data falls within 2 standard deviations.
• $P(-3 \le Z \le 3) \approx 99.73\%$: 99.7% of data falls within 3 standard deviations. Anything beyond Z=3 is a highly statistically significant outlier.

Example: A class has average test score $\mu = 70$, $\sigma = 10$. What is the probability a student scored $> 85$?
1. Find Z: $Z = (85 - 70)/10 = 1.5$.
2. We want $P(Z > 1.5)$.
3. Look up $Z=1.5$ in the table. The table usually gives $P(Z < 1.5) = 0.9332$.
4. Use the complement rule: $P(Z > 1.5) = 1 - 0.9332 = 0.0668$ (or 6.68%).

Unit 3 requires you to read word problems, identify the correct distribution based on physical context, and execute the formula.

⇒11.1 University Exam Checklist

• List the four conditions required for a random experiment to follow the Binomial distribution.
• Write the PMF formula for the Binomial distribution. Calculate probability given $n$, $p$, and $x$.
• State the Mean and Variance formulas for the Binomial distribution ($np$ and $npq$).
• Identify scenarios that require the Poisson distribution (events per time/space). Write its PMF formula.
• Explain the mathematical conditions under which the Poisson distribution can be used to approximate the Binomial distribution.
• Write the PDF for the Continuous Uniform distribution over $[a, b]$. Calculate its Mean and Variance.
• Write the PDF and CDF for the Exponential distribution. What parameter ($\lambda$) connects it to the Poisson distribution?
• Explain the "Memoryless Property" of the Exponential distribution mathematically.
• Write the formula to convert any Normal random variable $X$ into a Standard Normal $Z$-score.
• Use the standard Z-Table to calculate complex interval probabilities under the bell curve.