Random variables and expectation notes — Unit 2

Sample spaces in probability are often sets of abstract objects like {Heads, Tails} or {Red, Blue, Green}. Mathematics, however, requires numbers.

⇒1.1 The Definition

A Random Variable (RV) is not a variable at all. It is a strict mathematical function.

A Random Variable $X$ assigns a single real number to every possible outcome in the sample space $S$.
$X: S \rightarrow \mathbb{R}$

Example: Toss two coins. $S = \{HH, HT, TH, TT\}$.
Let $X$ be the function that counts the number of Heads.
$X(HH) = 2$
$X(HT) = 1$
$X(TH) = 1$
$X(TT) = 0$

Now, instead of talking about abstract sets of coins, we can ask questions mathematically: "What is $P(X = 1)$?" By mapping outcomes to numbers, we unlock Calculus.

Random variables are split into two completely separate mathematical universes, requiring different mathematical tools.

⇒2.1 Discrete Random Variables

A random variable is Discrete if it can only take on a countable number of distinct values. These are usually integers.

Examples:
- The number of defective laptops in a shipment (0, 1, 2, 3...).
- The number of network packets dropped per minute.
Discrete math uses Summations ($\sum$).

⇒2.2 Continuous Random Variables

A random variable is Continuous if it can take on ANY infinitely precise real number within an interval. It represents physical measurements.

Examples:
- The exact temperature of a CPU (e.g., $72.158349...^\circ$C).
- The exact time it takes to download a file.
Because there are infinite fractions between 1 and 2, continuous math requires Integration ($\int$).

For a Discrete Random Variable $X$, the Probability Mass Function $p(x)$ gives the exact probability that $X$ will take a specific value $x$.

$p(x) = P(X = x)$

⇒3.1 Properties of a Valid PMF

To be mathematically valid, a PMF must strictly satisfy two conditions:

• 1. Non-negativity: $p(x) \ge 0$ for all $x$.
• 2. Sums to 1: The sum of probabilities for all possible values of $x$ must equal exactly 1.
  $\sum p(x) = 1$

Example Check: Given the function $p(x) = cx$ for $x = 1, 2, 3$. Find constant $c$ to make this a valid PMF.
We enforce the sum condition: $\sum p(x) = p(1) + p(2) + p(3) = 1$.
$c(1) + c(2) + c(3) = 1$
$6c = 1$
Therefore, $c = 1/6$.

For a Continuous Random Variable, things change drastically. The probability of hitting an infinitely precise, exact decimal number (e.g., $P(X = 72.158349)$) is mathematically Zero.

Instead of probabilities at exact points, we use a Probability Density Function $f(x)$ to calculate probabilities over an Interval (an area under a curve).

⇒4.1 Area under the Curve

The probability that $X$ falls between $a$ and $b$ is the definite integral of the PDF from $a$ to $b$:

$P(a \le X \le b) = \int_{a}^{b} f(x) dx$

⇒4.2 Properties of a Valid PDF

• 1. Non-negativity: $f(x) \ge 0$ for all real $x$. (The curve must never dip below the x-axis).
• 2. Total Area is 1: The integral across the entire real number line must equal exactly 1.
  $\int_{-\infty}^{\infty} f(x) dx = 1$

*(Note: The value of $f(x)$ itself is a density, NOT a probability. It is possible for $f(x) > 1$, as long as the total area under the curve remains 1).*

The CDF, denoted as $F(x)$, is a universal function that applies to BOTH discrete and continuous variables. It acts as a running total.

⇒5.1 The Definition

The CDF specifies the probability that the random variable $X$ will take a value less than or equal to a specific number $x$.

$F(x) = P(X \le x)$

For Discrete: $F(x) = \sum_{t \le x} p(t)$ (A step function that jumps up at each value).

For Continuous: $F(x) = \int_{-\infty}^{x} f(t) dt$ (A smooth, non-decreasing curve).

⇒5.2 Using the CDF

The CDF makes finding interval probabilities extremely easy via simple subtraction:
$P(a < X \le b) = F(b) - F(a)$

By the Fundamental Theorem of Calculus, the derivative of the continuous CDF gives you back the PDF:
$f(x) = \frac{d}{dx} F(x)$

The Expected Value, denoted as $E[X]$ or $\mu$, is the "center of mass" of a random variable. It represents the long-term, theoretical average value you would get if you repeated the random experiment millions of times.

⇒6.1 The Formulas

You calculate it by multiplying every possible value by its probability (weight) and summing them.

Discrete: $\mu = E[X] = \sum_{x} x \cdot p(x)$

Continuous: $\mu = E[X] = \int_{-\infty}^{\infty} x \cdot f(x) dx$

⇒6.2 Expectation of a Function

If you apply a function $g()$ to a random variable, the expectation changes:

$E[g(X)] = \sum_{x} g(x) \cdot p(x)$ (Discrete)

Example: A casino game pays you $X^2$ dollars, where X is the roll of a die.
E[X^2] = (1^2)(1/6) + (2^2)(1/6) + ... + (6^2)(1/6) = 91/6 \approx \15.16$.
This is the fair price to play the game.

The Expected Value operator $E[\cdot]$ is a mathematically Linear Operator. This makes algebraic manipulation extremely easy.

⇒7.1 Linearity Rules

Let $a$ and $b$ be numeric constants, and let $X$ and $Y$ be random variables.

• $E[c] = c$: The expected value of a constant is just the constant.
• $E[aX] = a \cdot E[X]$: Constants can be pulled outside the expectation.
• $E[aX + b] = a \cdot E[X] + b$: Linearity applies to shifts.
• $E[X + Y] = E[X] + E[Y]$: This is ALWAYS true, absolutely regardless of whether $X$ and $Y$ are independent or heavily dependent.

⇒7.2 Multiplication Rule

$E[XY] = E[X] \cdot E[Y]$

CRITICAL WARNING: This multiplication rule is ONLY true if $X$ and $Y$ are mathematically Independent. If they are dependent, this equation fails.

Expectation only tells us the center. It tells us nothing about how widely the data is spread out. A stock that fluctuates between $49 and$51 has the same mean ($50) as a stock that violently swings between$10 and $90.

Variance ($\sigma^2$) measures this spread. It is the average squared distance from the mean.

⇒8.1 The Formula

Definition: $Var(X) = E[(X - \mu)^2]$

However, this is difficult to compute. The universally preferred computational formula is derived via algebra:

$Var(X) = E[X^2] - (E[X])^2$
(The expected value of the squares, minus the square of the expected value).

⇒8.2 Standard Deviation ($\sigma$)

Because variance is squared, its units are physically meaningless (e.g., "squared dollars"). Standard Deviation brings the measurement back to the original units by taking the square root.

$\sigma = \sqrt{Var(X)}$

Unlike Expectation, the Variance operator $Var(\cdot)$ is NOT linear. It behaves very differently with constants.

⇒9.1 Variance Rules

• $Var(c) = 0$: A constant never varies. Its spread is zero.
• $Var(X + b) = Var(X)$: Shifting the entire graph to the right by $b$ units changes the mean, but it does NOT change how spread out the data is.
• $Var(aX) = a^2 \cdot Var(X)$: CRITICAL RULE. If you multiply a random variable by a constant $a$, its variance scales by $a$ squared.
• $Var(aX + b) = a^2 \cdot Var(X)$: Combining the two rules above.

⇒9.2 Adding Variances

$Var(X + Y) = Var(X) + Var(Y)$

CRITICAL WARNING: This equation is ONLY true if $X$ and $Y$ are mathematically Independent. (If they are dependent, you must add the Covariance term).

Mean and Variance are just the first two "Moments" of a distribution. There are infinite moments that describe skewness, kurtosis (peakiness), and the entire mathematical shape of the curve.

⇒10.1 Definitions of Moments

• $n$-th Moment about the Origin: $\mu_n' = E[X^n]$
  (Note: $E[X^1]$ is the Mean).
• $n$-th Central Moment (about the Mean): $\mu_n = E[(X - \mu)^n]$
  (Note: $E[(X - \mu)^2]$ is the Variance).

⇒10.2 The Moment Generating Function (MGF)

The MGF, denoted as $M_X(t)$, is a single, massive algebraic function that encodes absolutely every moment of the distribution.

Definition: $M_X(t) = E[e^{tX}]$

To extract the $n$-th moment $E[X^n]$ from the MGF, you simply calculate the $n$-th derivative of the MGF with respect to $t$, and then evaluate that derivative at $t=0$.

$E[X^n] = M_X^{(n)}(0)$

This beautiful property completely eliminates the need to calculate complex summations or integrals when finding higher-order moments.

Unit 2 bridges probability and calculus. You must be comfortable with both discrete summations and continuous integrals.

⇒11.1 University Exam Checklist

• Define a Random Variable mathematically.
• Differentiate between Discrete and Continuous random variables, giving two examples of each.
• State the two mathematical properties required for a valid PMF ($p(x)$) and a valid PDF ($f(x)$).
• Given a PDF $f(x) = cx^2$ over $[0, 2]$, use integration to solve for the constant $c$, and then calculate $P(X > 1)$.
• Define the Cumulative Distribution Function (CDF). Calculate the CDF by integrating a given PDF.
• Write the formulas for Expected Value $E[X]$ for both discrete and continuous cases.
• State the Linearity of Expectation rules. Calculate $E[3X + 5]$.
• Write the computational formula for Variance: $Var(X) = E[X^2] - (E[X])^2$.
• State the variance scaling rule. Calculate $Var(-2X + 7)$ given $Var(X) = 4$.
• Define the Moment Generating Function (MGF). Use it to find the Mean and Variance by taking the first and second derivatives.