Continuous Random Variables

PDF, CDF, expectation, transformations, and convolutions

1. Introduction

A random variable is continuous if it can take any value in an interval. Individual values have probability zero — probability is only meaningful for intervals. The PDF plays the role of the PMF, but it's a density, not a probability.

2. PDF and CDF

Probability Density Function

A random variable \(X\) is continuous with PDF \(f_X(x)\) if:

\[\mathbb{P}(a \leq X \leq b) = \int_a^b f_X(x)\,dx\]

Valid PDFs satisfy: \(f_X(x) \geq 0\) and \(\int_{-\infty}^{\infty} f_X(x)\,dx = 1\).

Cumulative Distribution Function

\[F_X(x) = \mathbb{P}(X \leq x) = \int_{-\infty}^x f_X(t)\,dt\]

The PDF is the derivative of the CDF: \(f_X(x) = F_X'(x)\).

Figure 1 — The PDF (left) gives probability as area under the curve. The CDF (right) is its integral, ranging from 0 to 1.

3. Expectation and Variance

\[\mathbb{E}[X] = \int_{-\infty}^{\infty} x\, f_X(x)\,dx, \qquad \mathbb{E}[g(X)] = \int_{-\infty}^{\infty} g(x)\, f_X(x)\,dx\] \[\text{Var}(X) = \mathbb{E}[X^2] - (\mathbb{E}[X])^2\]

Linearity of expectation holds: \(\mathbb{E}[aX+b] = a\mathbb{E}[X]+b\). Properties of variance: \(\text{Var}(aX+b) = a^2\text{Var}(X)\).

4. One-Variable Transformations

If \(Y = g(X)\) where \(g\) is strictly monotone and differentiable:

\[f_Y(y) = f_X(g^{-1}(y))\left|\frac{d}{dy}g^{-1}(y)\right|\]

The absolute value of the derivative (Jacobian) accounts for stretching/compressing of probability mass under the transformation.

5. Two-Variable Transformations

Given the joint PDF \(f_{X,Y}(x,y)\) and a transformation \((U,V) = g(X,Y)\), the joint PDF of \((U,V)\) is:

\[f_{U,V}(u,v) = f_{X,Y}(x,y) \left|\det\frac{\partial(x,y)}{\partial(u,v)}\right|\]

The Jacobian matrix \(\frac{\partial(x,y)}{\partial(u,v)}\) has entries \(\partial x/\partial u, \partial x/\partial v, \partial y/\partial u, \partial y/\partial v\). In 2D: \(|\det J| = |ad-bc|\) for a 2×2 matrix.

6. Convolutions

If \(X\) and \(Y\) are independent continuous random variables, the PDF of \(Z = X + Y\) is the convolution of their PDFs:

Convolution

\[f_{X+Y}(t) = \int_{-\infty}^{\infty} f_X(x)\, f_Y(t-x)\, dx\]

Example: If \(X, Y \sim \mathcal{N}(0,1)\) i.i.d., then \(f_{X+Y}(t) = \int_{-\infty}^{\infty} \frac{1}{\sqrt{2\pi}}e^{-x^2/2} \cdot \frac{1}{\sqrt{2\pi}}e^{-(t-x)^2/2}\,dx\). Completing the square shows \(X+Y \sim \mathcal{N}(0,2)\).

7. Important CDFs

Standard Normal \(\Phi\): \(\Phi(x) = \mathbb{P}(Z \leq x)\) for \(Z \sim \mathcal{N}(0,1)\). No closed form — values tabulated or computed numerically.
Exponential(\(\lambda\)): \(F(x) = 1 - e^{-\lambda x}\) for \(x \geq 0\).
Uniform(0,1): \(F(x) = x\) for \(x \in (0,1)\).