Discrete Random Variables

PMF, expectation, variance, and the key discrete distributions

1. Random Variables

A random variable \(X\) is a function \(X: \Omega \to \mathbb{R}\) that assigns a real number to each outcome. It converts events into numbers, making probability calculations algebraic.

2. Probability Mass Function

PMF

\(p_X(x) = \mathbb{P}(X = x) = \mathbb{P}(\{\omega \in \Omega : X(\omega) = x\})\)

Valid PMFs satisfy: \(p_X(x) \geq 0\) and \(\sum_x p_X(x) = 1\).

3. Expectation

\[\mathbb{E}[X] = \sum_x x\, p_X(x)\]

The expected value is the probability-weighted average. For a function \(g(X)\): \(\mathbb{E}[g(X)] = \sum_x g(x)\, p_X(x)\).

Linearity of Expectation

\(\mathbb{E}[aX + b] = a\,\mathbb{E}[X] + b\). Holds even for dependent random variables.

4. Variance

\[\text{Var}(X) = \mathbb{E}\!\left[(X - \mathbb{E}[X])^2\right] = \mathbb{E}[X^2] - (\mathbb{E}[X])^2\]

5. Key Discrete Distributions

Bernoulli

\(X \sim \text{Ber}(p)\): single trial, success with probability \(p\).

\[p_X(k) = \begin{cases} p & k=1 \\ 1-p & k=0 \end{cases}, \quad \mathbb{E}[X]=p, \quad \text{Var}(X)=p(1-p)\]

Binomial

\(X \sim \text{Bin}(n,p)\): number of successes in \(n\) independent Bernoulli(\(p\)) trials.

\[p_X(k) = \binom{n}{k}p^k(1-p)^{n-k}, \quad k=0,1,\ldots,n\] \[\mathbb{E}[X]=np, \quad \text{Var}(X)=np(1-p)\]
0 1 2 3 4 k Bin(10, 0.3) — peaked near np = 3
Figure 1 — Binomial PMF. The distribution is peaked near the mean np.

Geometric

\(X \sim \text{Geo}(p)\): number of failures before first success. \(X \in \{0, 1, 2, \ldots\}\).

\[p_X(k) = (1-p)^k p, \quad \mathbb{E}[X] = \frac{1-p}{p}, \quad \text{Var}(X) = \frac{1-p}{p^2}\]

Memorylessness: \(\mathbb{P}(X > s+t \mid X > s) = \mathbb{P}(X > t)\). Past failures carry no information about future trials.

Discrete Uniform

\(X \sim \text{Unif}(a, b)\): equally likely values \(\{a, a+1, \ldots, b\}\).

\[\mathbb{E}[X] = \frac{a+b}{2}, \quad \text{Var}(X) = \frac{(b-a)(b-a+2)}{12}\]

6. Conditional PMF and Total Expectation

The conditional PMF of \(X\) given event \(A\): \(p_{X|A}(x) = \mathbb{P}(X=x \mid A)\).

Total Expectation Theorem

For a partition \(A_1, \ldots, A_n\) of \(\Omega\):

\[\mathbb{E}[X] = \sum_i \mathbb{P}(A_i)\,\mathbb{E}[X \mid A_i]\]

The expected value is a weighted average of conditional expectations — one of the most useful tools for computing expectations of complex processes.

7. Summary Table

DistributionPMF \(p_X(k)\)\(\mathbb{E}[X]\)\(\text{Var}(X)\)
Bernoulli(\(p\))\(p^k(1-p)^{1-k}\)\(p\)\(p(1-p)\)
Binomial(\(n,p\))\(\binom{n}{k}p^k(1-p)^{n-k}\)\(np\)\(np(1-p)\)
Geometric(\(p\))\((1-p)^k p\)\((1-p)/p\)\((1-p)/p^2\)
Discrete Uniform(\(a,b\))\(1/(b-a+1)\)\((a+b)/2\)\((b-a)(b-a+2)/12\)