Probability theory

Builds on

Required prerequisites, all the way down: 40

You will learn to

• Compute expectations and variances for the binomial, geometric, Poisson, uniform, exponential and normal distributions.
• Use linearity of expectation and indicator variables, including for dependent events.
• Prove the weak law of large numbers via Chebyshev's inequality, state the central limit theorem, and use the normal approximation.

Definition

• A random variable is a numerical quantity determined by the outcome of a random experiment.
• E[X] = Σ x P(X = x) in the discrete case and ∫ x f(x) dx in the continuous case, when the sum or integral converges absolutely.
• Var(X) = E[(X − E[X])²].

Theorem

• Linearity of expectation holds for all random variables with finite expectation, whether or not they are independent.
• Chebyshev: P(|X − μ| ≥ kσ) ≤ 1/k² for k > 0.
• Central limit theorem: for independent identically distributed variables with finite positive variance, the standardised sums converge in distribution to the standard normal distribution.
• Var(X + Y) = Var(X) + Var(Y) when X and Y are uncorrelated, in particular when they are independent.

Common misconceptions

• Assuming E[XY] = E[X]E[Y] for all random variables; it requires X and Y to be uncorrelated.
• Believing the central limit theorem says the data themselves become normally distributed.

Leads to

Optional discoveries

• Measure theory and integration Making 'random variable' fully rigorous, especially for continuous distributions, requires measure theory.
• ← Sampling and statistical reasoning

Learning pathways

• Applied mathematics

Further reading

• Geoffrey Grimmett and David Stirzaker, Probability and Random Processes
• William Feller, An Introduction to Probability Theory and Its Applications, Volume I (1950)