本科高年级Probability & Statistics计划中
Markov chains
Discrete-time Markov chains: transition matrices, classification of states, stationary distributions, convergence to equilibrium, hitting times and random walks.
math.prob.markov-chains
本主题的课程尚未编写。下面的提纲是课程地图的一部分，尚未经过审校。
前置知识
42 项必需的前置知识（一直追溯到底）
你将学会
- Model processes with transition matrices and compute n-step transition probabilities.
- Classify states as recurrent or transient and find stationary distributions.
- Prove convergence to equilibrium for irreducible aperiodic finite chains.
定义
- A Markov chain is a sequence of random variables in which the conditional distribution of the next state, given the past, depends only on the current state.
- A probability distribution π (as a row vector) is stationary for transition matrix P if πP = π.
- A chain is irreducible if every state can be reached from every other state.
定理
- An irreducible Markov chain on a finite state space has a unique stationary distribution.
- If it is also aperiodic, the distribution at time n converges to the stationary distribution from any starting distribution.
- Pólya: simple random walk is recurrent on ℤ and ℤ² and transient on ℤ³.
常见误解
- Reading 'memoryless' as 'forgets everything'; it means the future depends on the past only through the present.
通向
可选发现
- Mathematical biology Stochastic models of populations and of genetic drift, such as the Wright–Fisher model, are Markov chains.
延伸阅读
- J. R. Norris, Markov Chains (1997)