A stochastic matrix has a stationary distribution
Statement
Let be row-stochastic (, ). Show that there is a probability row vector (that is , ) with .
Hint
Apply repeatedly to a probability vector, average, and use compactness of the simplex.
Show solutionHide solution
Solution
The simplex is invariant. Let , a compact convex set. If then and , so .
Cesàro averages. Let and
(an average of points of the convex set ). Then
Since have -norm , .
Compactness. is compact, so a subsequence converges to some . By continuity of ,
Example. : does not converge, but the averages , which is stationary.
Remark. This is the Markov-Kakutani / Brouwer-type argument. The vector is generally not unique (e.g. ) and need not converge, which is why Cesàro averaging is needed; under the positivity assumption of the next problem, uniqueness and convergence hold.