Maths WorkshopMaths problems with full solutions
Exercise 2026Topology › Topology of Mn(ℝ) › Rank, spectrum and normsM1Added 6 Nov 2026

A stochastic matrix has a stationary distribution

#stochastic matrices#compactness#Cesàro averages#Markov chains

Statement

Let P∈Mn(R)P\in M_n(\mathbb{R}) be row-stochastic (Pij≥0P_{ij}\ge0, ∑jPij=1\sum_jP_{ij}=1). Show that there is a probability row vector π\pi (that is πi≥0\pi_i\ge0, ∑πi=1\sum\pi_i=1) with πP=π\pi P=\pi.

Hint

Apply PP repeatedly to a probability vector, average, and use compactness of the simplex.

Show solutionHide solution

Solution

The simplex is invariant. Let Δ={π∈Rn:πi≥0,∑πi=1}\Delta=\{\pi\in\mathbb{R}^n:\pi_i\ge0,\sum\pi_i=1\}, a compact convex set. If π∈Δ\pi\in\Delta then πP≥0\pi P\ge0 and ∑j(πP)j=∑iπi∑jPij=1\sum_j(\pi P)_j=\sum_i\pi_i\sum_jP_{ij}=1, so πP∈Δ\pi P\in\Delta.

Cesàro averages. Let π0=(1n,…,1n)\pi_0=(\frac1n,\dots,\frac1n) and
mN=1N∑k=0N−1π0Pk∈Δm_N=\frac1N\sum_{k=0}^{N-1}\pi_0P^k\in\Delta
(an average of points of the convex set Δ\Delta). Then
mNP−mN=1N(∑k=1Nπ0Pk−∑k=0N−1π0Pk)=π0PN−π0N.m_NP-m_N=\frac1N\Big(\sum_{k=1}^{N}\pi_0P^k-\sum_{k=0}^{N-1}\pi_0P^k\Big)=\frac{\pi_0P^N-\pi_0}N.
Since π0PN,π0∈Δ\pi_0P^N,\pi_0\in\Delta have 11-norm 11, ∥mNP−mN∥1≤2N→0\|m_NP-m_N\|_1\le\frac2N\to0.

Compactness. Δ\Delta is compact, so a subsequence mNjm_{N_j} converges to some π∈Δ\pi\in\Delta. By continuity of x↦xP−xx\mapsto xP-x,
πP−π=lim⁡j(mNjP−mNj)=0.\pi P-\pi=\lim_j(m_{N_j}P-m_{N_j})=0.

Example. P=(0110)P=\begin{pmatrix}0&1\\1&0\end{pmatrix}: PkP^k does not converge, but the averages mN→(12,12)m_N\to(\frac12,\frac12), which is stationary.

Remark. This is the Markov-Kakutani / Brouwer-type argument. The vector π\pi is generally not unique (e.g. P=IP=I) and π0Pk\pi_0P^k need not converge, which is why Cesàro averaging is needed; under the positivity assumption of the next problem, uniqueness and convergence hold.