The Neumann series

Idea. This is the geometric series $\frac1{1-a}=\sum a^k$ for $|a|<1$, transported to matrices; the matrix norm plays the role of $|a|$.

Step 1: absolute convergence. By submultiplicativity, $\|A^k\|\le\|A\|^k$ so $\sum_k\|A^k\|\le\sum_k\|A\|^k=\frac1{1-\|A\|}<\infty$. Since $M_n(\mathbb{R})$ is complete, an absolutely convergent series converges: let $S=\sum_{k\ge0}A^k$.

Step 2: $S$ is the inverse. Let $S_N=\sum_{k=0}^NA^k$. Then
$$(I-A)S_N=S_N(I-A)=I-A^{N+1}.$$
As $N\to\infty$, $\|A^{N+1}\|\le\|A\|^{N+1}\to0$, and $S_N\to S$. Multiplication is continuous, so $(I-A)S=S(I-A)=I$. Hence $I-A$ is invertible with inverse $S$.

Step 3: the bound. By the triangle inequality for series,
$$\|S\|\le\sum_{k\ge0}\|A^k\|\le\sum_{k\ge0}\|A\|^k=\frac1{1-\|A\|}.$$

Example. For $A=\tfrac12\begin{pmatrix}0&1\\0&0\end{pmatrix}$, $A^2=0$, so $(I-A)^{-1}=I+A=\begin{pmatrix}1&1/2\\0&1\end{pmatrix}$ and indeed $\|I+A\|\approx1.28\le\frac1{1-1/2}=2$.

Remark. The result uses only that the norm is submultiplicative and complete (a *Banach algebra*), so it is valid for any induced norm and for the Frobenius norm. It also shows that the open unit ball around $I$ is contained in $GL_n(\mathbb{R})$; the next problem generalises this to a ball around any invertible matrix.