The inverse blows up near singular matrices

Idea. A matrix very close to a singular one is very close to the singular set, and the inverse's norm is the reciprocal of that distance.

Step 1: the distance is small. Since $A_0$ is singular, $\operatorname{dist}(A,\text{singular})\le\|A-A_0\|$.

Step 2: apply the formula. By the previous problem,
$$\|A^{-1}\|=\frac1{\operatorname{dist}(A,\text{singular})}\ge\frac1{\|A-A_0\|}\xrightarrow[A\to A_0]{}+\infty.$$

Consequence. Inversion $A\mapsto A^{-1}$ is continuous on $GL_n$ but it cannot be extended continuously to any singular matrix, and it is unbounded near the boundary of $GL_n$. In particular $\{A\in GL_n:\|A^{-1}\|\le M\}$ is closed in $M_n$ (not just in $GL_n$): if $A_k$ lie in it and $A_k\to A_0$, then $\operatorname{dist}(A_k,\text{singular})\ge1/M$ and the distance function is $1$-Lipschitz, so $\operatorname{dist}(A_0,\text{singular})\ge1/M>0$; thus $A_0$ is invertible and $\|A_0^{-1}\|\le M$.

Example. $A_\varepsilon=\operatorname{diag}(1,\varepsilon)\to A_0=\operatorname{diag}(1,0)$ and $\|A_\varepsilon^{-1}\|=1/\varepsilon$, which matches the bound $\ge1/\|A_\varepsilon-A_0\|=1/\varepsilon$ exactly.

Remark. The numerical consequence is the ill-conditioning of linear systems $Ax=b$ for nearly singular $A$: a tiny perturbation of the data produces a perturbation of the solution of size up to $\|A^{-1}\|$ times larger.