Invertibility is stable under small perturbations

Idea. Factor out the invertible matrix $A$, so that what remains is a small perturbation of the identity, for which we have the Neumann series.

Step 1: the factorisation. Let $E=A^{-1}(A-B)$. Then
$$A(I-E)=A-AA^{-1}(A-B)=A-(A-B)=B.$$

Step 2: $E$ is small. By submultiplicativity,
$$\|E\|\le\|A^{-1}\|\|A-B\|<\|A^{-1}\|\cdot\frac1{\|A^{-1}\|}=1.$$

Step 3: conclusion. By the Neumann series, $I-E$ is invertible. Hence $B=A(I-E)$ is a product of two invertible matrices, so it is invertible, with $B^{-1}=(I-E)^{-1}A^{-1}$.

Consequence. Every $A\in GL_n$ has a ball $B(A,1/\|A^{-1}\|)\subset GL_n$, so $GL_n(\mathbb{R})$ is open. Moreover this gives the quantitative bound $\|B^{-1}\|\le\dfrac{\|A^{-1}\|}{1-\|A^{-1}\|\|A-B\|}$.

Example. $A=\operatorname{diag}(1,\varepsilon)$ has $1/\|A^{-1}\|=\varepsilon$ and $B=\operatorname{diag}(1,0)$ is at distance $\varepsilon$ from $A$ and singular: the radius $1/\|A^{-1}\|$ cannot be improved (see the next problems).

Remark. This is the first of several ways to see that $GL_n(\mathbb{R})$ is open; another is that $\det$ is continuous and $GL_n=\det^{-1}(\mathbb{R}\setminus\{0\})$. The Neumann argument is more quantitative and works in any Banach algebra, including infinite-dimensional ones, where $\det$ is not available.