Matrices with an eigenvalue of modulus one

Closed. Let $A_k\in Z$, $A_k\to A$. Choose eigenvalues $\lambda_k\in\mathbb{C}$ with $|\lambda_k|=1$ and $\det(A_k-\lambda_kI)=0$. The unit circle is compact, so a subsequence $\lambda_{k_j}\to\lambda$ with $|\lambda|=1$. The map $(M,\mu)\mapsto\det(M-\mu I)$ is continuous (polynomial in the entries of $M$ and in $\mu$), so
$$\det(A-\lambda I)=\lim_j\det(A_{k_j}-\lambda_{k_j}I)=0.$$
So $\lambda$ is an eigenvalue of $A$ of modulus $1$: $A\in Z$.

Empty interior. Take any $A\in M_n(\mathbb{R})$; we show every neighbourhood of $A$ contains matrices outside $Z$.
- The eigenvalues of $tA$ are $t\lambda$, where $\lambda$ ranges over the eigenvalues of $A$. So $tA\in Z$ iff $|t||\lambda|=1$ for some eigenvalue $\lambda$ of $A$, i.e. $|t|=1/|\lambda|$ with $\lambda\ne0$.
- $A$ has at most $n$ eigenvalues, so only finitely many $t>0$ satisfy this.
- Hence there are $t>0$ arbitrarily close to $1$ with $tA\notin Z$, and $\|tA-A\|=|t-1|\|A\|\to0$.

So $A$ is not an interior point of $Z$.

Remark. The complement of $Z$ splits into the open sets of matrices with all eigenvalues in the open unit disc ("Schur stable") and with some eigenvalue outside. That is relevant for dynamical systems $x_{k+1}=Ax_k$. Lesson. Scaling $A\mapsto tA$ is a one-parameter family that quickly shows a set is thin.