The rotation angle of a 3D rotation is continuous

Idea. Choose an orthonormal basis adapted to the axis. In that basis $U$ is block diagonal and its trace is $1+2\cos\varphi$.

Step 1: normal form. By the previous problem there is a unit vector $e$ with $Ue=e$. Since $U$ is orthogonal, it maps $e^\perp$ to itself and acts there as an orthogonal map of determinant $\det U/1=1$, i.e. as a rotation $R_\varphi$ of the plane $e^\perp$ (identified with $\mathbb{R}^2$ via an orthonormal basis $(f,g)$). In the orthonormal basis $(e,f,g)$,
$$U\sim\begin{pmatrix}1&0\\0&R_\varphi\end{pmatrix},\qquad R_\varphi=\begin{pmatrix}\cos\varphi&-\sin\varphi\\\sin\varphi&\cos\varphi\end{pmatrix}.$$

Step 2: trace. The trace is invariant under conjugation, so $\operatorname{tr}U=1+2\cos\varphi$, i.e. $\dfrac{\operatorname{tr}U-1}2=\cos\varphi\in[-1,1]$. This shows both that $\arccos$ is defined at this number and that $\arccos(\cos\varphi)=|\varphi|$ (mod $2\pi$, reduced to $[0,\pi]$) is the rotation angle of $U$, up to the sign ambiguity coming from the orientation of the axis.

Step 3: continuity. $U\mapsto\operatorname{tr}U$ is linear, hence continuous; $x\mapsto\arccos x$ is continuous on $[-1,1]$. So $\theta$ is continuous on $SO(3)$ as a composition.

Check. For $U=I$: $\theta=\arccos1=0$. For $U=\operatorname{diag}(1,-1,-1)$ (rotation by $\pi$ about the first axis): $\operatorname{tr}U=-1$, $\theta=\arccos(-1)=\pi$.

Remark. The angle alone does not determine the rotation (there is also the axis), and the axis direction is not a continuous function on $SO(3)$ (it is undefined at $I$); but the angle is, as shown.

Lesson. Conjugation invariants such as the trace give continuous invariants of the conjugacy class.