Maths WorkshopMaths problems with full solutions
Exercise 1397Topology › Topology of Mn(ℝ) › Norms and metric structureL3Added 7 Nov 2026

The operator norm via the largest eigenvalue of Aáµ€A

#spectral norm#singular values#Rayleigh quotient

Statement

Show that ∥A∥2\|A\|^2 is the largest eigenvalue of ATAA^TA, and that ∥A∥F2=∑iσi2\|A\|_F^2=\sum_i\sigma_i^2, where σ1≥⋯≥σn≥0\sigma_1\ge\dots\ge\sigma_n\ge0 are the singular values of AA (the square roots of the eigenvalues of ATAA^TA).

Hint

Write ∣Ax∣2=xT(ATA)x|Ax|^2=x^T(A^TA)x and note that ATAA^TA is symmetric positive semi-definite.

Diagonalise ATAA^TA in an orthonormal basis to compute the maximum of xTATAxx^TA^TAx over unit vectors.

Show solutionHide solution

Solution

Idea. The quantity ∣Ax∣2|Ax|^2 is a quadratic form in xx with matrix S=ATAS=A^TA; the maximum of a quadratic form on the unit sphere is the largest eigenvalue of its (symmetric) matrix.

Step 1: S=ATAS=A^TA is symmetric PSD. ST=AT(AT)T=SS^T=A^T(A^T)^T=S, and xTSx=∣Ax∣2≥0x^TSx=|Ax|^2\ge0. So the spectral theorem gives an orthonormal eigenbasis u1,…,unu_1,\dots,u_n with eigenvalues μ1≥⋯≥μn≥0\mu_1\ge\dots\ge\mu_n\ge0 (non-negative since μi=uiTSui=∣Aui∣2\mu_i=u_i^TSu_i=|Au_i|^2).

Step 2: maximum of the quadratic form. Write x=∑ciuix=\sum c_iu_i with ∑ci2=∣x∣2=1\sum c_i^2=|x|^2=1. Then
∣Ax∣2=xTSx=∑iμici2≤μ1∑ici2=μ1,|Ax|^2=x^TSx=\sum_i\mu_ic_i^2\le\mu_1\sum_ic_i^2=\mu_1,
with equality for x=u1x=u_1. Hence ∥A∥2=sup⁡∣x∣=1∣Ax∣2=μ1\|A\|^2=\sup_{|x|=1}|Ax|^2=\mu_1, so ∥A∥=σ1=μ1\|A\|=\sigma_1=\sqrt{\mu_1}.

Step 3: Frobenius norm. ∥A∥F2=tr⁡(ATA)=tr⁡(S)\|A\|_F^2=\operatorname{tr}(A^TA)=\operatorname{tr}(S) and the trace of a symmetric matrix is the sum of its eigenvalues. So ∥A∥F2=∑iμi=∑iσi2\|A\|_F^2=\sum_i\mu_i=\sum_i\sigma_i^2.

Remark. Consequently ∥A∥=σ1≤(∑σi2)1/2=∥A∥F≤n σ1=n∥A∥\|A\|=\sigma_1\le\big(\sum\sigma_i^2\big)^{1/2}=\|A\|_F\le\sqrt n\,\sigma_1=\sqrt n\|A\|, which reproves the comparison of the two norms, with equality cases visible: ∥A∥=∥A∥F\|A\|=\|A\|_F iff only one singular value is non-zero (rank ≤1\le1), and ∥A∥F=n∥A∥\|A\|_F=\sqrt n\|A\| iff all singular values are equal (multiples of orthogonal matrices).