Exercise 1478Calculus › Multivariable calculus › Optimisation in several variablesL3Added 2 Nov 2026
Least squares as an optimisation problem
#least squares#optimisation#regression
Statement
Let be points. Minimize
over .
Hint
Compute the partial derivatives and , set them to zero, and solve the system.
Show solutionHide solution
Solution
Partial derivatives.
Normal system. Dividing by :
From the second equation, . Substituting into the first:
Hence
This is the slope of the least-squares line.
Global minimum. is a convex quadratic function (sum of squares of affine forms), so the critical point is a global minimum.