Residual sum of squares


In statistics, the residual sum of squares, also known as the sum of squared residuals or the sum of squared estimate of errors, is the sum of the squares of residuals. It is a measure of the discrepancy between the data and an estimation model. A small RSS indicates a tight fit of the model to the data. It is used as an optimality criterion in parameter selection and model selection.
In general, total sum of squares = explained sum of squares + residual sum of squares. For a proof of this in the multivariate ordinary least squares case, see partitioning in the general OLS model.

One explanatory variable

In a model with a single explanatory variable, RSS is given by:
where yi is the ith value of the variable to be predicted, xi is the ith value of the explanatory variable, and is the predicted value of yi.
In a standard linear simple regression model,, where a and b are coefficients, y and x are the regressand and the regressor, respectively, and ε is the error term. The sum of squares of residuals is the sum of squares of estimates of εi; that is
where is the estimated value of the constant term and is the estimated value of the slope coefficient b.

Matrix expression for the OLS residual sum of squares

The general regression model with observations and explanators, the first of which is a constant unit vector whose coefficient is the regression intercept, is
where is an n × 1 vector of dependent variable observations, each column of the n × k matrix is a vector of observations on one of the k explanators, is a k × 1 vector of true coefficients, and is an n× 1 vector of the true underlying errors. The ordinary least squares estimator for is
The residual vector = ; so the residual sum of squares is:
. In full:
where is the hat matrix, or the projection matrix in linear regression.

Relation with Pearson's product-moment correlation

The least-squares regression line is given by
where and, where and
Therefore,
where
The Pearson product-moment correlation is given by therefore,